
What happens when your startup gets debunked by science? - asanwal
http://www.fastcompany.com/3008066/tech-forecast/what-happens-when-your-startup-gets-debunked-science
======
auctiontheory
Most consumer "health" products (running shoes come glaringly to mind) are not
backed by science.

They may need to adjust their marketing to not run afoul of the FTC, but I'm
sure a slight pivot will suffice. They will live or die by how well they
satisfy a (psychological) consumer need, not whether the product "works." Do
any of the exercise products advertised at 3am ... "work"? It doesn't matter.

[Note: this comment is intended as an unsnarky statement of fact.]

~~~
tel
Depends on your customer. Many of these consumer health companies are also at
least at heart gunning for reimbursement and payor customers. You're going to
need to do more than a slight pivot to keep on sight with that strategy.

------
victoro
The Atkins diet (and almost every other diet I can think of) has been
debunked, un-debunked, debated, berated, and praised almost from its inception
half a century ago. It has not stopped the companies that promote the diet,
produce the food, and publish the cookbooks, from making money hand over fist.

With any physiological system of sufficient complexity, whether it be human
digestive/metabolic systems or the neurological systems responsible for memory
and intelligence, there will be opportunities for profit as long as
improvement of that system is desirable and a small fraction of the population
believes they have the chance to improve it and the cost is reasonable. Plus,
there will likely always be "success stories" to cite simply due to the
placebo effect.

We have a massive alternative "medicine" industry that supports not only the
people in it, but also, ironically, the people producing the constant flow of
content criticizing alternative medicine. Lumosity will be fine.

------
thaumaturgy
Homeopathy. Next question.

People are remarkably good at ignoring things they don't like to believe.
Other people are good at exploiting those people.

~~~
alex_doom
But, but, you cure cancer with sugar pills!

~~~
claudius
Tim Cook would need another job if that worked.

------
xijuan
When I first saw the title of the article, I thought about Lumosity. And it is
about Lumosity!!! When I first saw its ads on youtube, I was like "Are you
joking?... Pseudo-science trying to rip money off ignorant people?" The
problem with Lumosity is that their claims are outrageous and exaggerated..

------
DanBC
Anyone who's read Ben Goldacre's _Bad Science_ book will realise that you can
sell anything and the science of whether it works or not is irrelevant.

See, for a particularly bad example, "Brain Gym". This is a set of nonsense
exercises that children are forced to do to increase mental alertness. It has
some bizarre statements ('processed food does not contain water'; breathing
through the nose gets oxygen to the brain faster, etc).

This nonsense is depressingly common in UK schools.

------
breadbox
Unfortunately the all-too-brief article doesn't really answer this question
(unless the answer is the usual "Deny everything and carry on".)

~~~
lutusp
> Unfortunately the all-too-brief article doesn't really answer this question
> ...

I think that was intentional, and that the answer is either "no one knows" or
"caveat emptor". Or both.

------
richardjordan
I hope this means NPR will stop pimping this garbage during their pledge
drives (a $450 value) and lending what little credibility they have left to
junk science. It is genuinely something that puts me off pledging... It just
reminds me of their general decline in standards and terrible science
reportage... And funnily enough since I stopped pledging I stopped listening
pretty much, just catch it occasionally at the parents, but i digress.

I suppose the broader point I'm trying to make is they won't have to change as
it is hardly like there's any scientifically literate mainstream media to hold
them to account, and they've bought off the public media alternative.

~~~
rdl
They definitely have gone down in quality from 1998 to now. What caused this?
Trying to be more commercial?

~~~
lutusp
> They definitely have gone down in quality from 1998 to now. What caused
> this?

A gradual decline in public interest in public television and radio. That, in
turn, has been caused by the increasing diversity of offerings on cable and
satellite TV and radio, compared to 30 years ago. There was a time when the
only way to see anything resembling scientific or educational programming was
to tune into public TV or radio. Not any more -- there are dozens of channels
offering this kind of content now, of widely varying quality, but certainly
competing with public TV.

As the number of advertisements and pledge drives on public TV has increased,
the public has begun to see public TV as just another commercial-supported TV
offering, nothing special. And guess what? They're right -- there's no
difference.

~~~
illuminate
"A gradual decline in public interest in public television and radio"

Don't forget the conservatives in management who want to "diversity" the
programming and defunding due to the "socialist" content.

~~~
lutusp
> Don't forget the conservatives in management who want to "diversity" the
> programming and defunding due to the "socialist" content.

Yes, very true. Management reacts to a conservative social trend by avoiding
content that might be looked on as too liberal, this offends liberals, which
causes a further drop in audience numbers, and a death spiral begins.

This speaks to a general rule in television programming -- everybody want to
have the "least offensive program". The least offensive program is the program
that is so bland that it gets the largest audience simply because it offended
the smallest number of viewers.

------
stevewillows
While it may not make your mind better, you can get positive results with
problem solving - - like doing a crossword each day.

------
tel
Why don't these companies have a scientific strategy running alongside your
product/customer/marketing/financial strategies? From a company making bold
claims like this I want to see a continual string of increasingly
sophisticated trials, each one giving increasingly robust indications of
success. Luminosity should be fighting alongside those meta-analyses and
investors should be demanding proof points in terms of p-values. Fail
(scientifically) fast.

~~~
gwern
The companies _do_ have scientific strategies; Posit Science, for example, has
a bunch of associated studies. They're an integral part of the marketing.

> Luminosity should be fighting alongside those meta-analyses and investors
> should be demanding proof points in terms of p-values.

How many $$$ is a p-value or meta-analysis worth?

> Fail (scientifically) fast.

That doesn't sound very profitable. How does that put more $$$ into the
founders' pockets?

~~~
hmahncke
Thanks gwern - I work at Posit Science, and that's been our approach since our
beginning. When we got started, we ran an initial randomized controlled
trial[1] which showed our cognitive training program improved memory, then ran
a larger study[2], and then ran the largest study that had ever been done with
commercially available brain training program[3]. Other research groups have
gone on to show benefits in chemobrain[4], HIV associated neurocognitive
impairment[5], and schizophrenia[6].

Different cognitive training programs are different - just like different
anti-cancer molecules are different. Some are going to work, and some aren't.

[1] <http://www.ncbi.nlm.nih.gov/pubmed/17046669> [2]
<http://www.ncbi.nlm.nih.gov/pubmed/16888038> [3]
<http://www.ncbi.nlm.nih.gov/pubmed/19220558> [4]
<http://www.ncbi.nlm.nih.gov/pubmed/22918524> [5]
<http://www.ncbi.nlm.nih.gov/pubmed/22579081> [6]
<http://www.ncbi.nlm.nih.gov/pubmed/19448187>

~~~
lutusp
> I work at Posit Science, and that's been our approach since our beginning.
> When we got started, we ran an initial randomized controlled trial[1] which
> showed our cognitive training program improved memory, then ran a larger
> study[2], and then ran the largest study that had ever been done with
> commercially available brain training program[3].

Problems obvious to a scientist:

1\. Poor controls. Both the experimental subjects and the researchers know, or
can easily establish, which are experimental subjects and which are controls.
A built-in bias.

2\. The study was conducted by the company that stood to benefit from a
positive result. An obvious procedural flaw.

3\. Inadequate precautions against counting a placebo effect (an imaginary
benefit) or Hawthorne effect (any attention paid to the subjects improves
their state) as a meaningful outcomes correlated with the specific treatment
rather then any treatment.

But I am sure you already know this -- it presents the usual litany of
legitimate objections to this kind of study.

> Different cognitive training programs are different - just like different
> anti-cancer molecules are different. Some are going to work, and some
> aren't.

That's true but irrelevant. The only meaningful question is which methods work
_in principle_. There are no methods that consistently work, and no one can
_explain_ the outcomes, only _describe_ them. Which means it's not science.

~~~
hmahncke
You're right - those are the usual litany of legitimate objections to this
kind of study - and they are good ones. So we addressed them:

1\. The trials are blinded, with more advanced studies using active controls
(e.g., ordinary computer games[6], adult education[1,2,3]). Study staff
administering cognitive assessments are always blind, and participants are
blind in studies with active controls. The human consent forms describe the
studies as comparing two forms of cognitive training. This is a good blind -
in one study the wife of a control participant called my to tell me her
husband loved the cognitive training because the art history education made
him go to the museum more with her :-)

2\. Study [3] in my post was conducted by Mayo and USC (and funded by Posit -
the money has to come from somewhere), studies [4],[5],[6] were conducted
entirely by independent academic investigators.

3\. Active control groups control for the placebo/Hawthorne effect.

As I see it, the only meaningful question is which methods work in practice -
and the studies show us that. If what you mean is do we know what the
mechanism of action is, we are speeding up information processing flow through
the brain and making it more efficient - speaking loosely, we're improving
signal/noise ratios. Reference [1] in my post has an extensive conceptual
background to this approach. And we've seen that with studies of EEG/working
memory [7]

[7] <http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904363/>

~~~
gwern
> You're right - those are the usual litany of legitimate objections to this
> kind of study - and they are good ones. So we addressed them:

But you guys were apparently unable to address Robert's questions:
[http://blog.sethroberts.net/2013/03/09/a-few-answers-from-
po...](http://blog.sethroberts.net/2013/03/09/a-few-answers-from-posit-
science/)

~~~
hmahncke
Seth Roberts sent me a list of questions and I answered all of them. He then
selectively quoted my answers. It's his blog, and there's nothing I can do
about it.

------
vital
I wonder, who is this science?

~~~
illuminate
The whole of objective reality.

~~~
illuminate
Well, that is what "science" is. The measure and best possible understanding
of the reality in which all beings live in.

------
ExpiredLink
The irony in this case is that 'brain science' itself is mostly pseudo
science.

~~~
illuminate
Pop applications of it aren't neurobiology, sure.

------
rtkwe
The font on the mobile site on Android is shockingly fuzzy and hard to read.
Anyone else seeing that?

