
Study finds Lumosity users show no improvements in cognitive performance - sjcsjc
https://arstechnica.co.uk/science/2017/07/lumosity-doesnt-improve-brain-function/
======
jmcgough
I spent three years of undergrad working on neurogenesis research - was
actually able to show that voluntary exercise could help ameliorate memory
deficits caused by a model of a neurodegenerative disease we'd induced in
rats.

A Luminosity recruiter contacted me after I left, and I asked for evidence
that their research is actually helpful. They were very kind and passed along
a number of studies that they'd done. I wasn't super impressed. Among other
things, they were marketing their games to the general public, not the
populations represented in their small studies. Quite honestly, aerobic
exercise causes a huge upregulation of neurogenesis (and many other positive
things), so people would be better off going for a light jog 2-3 times a week.
It felt like they were taking advantage of families desperate to help a family
member with dementia and willing to try anything.

~~~
rayalez
Can you share some more recommendations or advice on how one can increase
neuroplasticity/neurogenesis?

Aside from the diet/exercise/socialization/meditation, what should people be
aware of?

Are there some effective supplements/nootropics, or some promising research to
look into?

~~~
jmcgough
I left academia about five years ago, and don't want to create the impression
that I'm some brilliant academic researcher (I spent three years doing
rat/staining/counting work in a lab and wrote an honors thesis on the topic).
I try to keep up on current research when I can though, the work pretty much
consumed my life and I kind of miss it sometimes.

Regular aerobic exercise (stuff that elevates your heart rate) is still the
best thing we know of. It's good for upregulation of neurogenesis and BDNF.

There are dozens of papers out there showing that a certain
treatment/activity/drug increases neurogenesis. It's funny, I was talking
about neuroscience to a lyft driver a way back who was curious about
neurogenesis/neuroplasticity, and I told him that exercise is your best bet,
but we also had some good results treating our brain-damaged rats with an
acetylcholinesterase inhibitor. He shrugged off the exercise bit but of course
asked me to spell out ACh inhibitor for him (which I really wouldn't recommend
any random person off the street taking).

I experimented for a bit with nootropics after I graduated. Gwern's posts are
really interesting if you haven't read him
([https://www.gwern.net/Nootropics](https://www.gwern.net/Nootropics)). A
number of people have found success with a basic choline and -racetam stack.
As much as tech blogs love blogging about how we're all hooked on nootropics
in the valley, it's unlikely that taking some pills in the morning is going to
dramatically change your life (though it might nudge it in the right
direction). As one of the other replies said, lifestyle changes are still the
most effective approach.

------
closed
Slightly off-topic, but the pushback against brain training owes a huge debt
of thanks to Randy Engle (and others), for loudly questioning the methods and
rational of the early "become smarter by the magic of dual n-back" research.

Here's an article where he compares it to cold fusion:
[http://www.nytimes.com/2012/04/22/magazine/can-you-make-
your...](http://www.nytimes.com/2012/04/22/magazine/can-you-make-yourself-
smarter.html)

~~~
michaelcampbell
Was the claim not "doing n-back helps working memory"? Hardly "become
smarter". Whether or not it works is worthy of discussion, but straw-man-ing a
hyperbolic claim that wasn't made doesn't add to the discussion.

~~~
closed
That was half the claim. The full claim was that it improves fluid
intelligence _by_ improving working memory. The title of Jaeggi 2008 is
"Improving fluid intelligence with training on working memory".

[http://www.pnas.org/content/105/19/6829.abstract](http://www.pnas.org/content/105/19/6829.abstract)

------
basseq
My wife is a PhD in applied cognition (field of psychology), wrote her thesis
on improvements in working memory and attention, and is working on an early-
stage business around better brain coaching[1]. (More of a 1:1 model than a
tech stack.)

She has long stated that Lumosity is crap. For two three main reasons:

1\. The "improvement" you see in playing Lumosity games is due to getting good
at the game, not actually underlying improvements.

2\. The games you do want to pursue need to to be strategic and challenge the
mind. They need to be different, and involve complex scenarios. She likes
_Rise of Nations_ , by way of example. Flash minigames don't cut it (and
neither do crosswords).

3\. Brain games are 20% of a total set of activities that have an impact on
brain health, the other 80% including physical exercise, diet, socialization,
and lack of stress (mindfulness training helps here).

Anyway, fascinating stuff, and glad Lumosity's "bad science" and false claims
are coming to the fore.

[1] [http://brainevolved.com](http://brainevolved.com)

------
codyb
On the other hand, things like exercise and learning a new language have
pretty conclusively shown benefits when it comes to reducing symptoms of
dementia and brain function loss. [0] [1]

If you'd like to improve your memory I suspect you'd be far better off
studying the masters from Aristotle to Dominic O'Brien whose memory palace
techniques seem incontrovertibly proven if you consider the techniques of the
winners of the world memory championships. [2] [3]

Finally, meditation is not causally linked to higher levels of grey matter but
the research seems promising [4]

(I tried to find decent links but being at work can't spend a significant
portion of time on the post, these should be good starting points for anyone
interested).

[0] - [http://www.health.harvard.edu/blog/regular-exercise-
changes-...](http://www.health.harvard.edu/blog/regular-exercise-changes-
brain-improve-memory-thinking-skills-201404097110) [1] -
[https://www.livescience.com/46048-learning-new-language-
brai...](https://www.livescience.com/46048-learning-new-language-brain.html)
(sorry for the ads, and I think it autoplays a video :( ) [2] -
[https://en.wikipedia.org/wiki/Art_of_memory](https://en.wikipedia.org/wiki/Art_of_memory)
[3] -
[https://en.wikipedia.org/wiki/Dominic_O%27Brien](https://en.wikipedia.org/wiki/Dominic_O%27Brien)
[4] -
[https://www.sciencedaily.com/releases/2015/02/150205142951.h...](https://www.sciencedaily.com/releases/2015/02/150205142951.htm)

------
kranner
Title should include "for healthy young adults" as in the study:

"The researchers note limitations in the study, most importantly that they
only looked at young, healthy people. Brain-training games may still prove
useful for aging adults or people suffering from addiction or unhealthy
behaviors, the researchers caution."

~~~
abtinf
"Brain-training games may still prove useful for aging adults or people
suffering from addiction or unhealthy behaviors, the researchers caution.

Given the context of the article, this is an unscientific and immoral
statement.

It is irrational to believe something is true or possible unless you have
evidence for it.

If the researchers have a positive reason to believe brain-training games are
a form of medical treatment for aging or addicted populations, they should
state that. None of this "may" nonsense.

~~~
ncallaway
> It is irrational to believe something is true or possible unless you have
> evidence for it.

This is actually two statements.

> It is irrational to believe something is true unless you have evidence for
> it.

Sure, I'd agree with that.

> It is irrational to believe something is possible unless you have evidence
> for it.

Whoa, there. This seems like it goes...way beyond rationality. I would argue
that you would need strong evidence if you were going to claim something was
impossible.

~~~
abtinf
A claim of possibility or impossibility are both claims about the nature of
the universe and both require evidence.

If you have no evidence for a claim, then any statement made regarding that
claim, either in support of or opposed to, is arbitrary and lacks any
intellectual value.

For example, any claim about the (non)existence of god is arbitrary and not
worth even a moment of consideration, because there is no evidence whatever in
support of the claim either way.

~~~
ncallaway
I suppose there are two different claims of possibility that are getting
confused.

There is a hard claim of possibility, that is a claim of the state of nature
(e.g. it is _possible_ for a photon to travel at 0.5c). This hard claim is one
where we definitively say that it is _certain_ that it's a physical
possibility.

Then there is a soft claim of possibility, that is a claim of our state of
knowledge. (e.g. It's _possible_ that Rob went to the store, but I'm not
sure).

These are two different claims. One is a hard claim that should only be made
with evidence. The other is a statement of our uncertainty and an open mind;
that doesn't need any evidence to make.

It's a shame that these two different meanings of these words exist, but such
is the English language. They are saying that their research does not close
the door on that eventuality, and that their state of knowledge on that topic
is uncertain.

~~~
abtinf
If I ask you, "Where did Rob go?" and you, having never even heard of Rob,
respond "he might be out cheating on his wife," it would be an arbitrary
statement bordering on malice. Qualifying the statement with "but I'm not
sure," doesn't help you, because even the suggestion of the initial
possibility needs to be based on evidence.

If, instead, you know Rob well and you happen to have a general understanding
of his schedule and needs, you might respond with "It's possible he went to
the store; last night he mentioned he ran out of toilet paper." This is an
evidence-backed claim.

A claim about the state of nature is the same. Your knowledge leads you toward
certain avenues of research, so you might think, "photons have properties x,
y, z, so maybe they can travel at .5c, let me look into it." However, it would
be improper to base your research on whim, like "I think photons are little
gremlins and they run really fast, but can they run slow too?" Such a claim is
not a claim about reality, it is arbitrary.

Claims about the world must be grounded in knowledge, and knowledge must be
based on evidence.

~~~
ncallaway
> photons have properties x, y, z, so maybe they can travel at .5c, let me
> look into it.

No, the claim about what is possible for a photon was much stronger. I am
asserting now that photons _can travel at .5c_ , full stop. I will not look
into it, as I have already determined it is possible. I am not making a
hypothesis about the universe, I am making a claim about how I believe it to
actually work. Anyway, that is an aside.

I still think this just comes down to semantics about what is a claim about
the world and what is a statement about uncertainty.

I interpret them saying something is possible as them making a statement about
their own partial knowledge. I specifically _do not_ interpret it as a claim
about the world. I _do_ interpret it as a claim about their own research and
their own partial knowledge.

Their research does not demonstrate an impossibility. That is the claim they
present. I still believe you are making a semantic interpretation of the word
"possibility" that is different from the intent of the researchers. I think
you are interpreting a stronger claim than the researchers are making.

Here is, for example, the same qualification as reported in a different
article:

> Although, in this study, the researchers found that commercial cognitive
> training alone would not have an influence on one's decision-making process
> or cognitive abilities, they believe that it was still an avenue worthy of
> rigorous investigation.

[https://www.sciencedaily.com/releases/2017/07/170710135410.h...](https://www.sciencedaily.com/releases/2017/07/170710135410.htm)

------
moron4hire
I'm in the middle of planning a startup with a doctor who does research on
therapy methods for traumatic brain injury. She frequently goes on rants about
how much bullshit Lumosity is. So literal appeal to authority, but this
particular authority was doing research at NIH for 17 years. I think she knows
what she's doing.

~~~
notheguyouthink
Are there any that _aren 't_ bullshit?

I'm debating just setting up a spaced repetition app with my own custom
knowledge base. At least that way I may not be getting higher brain function,
but higher retention on memory items I care about.

~~~
dmreedy
If there are, they would be as difficult to recognize as germ theory to a 16th
century doctor. We simply don't know enough yet about the mechanism.

~~~
adrianN
You don't need germ theory to notice that washing your hands before performing
surgery improves outcomes.

~~~
dmreedy
Yes, but you do need to start moving towards germ theory to understand why
washing your hands with charcoal instead of soap results in dead patients
again. Of course scientific theory and scientific observation are bootstrapped
off of each other in the course of a dialectic, but raw statistics only get
you so far until deductive work is required for the next steps, and the cycle
repeats. It's easy to spot and distill in hindsight what is less obvious in
the act; and when you've got as many variables as the human experience does,
and as many ethical questions when you experiment with them, the process
becomes all the more vague.

~~~
volkk
can you explain a bit about the charcoal/soap thing?

------
overgard
I dont think abstract games like luminosity work, but I think its fatalistic
to assume that your intelligence is fixed at a certain point. You can no doubt
expand your intelligence (not just knowledge) by expanding your horizons, but
learning is a difficult process.

~~~
Bartweiss
This... feels more like optimism than evidence.

You can improve your performance on almost any single task, which is great
news. Certain forms of learning (e.g. language) help defer dementia, exercise
and meditation improve focus, memory tricks improve recall, learning diverse
topics improves synthesis.

But in terms of 'intelligence' in the clinical sense? Despite many people's
efforts, I've never seen a single convincing success for healthy adults
increasing their IQ. You can practice a specific test until it's inapplicable,
but if you take one test, try to raise your IQ, and take another test, you
won't see gains. Dual N-back is still the best thing known, and it doesn't
work appreciably.

Fortunately, the practical stuff - memory, focus, breadth of knowledge - is
awfully helpful in real life. But I've never seen any evidence at all that
intelligence in the IQ sense can be raised in adults.

~~~
overgard
Admittedly I have no evidence, but anecdotally when I started taking my health
and diet seriously I feel like I gained about 10 IQ points just from fueling
things better. Similarly, learning various mnemonics's and learning strategies
has really helped my ability to learn more rapidly. You could argue that I'm
simply capitalizing on my innate ability better -- and that would be true --
but the same argument would apply to gaining strength by training muscles.
Nobody sees a muscular person thats ripped and thinks they just are lucky to
be that way -- we recognize that there is both biology and effort in play, but
for some reason people seem to think intelligence is pure biology. I cant
prove my case, but im skeptical of that proposition.

No doubt people have different ceilings for their potential, but I feel like
most people who are viewed as smart arent just leaning on natural talent,
rather they've cultivated skills that get the most out of what they have

~~~
Bartweiss
I did dance around the lifestyle point a bit, for this reason. I've never seen
a study on "we took otherwise-healthy people with bad habits and got them to
eat, sleep, and exercise better", but I'd have higher hopes for that boosting
healthy-adult IQ than literally anything else.

And I agree on the ambiguity around 'innate ability'. One could argue that
general health is 'subtractive', that bad sleep/diet/exercise are drawing down
potential that would exist otherwise. But that's far less clear-cut than
something like "only works in dementia patients" or "only helps with Korsakoff
Syndrome". I think there's something to it - lifting heavier weights and
sleeping more are not unbounded sources of improvement - but I think it's also
kind of a copout. If those things operate on a continuum, and people we
consider "healthy adults" can still see benefits from them, that's what we
wanted to find.

------
rajadigopula
I am a lifetime member of lumosity for past few years. they used to have a
beta-version game of "Dual-N-Back" which is the only one that might have some
impact on brain. The irony is they removed it and keeping the useless ones;
even keep modifying the UI of the same games again and again..

~~~
gargarplex
Dual N Back is a test of memory. Where is the evidence that practicing does
anything besides decrease the accuracy of the test?

~~~
hoodwink
Here is the most comprehensive review of the DnB experiments I have ever seen:

[https://www.gwern.net/DNB%20meta-analysis](https://www.gwern.net/DNB%20meta-
analysis)

~~~
ice109
>This is the website of Gwern Branwen, writer & independent researcher. I
specialize in psychology, statistics, and technology

forgive me if i don't take this meta-analysis seriously.

~~~
Tenobrus
You may not have read any of Gwern's articles before. For many people, the
quality of articles they've written in the past is high enough to give them
(Gwern) quite a positive reputation. At least I'm likely to click on basically
anything written by Gwern.

Critically if you actually read the article you can judge for yourself whether
they know what they're talking about, as it's going to be both incredibly in
depth and include disclosures of basically all caveats.

------
eagsalazar2
This is honestly kind of depressing to me. I've never used an app like
Luminosity regularly but I like the idea that I could train my brain and
improve my focus and memory. Neuroplasticity right? Well actually no :(

~~~
pkghost
Neuroplasticity yes indeed; Lumosity users are at least getting better at
Lumosity games. _Generalization_ no.

------
seshagiric
So in general is there any thing that does help improve cognitive performance?

~~~
colorint
Well, you'll want to start by zeroing in on exactly what you mean by
"cognitive performance." The abstract of the study the linked article is about
has this to say:

>Pre- and post-training, participants completed cognitive assessments and
functional magnetic resonance imaging (fMRI) during performance of validated
decision-making tasks: delay discounting (choices between smaller rewards now
vs. larger rewards in the future) and risk sensitivity (choices between larger
riskier rewards vs. smaller certain rewards).

You could certainly imagine doing things to get better at these kinds of tasks
---including practicing these tasks. Whether this gets at some kind of murky
"cognitive performance" thing depends on how much you're into phrenology and
dualism.

Since it seems like you want a scientific answer to this question, you'll
always have to confront the realities of experimental design, and because of
that, you'll have to nail down exactly what it is you're after. And "cognitive
performance" isn't nearly specific enough.

------
eagsalazar2
It seems surprising to me that people agree you can increase focus and memory
through exercise but not IQ. I would think that by increasing focus and memory
your comprehension scores, for example, would increase dramatically. I know if
I'm distracted my comprehension is way lower. Is that now how IQ tests work?

------
l4xecz
cmd+f luminosity results in nearly as many occurrences as lumosity. That name
is cursed. Probably doesn't help that the correct name gets auto-corrected to
the incorrect.

------
b6
I have no idea what I'm talking about in this area, but the only brain games
I've run into that I think might actually be helpful are the ones that are
somewhat painful, e.g., BrainHQ. Definitely not fun. I start to sweat just
thinking about some of the tasks in there.

~~~
Bartweiss
Probably, no brain games at all boost IQ in healthy people. But some of them
definitely help with memory, focus, etc, and I agree that those seem to be the
hard ones.

Dual n-back, the general benchmark for these things, is _hard_. It's
exhausting like intricate coding or a tough math proof is exhausting.
Certainly I'd be suspicious of anything claiming to do better than that with
less mental effort.

------
manbearpigg
If luminosity challenges have any consistency at all, then you would think
that users must perform better on luminosity challenged themselves.

