
The Purpose of Sleep? To Forget, Scientists Say - azuajef
https://www.nytimes.com/2017/02/02/science/sleep-memory-brain-forgetting.html?_r=0
======
tprice7
As someone with no neuroscience background I’ve suspected this for a while
just based on how Boltzmann machines work. These are a type of unsupervised
artificial neural network.

The training algorithm for Boltzmann machines has two phases:

1\. (Awake phase) Feed the network training data, and increase the weights
between neurons that tend to get activated simultaneously in this phase.

2\. (Dreaming phase) Let the network run without any training input, and
decrease the weights between neurons that tend to get activated simultaneously
in this phase.

This training algorithm wasn’t designed to emulate how human brains work, it
just sort of falls out of the math. And yet (1) resembles the Hebbian
principle from neuroscience (neurons that fire together, wire together), and
(2) resembles the process described in this article. Actually I suspect
something stronger than what the article claims based on this: that the
connections that get pruned are the ones connecting neurons that fire together
while dreaming. It would be nice to have someone who knows stuff about
neuroscience comment on this.

~~~
dontreact
I wouldn't be so sure that Boltzmann machines were designed without how the
brain operated in mind. Terry Seknowski, one of the inventors, is a
computational neuroscientist.

In general, I think the influence of neuroscience on neural network research
has been subtle and perhaps underrated. For example, modern day convolutional
neural networks Havea lineage going back to Fukushima's neocognitron, which
was heavily inspired by Hubel an Wiesel's simple/complex cell model of visual
cortex based on single cell electrophysiology.

~~~
mlechha
They weren't. They were a generalization of the Hopfield networks. Boltzmann
machines are a stochastic version of the Hopfield network. The training
algorithm simply tries to minimize the KL divergence between the network
activity and real data. So it was quite surprising when it turned out that the
algorithm needed a "dream phase" as they call it. Francis Crick was inspired
by this and proposed a theory of sleep.

~~~
redler
Unpacking a comment like this is one of the quieter pleasures of reading HN.

~~~
mlechha
Haha I'm not sure if you're being sarcastic so I'll try to unpack the comment.
Hopfield networks were one of the first models of associative memory. They
themselves were based on a model of simple magnets called ising model
(generalized). Basically a group of binary units, each connected its nearest
neighbors with a coupling strength. Each unit prefers to be like their
neighbors. Hopfield developed a clever method to change the coupling so that
the networks can store and retrieve patterns of activity. In the Hopfield
network everything is deterministic, Hopfield himself realized that if this
constraint was relaxed this model could become a very powerful computational
machine. Which means that if instead of being always on or off, the units had
a probability of being on or off the networks could perform very general
computational tools [1]. Unfortunately, training these general stochastic
systems was not easy. With their Boltzmann machines Sejnowski and Hinton
proposed a possible solution. The activity of stochastic binary units
effectively encodes a probability distribution, so all they had to do was make
sure that the probability distribution being encoded by the activity of the
units was the same as that of the input. They did this by changing the
connection strengths between the units such that the activity pattern
minimized something called the Kullback-Leibler or KL divergence, which is a
measure of how close two probability distributions are (the one encoded by the
network activity, or the dream activity of the network and the probability
distribution of the real data e.g. a set of natural images). If two
distributions match exactly then the KLd is zero and if not it's large. When
they wrote out the math it turned out that the algorithm required two phases,
an awake phase where the connections were changed according to the real data,
and the sleep phase where the connections were pruned by the spontaneous
activity of the network without any input (or dreams). This analogy got a lot
of people excited, including Francis Crick and several others tried to test
this idea in real brains, but we are still waiting for a convincing result.

~~~
p1esk
Wow, this is a really good explanation! Thank you. I wonder though, are RBMs
being used in practice? Are they worth any future research?

~~~
mlechha
In applied machine learning not so much. They feel ancient! But some use them
to study the physics of computation. They were used to make the connection
between renormalization group (RG) and machine learning. RG is one of the main
workhorses of quantum field theory and condensed matter physics. The fact that
there's a mapping from RG to RBMs means that we can understand how deep
learning works by using the same techniques that modern physicists use to
understand the world! Here's a nice article on this topic if you're interested
[https://www.quantamagazine.org/20141204-a-common-logic-to-
se...](https://www.quantamagazine.org/20141204-a-common-logic-to-seeing-cats-
and-cosmos/)

------
RcouF1uZ4gsC
As an aside, in trying to make a title compelling (have a person as the
subject), the Times is inadvertently hurting science.

This article is about a hypothesis that has some evidence in mice, but tells
nowhere near the whole story, especially in humans with regard to sleep.
However, the title suggests more confidence than warranted. Likely, in the
near future, some newspaper will publish a similar article, except the purpose
of sleep will be to remember (which is also supported by studies).

The problem is the average person will see that and think, what scientists say
is always changing, and they are contradicting each other all the time. Then
when they see an article stating scientists say that climate change is
worsening, they will view it just like any article that has "scientists say"
in the title.

~~~
kbutler
> when they see an article stating scientists say that climate change is
> worsening, they will view it just like any article that has "scientists say"
> in the title.

Actually, you should see it exactly the same way - a possibly interesting
finding from probably a single study, and if you actually want to know the
scientific information behind it, you should go read the science, not the
newspaper, and look at multiple studies, and look for weaknesses in the study
that may weaken the findings.

...especially if it is something you already believe.

~~~
inimino
...which means newspapers are failing to educate the public, which is what the
OP is complaining about.

Just because they don't doesn't mean they shouldn't.

------
majkinetor
Not convincing for me.

While forgeting might be one of the effects that happens it is certainly not
the main purpose of sleep.

If you forbid people to sleep long enough they die and nobody knows why (not
counting microsleeps). Fatal Famililal Insomnia is erm.... fatal. Its
certainly not because you didn't forget. You can use alternatives for that,
such as boosting endocanabinoid system. THC in mices also make them forget
stresfull events in life and vice versa (blocking CB1 receptor makes them
remember). There is strong anecdotal evidence that it happens in all human
consumers too. Also, there are few people that can never forget [1], one
depicted in House MD episode and those people sleep and I bet they would still
die if they didn't.

All those facts do not align in my mind with above stated hypothesis.

AFAIK, some neural pathways get improved during the sleep while others are
pruned. That sounds more like a brain is in a maintenance mode, filtering out
irrelevant stuff from important (among other things that might happen).

[1] [http://discovermagazine.com/2006/jun/j-woman-
memory](http://discovermagazine.com/2006/jun/j-woman-memory)

~~~
roywiggins
It's possible Fatal Famililal Insomnia kills you by some other mechanism, and
the sleeplessness is just a symptom, I figure. I don't think we know that the
sleeplessness is the thing that kills you.

"Like all prion diseases, FFI is a progressive neurodegenerative disease,
which means over time there are fewer neurons (nerve cells). Loss of neurons
in the thalamus, as well as other mechanisms not yet fully understood, cause
the symptoms of FFI."

"Although the main target of FFI is the thalamus, other parts of the brain are
affected as well including the inferior olives. The inferior olives are part
of the medulla oblongata and are important for coordinating our movements
(motor control). Losing neurons in the inferior olives can make it harder for
a person to control their movements as seen in later stages of FFI."

[https://rarediseases.info.nih.gov/diseases/6429/fatal-
famili...](https://rarediseases.info.nih.gov/diseases/6429/fatal-familial-
insomnia)

When your whole brain is going haywire due to neuron loss, attributing death
to sleeplessness seems premature.

~~~
majkinetor
> When your whole brain is going haywire due to neuron loss, attributing death
> to sleeplessness seems premature.

Or it could be another way around, as its not easy to have experiments like
this I guess we can never know.

But yes, that is certainly possible that in FFI you die due to some other
reason. I didn't see that info you provided but I remember reading in NGeo
many years back that FFI people seem healthy in other aspects.

There was 1 experiment with mices tho - after 32 days they all died witout
sleep:

[1] [http://health.howstuffworks.com/mental-
health/sleep/basics/d...](http://health.howstuffworks.com/mental-
health/sleep/basics/die-if-never-sleep.htm)

------
sudoscript
The brain naturally employs dimensionality reduction for memory. Sleep is one
example. Another, simpler one is reading -- how far back can you remember word
for word when you are reading something? Maybe a sentence at most? But you
still remember enough to understand what you're reading, because of efficient
dimensionality reduction.

Some neural networks mimic this, such as LSTMs. But it's a poor mimicry at
best. The brain has a natural, built-in selection mechanism. It seems to
"know" what to remember and what to forget. How could we implement something
like this in a deep neural network?

(This is key step to giving computers "personality". Which emerges from a
selective set of memories and trained behavior)

~~~
grromrell
The process in the article is pretty similar to dropout in neural nets. But
instead of "knowing" what to get rid of, we randomly prune. The brain may do
it randomly or intelligently, hard to say based on these studies.

See here for info on dropout: [http://bit.ly/1mneaL5](http://bit.ly/1mneaL5)

~~~
ianai
Seems like it scores the pruning algorithm and adjusts it as well.

------
LanceH
I went through a 3 week period of sleeping no more than an hour a night, if at
all. Things like "this morning" became very muddled. If I thought about what I
had for breakfast, there was no difference between breakfast today and
yesterday and even a couple days before that -- they were recalled to mind
equally.

Multiple days in a row felt like "today." It's hard to describe the feeling
because I've never felt anything like it before or since. It just felt like
the stack was overflowing for "today". One night of sleep and things were back
to normal.

~~~
reitanqild
_I went through a 3 week period of sleeping no more than an hour a night, if
at all._

Can you explain why this happened? Off the top of my head it is either
military or having a child, but I have a feeling there might possibly be an
even more interesting explanation.

~~~
tcoff91
Could also be a manic episode.

~~~
reitanqild
Didn't think of that. Hope my original question didn't come off as rude, I'm
just seriously interested how this is possible since I have trouble just
pulling a single all-nighter :-|

------
deegles
It seems like forgetting is more of a side-effect than a purpose. I thought of
some more headlines in this vein:

"The purpose of fire? To create CO2, Scientists Say"

"The purpose of exercise? To Sweat, Scientists Say"

"The purpose of music? To Make Noise, Scientists Say"

~~~
Frenchgeek
Replace "Scientists" with "Journalists" and you got it.

------
Eerie
No, no, that's the purpose of Drink.

------
jfaucett
The whole premise of there being a single function or main function of sleep
seems slightly far-fetched IMHO.

For example, is the brain's function to think rationally? To speak and
understand language? To unconciously regulate bodily functions?

Wouldn't it be more likely that due to the brain's multifunctional nature that
an evolutionairly pruned process has developed which encompasses multiple
necessary subroutines (energy preservation, memory consolidation, waste
cleansing, etc.)

------
bitwize
In other words, to garbage-collect.

~~~
trendia
Damn, it'd be nice if our brain were stack-based and automatically called
destructors.

~~~
Klockan
That would be nice till you accidentally buffer overflow and writes garbage
over your entire brain.

------
scotty79
Since I learned that lack of sleep causes hallucinations I though sleep exists
to avoid neural network overfitting.

------
state_less
While I prune at night, I keep waking up to this strange american dream.

------
aftbit
My second favorite quote from the Kingkiller Chronicles:

>First is the door of sleep. Sleep offers us a retreat from the world and all
its pain. Sleep marks passing time, giving us distance from the things that
have hurt us.

Full quote: [https://www.goodreads.com/quotes/156661-perhaps-the-
greatest...](https://www.goodreads.com/quotes/156661-perhaps-the-greatest-
faculty-our-minds-possess-is-the-ability)

~~~
majkinetor
Meh... dreams can be very nasty and hurtful. When I dream bad, I am frecked up
entire day.

------
haberman
For a theory of sleep to convince me, it would need to explain how the benefit
from sleep can possibly be worth the huge disadvantage of leaving yourself
completely vulnerable for hours a day. Evolution-wise, that seems hard to
justify. A variant of human that didn't have to sleep seems like it would
overrun the sleep-needing humans, unless the non-sleeping human just isn't
possible.

------
misotaur
"Life without absorbing occupation is hell; joy consists in forgetting life."

------
mratzloff
Remember that sleep evolved very early on. Arthropods (insects, arachnids,
crustaceans, etc.) sleep. Worms experience a sleep-like state. It's even been
shown that cyanobacteria experience circadian rhythms.

Fruit flies whose sleep patterns were disrupted were slower to learn new
things and faster to forget the things they learned. I'm not sure there is a
single purpose for sleep, but memory is certainly a critical factor.

------
georgeecollins
People have speculated that sleep is meant to organize your thoughts for
years. I know when I worked in a Neuroscience lab years ago that is what many
researchers thought. But to say it is to "Forget" is almost certainly an over
simplification. It might be to optimize to remove the extraneous, but that is
probably not all of it either. Nothing about how the brain works is simple.

------
neaden
What's the animal with the least brain that sleeps? Jellyfish apparently take
prolonged rests on the sea floor which is a kind of sleep.

~~~
chc
If you count that as a kind of sleep, then it would surely be them, since you
can't have less brain than a jellyfish.

------
givan
Anybody that studies Carl Jung work on dreams will understand how superficial
this theory that tries to explain sleep as forgetting is, and how this theory
fails to account for symbols and archetypes that show in dreams, also any
event can be remembered by hypnosis so nothing is really forgotten with sleep.

~~~
gowld
Also they will understand how this work is scientific and not psuedoscientic
falsehoods.

------
burtonator
This article suffers from the single cause fallacy. Sleep exists, and evolved
for multiple reasons.

------
JoeAltmaier
I'm thinking its more like, to integrate memories. Lessen the impact of the
most recent memories lest we lose earlier lessons. Kind of like a moving
average or filter. Dim traumatic events so we can think about them without
over-reacting.

------
easy_rider
Surely it's not the other purpose. How about recovery and healing processed?

~~~
sametmax
s/other/only I guess.

Yeah, global maintenance of the mind and body is more like it.

------
eitally
Every parent probably intuits this, whether they consciously realize it or
not. The #1 cause of babies crying [that isn't pay or hunger]: overstimulation
& a need for sleep to "reset".

~~~
tossaway1
I think most would intuitively attribute that to sleep restoring energy levels
so the baby feels better, not anything to do with memory.

~~~
majkinetor
The main reason is probably is need to be calm while the body is experiencing
that level of growth. You wouldn't want to build a house with constant
earthquake I guess.

------
pizza
reposting a comment I made a while ago @
[https://news.ycombinator.com/item?id=11327157](https://news.ycombinator.com/item?id=11327157)

\-----

Very relevant PubMed article many of you will undoubtedly find interesting:
"Partial sleep in the context of augmentation of brain function."

Abstract: Inability to solve complex problems or errors in decision making is
often attributed to poor brain processing, and raises the issue of brain
augmentation. Investigation of neuronal activity in the cerebral cortex in the
sleep-wake cycle offers insights into the mechanisms underlying the reduction
in mental abilities for complex problem solving. Some cortical areas may
transit into a sleep state while an organism is still awake. Such local sleep
would reduce behavioral ability in the tasks for which the sleeping areas are
crucial. The studies of this phenomenon have indicated that local sleep
develops in high order cortical areas. This is why complex problem solving is
mostly affected by local sleep, and prevention of local sleep might be a
potential way of augmentation of brain function. For this approach to brain
augmentation not to entail negative consequences for the organism, it is
necessary to understand the functional role of sleep. Our studies have given
an unexpected answer to this question. It was shown that cortical areas that
process signals from extero- and proprioreceptors during wakefulness, switch
to the processing of interoceptive information during sleep. It became clear
that during sleep all "computational power" of the brain is directed to the
restoration of the vital functions of internal organs. These results explain
the logic behind the initiation of total and local sleep. Indeed, a mismatch
between the current parameters of any visceral system and the genetically
determined normal range would provide the feeling of tiredness, or sleep
pressure. If an environmental situation allows falling asleep, the organism
would transit to a normal total sleep in all cortical areas. However, if it is
impossible to go to sleep immediately, partial sleep may develop in some
cortical areas in the still behaviorally awake organism. This local sleep may
reduce both the "intellectual power" and the restorative function of sleep for
visceral organs.

[https://www.ncbi.nlm.nih.gov/pubmed/24822040](https://www.ncbi.nlm.nih.gov/pubmed/24822040)

------
supermatt
This sounds like defragmentation for the brain

------
auvrw
the article title (and indeed, upon groggy inspection, the article itself) is
misleading: it's likely the the _hypothalmus_ "forgets" while the the _cortex_
"remembers"/"learns" during sleep.

i think ppl have been generally aware of this since the 80's

------
stevehiehn
Haha, So basically its saying brains use the old JVM 'Stop the world' garbage
collector?

------
alphonsegaston
If so, then what function do dreams have? Are they akin to zero-filling a
disk?

~~~
randomdata
Perhaps dreams are the mechanism that allows the dimensionality reduction to
take place? Or at least a side effect of us becoming semi-conscious while the
process is taking place?

------
blazespin
Man, I have to ask, how the hell does something like this actually evolve.

~~~
bbrian
I frequently recommend the Foundational Falsehoods of Creationism (AronRa)
videos on YouTube. They're obviously more about addressing creationism, but
it's enjoyable and educational and necessarily explains a lot about evolution
that I didn't know previously. One great part is where he explains the lineage
of species by using folders on Windows.

[https://www.youtube.com/playlist?list=PL126AFB53A6F002CC](https://www.youtube.com/playlist?list=PL126AFB53A6F002CC)

------
jaequery
this study seems to contradict people being more forgetful when not getting
enough sleep

------
myf01d
My only complaint is that we sleep only 6-9 hours, oh god it should have been
20-22 hours

------
faceyspacey
If you trust your intuition, you've known this is one of the MANY purposes of
sleep for a while now. The western science culture of needing scientists to
validate what you know as true damages our own ability to trust ourselves.
You're not going to have statistical and scientifically proven evidence for
99.99% of most things you encounter in your daily life. That's why the mindset
of waiting around for science to prove something is harmful. You need to be
able to go with your own intuition most of the time. Anyway, this was obvious.
And keep in mind, it's just one of the purposes--can things not have multiple
purposes??

~~~
gus_massa
Let's pick some intuitive false facts:

* The lack of inertia is very intuitive. Once you stop pushing something it will get to rest soon. This is extremely intuitive. (Unless you live nearby a frozen lake???)

* The spontaneous generation of small creatures like worms and flies is intuitive. You put some crap and wait a while, and you'll get a few small animals. (Did someone ever believe that elephants were generated spontaneously?)

* Leeches are good for your health. I'm not even sure that this was intuitive, or that it was even a popular cure, but it was used in traditional western medicine. Modern medicine use penicillin, I don't think that using an extract from a mold is intuitively good.

~~~
NobleLie
> lack of inertia.

What? No, im pretty sure that some people knew what friction was and that if
theoretically, there was none of it, things would continue to move. Thats
intuitive to me.

> Spontaneous generation of animals.

Fair example; ive read this old theory before.

> Leeches Blood letting is still somehow advised as being healthy for certain
> conditions (too much iron, and "resetting" a percentage of ones blood. That
> or they just really want people to donate) Plus you even admit that youre
> not sure about it being 'false' fact brought by intuition, so why even
> include that?

OP's point is not that intuition is infallible. Not much is infallible
(perhaps nothing, if one is atheistic) It's that science has seemed to
shunt/reduce/eliminate the intuition of a subsection of the population. It's
his opinion (and mine) that this has harmful effects that are worth examining.

~~~
gus_massa
>> _lack of inertia._

> [...] _That 's intuitive to me._

It was a popular idea a long time ago
[https://en.wikipedia.org/wiki/Inertia#Early_understanding_of...](https://en.wikipedia.org/wiki/Inertia#Early_understanding_of_motion)
and after teaching physics I think that inertia is not so intuitive. What did
you thought when you were 13 years old and had no formal training in physics?

> _OP 's point is not that intuition is infallible._ [...]

The problem is that if you ask enough people, each one may have a different
intuitive idea. Some of them will be slightly different and some of them
completely contradictory. Everyone has an intuitive idea of how to manage
economy and a country, but not everyone has the same intuition about that.

That's why to do science you can use intuition to design experiments and make
theories, but you MUST confirm them with experiments.

