
Key-Value Memory Networks for Directly Reading Documents - shagunsodhani
https://gist.github.com/shagunsodhani/a5e0baa075b4a917c0a69edc575772a8
======
fpgaminer
It is my understanding, from a sparse reading of these kinds of networks, that
at a high level they basically just add memory units to the networking, and
then the "normal" fully connected/convolution parts consume and control those
memory units like any other input/output. So the "normal" parts of the network
have to learn how to use the memory facilities that they've been provided.
Much like how LSTMs are just fully connected networks with some storage values
that they can gate, and thus learn memory by learning proper gating.

It makes me wonder ... do human brains also have to learn how to use memories?
I mean that merely as a curious question, not as a means to inform biology or
vise versa. We have to learn how to use all our senses and all our muscles, so
perhaps it makes sense that our brains also have to learn how to use long term
memory facilities. In fact, there's the age of amnesia, which is around 6,
after which the majority of people forget most of their long term memories up
to that point. Perhaps that's an indicator that our brains do have to learn
how to store long term memory, and it takes 6 whole years to fully learn. Or
maybe it's a side effect of some biological process; the hippocampus maybe
changes at age 6 or something.

Just an interesting thought that we take memory as a de-facto feature of the
brain, but perhaps it's learned like everything else and perhaps its efficacy
is affected by how we learn to use it at a young age.

~~~
landhar
Could you provide a reference for this "age of amnesia" ? I've heard many
people claim that it's not possible to remember things before age 6. But I
personally have lots of memories from ages 2 to 4 (these memories are very
easy for me to accurately date, because I lived in different countries during
these years). And I would definitely categorize those memories as "long term":
I remember places, visits of relatives, toys I had, play dates and birthday
celebrations, bike rides, etc...

~~~
aab0
[https://en.wikipedia.org/wiki/Childhood_amnesia](https://en.wikipedia.org/wiki/Childhood_amnesia)

