
Brainwaves Encode the Grammar of Human Language - dnetesn
http://maxplanck.nautil.us/article/341/brainwaves-encode-the-grammar-of-human-language
======
sethbannon
"Being able to learn and use grammar is unique to humans."

While we haven't observed grammar in non-human animals yet, whether they can
learn and use it is very much still an open question.

"considering the rather limited number of experiments and the difficulty to
design experiments that unequivocally demonstrate more complex rule learning,
the question of what animals are able to do remains open." [1]

"when we consider how animal vocalizations are analyzed, the types of stimuli
and tasks that are used in artificial grammar learning experiments, the
limited number of species examined, and the groups to which these belong, I
argue that the currently available evidence is insufficient to arrive at firm
conclusions concerning the limitations of animal grammatical abilities." [2]

1:
[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/)

2:
[https://link.springer.com/article/10.3758/s13423-016-1091-9](https://link.springer.com/article/10.3758/s13423-016-1091-9)

~~~
combatentropy
It's easy to be a skeptic forever. "I think we need more studies." Steven
Pinker in his book _The Language Instinct_ states that language is unique to
humans. Whatever language animals have, it's like a pebble and the Chrysler
Building. My cat can well tell me when he's hungry. But think of the richness
of human language, which can express things in front of you and things a
million miles away, things now and things past, their color, shape, activity.
I can express whether something is, or hypothetically what it would be if, and
I can express how I feel about the possibility.

~~~
starbeast
We really do need more studies to make any kind of statement about the
uniqueness of grammar, given that we cannot communicate in a representative
sample of animal languages in order to make this claim strongly and also that
grammar can be quite simple, so we cannot infer this claim from the relative
difficulty of the task.

~~~
maoeurk
> animal languages

When I took a Linguistics course at university, both the professor and text
book were very emphatic that animals do not have (read: have never been
observed using) language, they have communication. They went as far as to say
that language was _the_ defining characteristic of humans.

~~~
starbeast
What is their definition of language?

Comprehension of sentences by bottlenosed dolphins -
[https://www.sciencedirect.com/science/article/pii/0010027784...](https://www.sciencedirect.com/science/article/pii/0010027784900039)

First evidence that birds tweet using grammar -
[https://www.newscientist.com/article/dn20615-first-
evidence-...](https://www.newscientist.com/article/dn20615-first-evidence-
that-birds-tweet-using-grammar/)

~~~
maoeurk
Hmm, that's a very good question. I'm not exactly sure, I don't have access to
either right now, so I'll have to guess.

I think it's probably in how sophisticated it is. The bird example seems
analogous to something like a handshake. A handshake has a sort of grammar to
it, has to learned by observation/instruction, and if you scrambled it, people
would be confused. However, I don't think the ceremony of handshaking
qualifies as a language.

The dolphin example as well, while more sophisticated, isn't very when
compared to what humans nearly universally do without effort. Though that
could be a limitation of the study. Given enough time we may discover a
dolphin language or teach them one comparable to what we can do.

Take what I said with a grain of salt though, I don't really know what I'm
talking about here.

~~~
starbeast
>Take what I said with a grain of salt though, I don't really know what I'm
talking about here.

I take it all with a lot of salt until we have a wide range of animal
languages down. One thing I found interesting is the recent work on dolphins
has found entire images of things they have echolocated embedded in their
clicks, suggesting a possible pictorial language.

Here's an excerpt from the summary -

>"We discovered transient wave patterns in the water cell that were strikingly
similar in shape to the objects being echolocated. To further investigate the
shapes in these Cyma Glyphs we converted the images to 3-D models. As far as
we know, this the first time such a method has been implemented.

>Transient wave images were found for the cube, cross, flowerpot and a human
by examining single still frames, or single frames acquired in bursts, usually
where there was high power and dense click trains in the recording. The
parameters involved in capturing echolocation images with the CymaScope
include careful control of the acoustic energy entering the visualizing cell.
Water requires a narrow acceptable “power window” of acoustic energy; too
little energy results in no image formation and too much energy results in
water being launched from the cell. As a result, many hours of work were
involved in capturing the still images of the echolocated objects. The imagery
for the human subject was captured in video mode and has been approximately
located in time code. The image formed between 19.6 to 20 seconds into the
recording and may derive from a set of dense clicks at approximately 20
seconds. The video was shot at 24 frames/second with a possible audio
synchronization error of plus or minus 3 frames."

[https://www.omicsonline.org/open-access/a-phenomenon-
discove...](https://www.omicsonline.org/open-access/a-phenomenon-discovered-
while-imaging-dolphin-echolocation-
sounds-2155-9910-1000202.php?aid=76570&view=mobile)

------
dhairya
"A key finding of the new study is that these artificial neural networks, when
fed example sentences, give off patterns of energy that mimic what the brain
does when it processes a sentence."

I wish the article fleshed out more details about the research. It looks like
they found a correlation between the output of a specific ANN that seems
similar to observed neural oscillations in the brain. The causality claim in
the article seems premature.

Also curious if the neural encoding of grammar is consistent across different
languages. So many questions about the specifics, but cool research for sure.

~~~
gumboshoes
The research the article is based on is linked at the bottom of the article:
[https://journals.plos.org/plosbiology/article?id=10.1371/jou...](https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2000663)

~~~
dhairya
Thanks for sharing that link! Totally missed it.

------
phaedrus
This reminds me of an idea I had when I was a freshman in college, in 2004 -
2005. I proposed a research project based on generalizing Markov chat bots to
amplitude waves through neuron-like nodes. Basically, instead of discretely
recognizing A -> B -> C but not A -> C -> B, the idea is the Markov node (or
neuron) structure learned would respond with a sine-wave shaped pulse that
initially rises and then goes negative, so that a chain of nodes "A -> B -> C"
would still respond to a slightly out-of-order or different sequence, just not
as strongly.

Initially my undergraduate CS advisor was excited by my idea, but only, it
turned out, in the context of using it in genetic sequencing (because
buzzwords). I gave him my honest opinion that adding my contrivance could only
perform worse that existing genetic sequencing algorithms; my intent was to
propose and investigate a way the brain might reasonably be recognizing
language. He was entirely uninterested in this, and I got too busy with
college / work and never developed the idea beyond a toy demo. Maybe I should
look at it again...

~~~
patzal
It is depressing how many good thesis topics not getting accepted because of
some kind of personal preference/agenda of the advisors. Similar story
happened to me.

~~~
AlexCoventry
I think in this case it was probably more about funding opportunities.

------
dhairya
The experiment is fascinating. It looks like they developed a model (DORA) for
learning relational reasoning. It turns out that model showed similar
oscillatory activation patterns to observed cortical signals. As a comparison,
they also looked at a fully connected RNN with a which did not produce
oscillations. So the key research finding is that time-binding is a requisite
to produce oscillations in neural networks.

The paper's conclusions:

"In sum, we remain relatively agnostic about the specific details of the
required representational hierarchy because we do not yet know how to link the
predicate calculus representations we use to natural language mental and
cortical representations. What we are not agnostic about is the need for
asynchronous time-based binding in order to produce oscillations in a neural
network, as well as the need for representational hierarchy to produce the
particular pattern of oscillations observed here and in"

The paper on evaluating if NLP parsers with representational hierarchies
reflect what the cortical functionality:

"Our results naturally beg the converse question, as to whether any system
with representational hierarchy could produce the oscillatory pattern of
activation that [6] and DORA show. For example, could natural language
processing (NLP) parsers, which feature representational hierarchy and were
developed to specifically parse natural language in a machine, produce
oscillations and the pattern seen in [6] and in our simulations? In principle,
any system of hierarchy that is (de)compositional has the representational
ingredients to encode units that could be fired in a sequence. However, we
would argue that any given representational hierarchy could only produce
oscillations if it were combined with time-based binding, which, as far as we
know, no NLP system features. In DORA, time-based binding is the oscillation
of activity throughout the network, which is part of the reason why the RNN
did not show oscillatory activity nor the specific pattern from [6]. The
representational structure of DORA (a (de)compositional role-filler binding
predicate logic) is what makes the oscillations take the form that the data
from [6] have. In terms of the specifics of the observed 1-2-4 Hz pattern,
both [6] and our simulations are highly shaped by the word presentation rate
of 250 ms/4 Hz. But without time-based binding, there is no mechanism to
produce oscillations in a network, even in a NLP parser or other system with
representational hierarchy."

------
buboard
This is a model, a hypothesis, the title of nautilus makes it as it is a
proven fact. "A mechanism for the cortical computation of hierarchical
linguistic structure" is the original title.

Reminds of another proposed model from Gyuri Buszaki about how natural brain
rhythms might be used to encode sentences.

[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/)

------
carapace
This article reminded me of the (poorly named) "Feynman Machine":

[https://arxiv.org/abs/1609.03971](https://arxiv.org/abs/1609.03971)

> Efforts at understanding the computational processes in the brain have met
> with limited success, despite their importance and potential uses in
> building intelligent machines. We propose a simple new model which draws on
> recent findings in Neuroscience and the Applied Mathematics of interacting
> Dynamical Systems. The Feynman Machine is a Universal Computer for Dynamical
> Systems, analogous to the Turing Machine for symbolic computing, but with
> several important differences. We demonstrate that networks and hierarchies
> of simple interacting Dynamical Systems, each adaptively learning to
> forecast its evolution, are capable of automatically building sensorimotor
> models of the external and internal world. We identify such networks in
> mammalian neocortex, and show how existing theories of cortical computation
> combine with our model to explain the power and flexibility of mammalian
> intelligence. These findings lead directly to new architectures for machine
> intelligence. A suite of software implementations has been built based on
> these principles, and applied to a number of spatiotemporal learning tasks.

YMMV

~~~
dmos62
Sounds interesting, but this 2016 paper has only one citation. I'm reading it
now and this paragraph from the introduction compares it to traditional Deep
Learning networks:

> Our artificial Feynman Machines have several interesting properties which
> distinguish them from existing Deep Learning and similar systems. In
> particular, due to the much higher density and locality of processing, a
> Feynman Machine-based system can perform at least comparably while
> dramatically reducing the footprint in computational power, training data
> and fine-tuning. Feynman Machines can be arbitrarily split into modules
> distributed across clusters and the Internet, and systems running on low
> power devices such as phones can be dynamically and robustly augmented using
> low-bandwidth connections to larger networks running in the cloud. Models
> can be trained on powerful infrastructure and transferred for operation and
> further custom learning on smaller devices. Importantly, the same
> architecture can be used as a component in unsupervised, semi-supervised,
> fully supervised and reinforcement learning contexts. A variant - the Routed
> Predictive Hierarchy - is described, which allows a Feynman Machine to
> directly control a traditional Deep Learning network by switching it to use
> spatiotemporally-selected subnetworks.

This is the abstract of the previous paper that detailed the theoretical basis
[0]:

> Reverse engineering the brain is proving difficult, perhaps impossible.
> While many believe that this is just a matter of time and effort, a
> different approach might help. Here, we describe a very simple idea which
> explains the power of the brain as well as its structure, exploiting complex
> dynamics rather than abstracting it away. Just as a Turing Machine is a
> Universal Digital Computer operating in a world of symbols, we propose that
> the brain is a Universal Dynamical Systems Modeller, evolved bottom-up
> (itself using nested networks of interconnected, self-organised dynamical
> systems) to prosper in a world of dynamical systems. Recent progress in
> Applied Mathematics has produced startling evidence of what happens when
> abstract Dynamical Systems interact. Key latent information describing
> system A can be extracted by system B from very simple signals, and signals
> can be used by one system to control and manipulate others. Using these
> facts, we show how a region of the neocortex uses its dynamics to
> intrinsically "compute" about the external and internal world. Building on
> an existing "static" model of cortical computation (Hawkins' Hierarchical
> Temporal Memory - HTM), we describe how a region of neocortex can be viewed
> as a network of components which together form a Dynamical Systems modelling
> module, connected via sensory and motor pathways to the external world, and
> forming part of a larger dynamical network in the brain. Empirical modelling
> and simulations of Dynamical HTM are possible with simple extensions and
> combinations of currently existing open source software. We list a number of
> relevant projects.

Edit:

Apparently, the project is called Ogma and they have a youtube channel. Here's
their video [1] of a toy car learning to self-drive after a few laps of a
person driving it with an xbox controller. It runs on a NanoPi Duo. Not sure
if that's noteworthy or not.

[0] [https://arxiv.org/abs/1512.05245](https://arxiv.org/abs/1512.05245)

[1]
[https://www.youtube.com/watch?v=gCuadXj9KDc](https://www.youtube.com/watch?v=gCuadXj9KDc)

------
laretluval
More like, brainwaves can encode grammatical structure in one particular
model.

------
caublestone
I wonder if the structure of brain waves is a result of a gravity time
crystal.

[https://arxiv.org/pdf/1708.05014.pdf](https://arxiv.org/pdf/1708.05014.pdf)

~~~
kaghyu1667
Do you even know what that means?

~~~
blattimwind
I think microgravitational brain oscillations related to intermittent
contacting of crystal-based planars should be explored more deeply.

------
Animats
Is this a real result, or just speculation?

------
bra-ket
Also see Pulvermuller, The Neuroscience of Language:
[https://www.amazon.com/Neuroscience-Language-Brain-
Circuits-...](https://www.amazon.com/Neuroscience-Language-Brain-Circuits-
Serial/dp/0521793742)

------
tlow
Very skeptical of the veracity of any claim that, "what the brain does when it
processes a sentence." is at all universal across a wide sample of humans.

What about, for example, differences in languages, culture, personal
background, etc.

~~~
ahartmetz
AFAIK there is scientific consensus that much of human language is hardwired.
It's how infants can learn it so quickly.

