
Training agents to invent a language - runesoerensen
https://www.openai.com/blog/learning-to-communicate/
======
giardini
Luc Steels was doing this years ago (since mid-1990s IIRC):

[https://www.google.com/search?client=opera&q=luc+steels+inve...](https://www.google.com/search?client=opera&q=luc+steels+inventing+language&sourceid=opera&ie=utf-8&oe=utf-8&channel=suggest&gws_rd=ssl#spf=1)

Ah, yes, I see that a paper referenced on the website:

"A Paradigm for Situated and Goal-Driven Language Learning Jon Gauthier, Igor
Mordatch"

grants a footnote to one of Steels' 2012 publications, stating:

"A related line of work in evolutionary linguistics constructs a similar
language learning scenario entirely without fixed-language agents (Smith et
al., 2003; Steels, 2012; Kirby et al., 2014). All of the agents in these
environments construct a novel language simultaneously to accomplish some
shared task. This is an interesting separate line of research, but ultimately
a separate task from the understanding and acquisition problems discussed in
this abstract."

~~~
laretluval
It sounds like you're frustrated that deep learning is rediscovering stuff
that your field discovered years ago. I've come to the conclusion that the
only solution to that is to try to communicate more effectively; the reason
they are not talking to us is that they can't understand what we're saying.

~~~
mackan_swe
Why the frustration? We are progressing, are we not? A sign of us making no
progress would be when the new guys keep disproving the old theories.

~~~
laretluval
> when the new guys keep disproving the old theories

Most old theory seems to be incommensurable with deep learnings results, so
it's hard to evaluate whether this might be true.

------
sidhantgandhi
Implications:

\- Every Agent can have it's own language best suited for work in its domain.
It can then auto-translate that to English at the end. This adds ambiguity
once instead of at every step of the work if all of it was done in English.
Like doing math with floating point then converting to int vs doing math in
int the whole time. Concrete example: Law.

\- Agents could evolve a language that averages the Sapir-Whorf biases of
known human languages. This new language would help humans understand each
other better. Concretely, this language could be used as an intermediary in UN
Meetings from which known languages could be auto-translated to while
mitigating learned Sapir-Whorf biases.

\- As @amelius alluded to out, Agents could automate the process of learning
alien languages: what we saw in the film Arrival (Dan's short story is so much
better).

~~~
taurath
I certainly would love to read a list of Sapir-Whorf biases included in
English!

~~~
microcolonel
Biases against what baseline? A language optimized to express a finite set of
things?

~~~
sidhantgandhi
Yeah there is no objective baseline. However, if the Agent could evolve a
language that averages the biases of all human, that could be used as the
baseline couldn't it?

~~~
microcolonel
Some humans can not make mouth sounds; some humans don't have arms. You
already can't hit that baseline with one language.

As somebody who has done a great deal of idle bloviation about ideal
languages, I can say that there are many obstacles. For example, you need to
add a true noun because a new material is discovered and it's getting very
popular. You want a short and efficient name for it, but all the space is
allocated. So you add it to your table of weights, and suddenly every prefix-
coded word is pronounced and written completely differently.

If you can't use prefix trees, how will you allocate the words efficiently?

------
caublestone
I really like the introduction of "cost". Have you considered having a multi-
variable time decay distribution function for all learning so that the agents
need to reinforce every lesson?

Our brains forget over time because energy is limited and new connections take
energy from older memories. So you could introduce a global set energy
variable that is the ultimate denominator.

Lastly, have you considered connecting your agents in a one to one public
channel where a person interacts with an agent that asks the person about
"meaning of objects" and collects opinions from people to create it's own
opinion?

~~~
halflings
Regarding memory/forgetting, this was published yesterday:

[https://deepmind.com/blog/enabling-continual-learning-in-
neu...](https://deepmind.com/blog/enabling-continual-learning-in-neural-
networks/)

------
imh
Also interesting:

[https://github.com/iassael/learning-to-
communicate](https://github.com/iassael/learning-to-communicate)

and the paper:
[https://arxiv.org/abs/1605.06676](https://arxiv.org/abs/1605.06676) "Learning
to Communicate with Deep Multi-Agent Reinforcement Learning"

~~~
ibgib
Yes, they both seem to be focused on the differentiable inter-agent
communication aspect. I wonder if this is related to the recent articles on
how honey bees communicate to each other the learning required for pulling
strings, rolling balls, etc. (e.g. on HN
[https://news.ycombinator.com/item?id=13723645](https://news.ycombinator.com/item?id=13723645))

------
gallerdude
So many opportunities here! What if they let the agents just spout out random
characters and let the other agents listen?

They also talk about a lot of ways to get the agents to stop "cheating," but I
think a better method would just be better goals.

~~~
infogulch
Yes that's mostly what they did. E.g. they adjusted the goal to favor fast
communication, and then to impose a cost on exponential vocabularies. Both of
these sound very reasonable.

------
JamilD
It's interesting that a lot of added constraints mirror "real-world"
scenarios. We don't have an intrinsic global co-ordinate system, and
brevity/laziness is generally rewarded (which results in Zipf's Law in natural
language).

