
Learning to Cooperate, Compete, and Communicate - maccaw
https://blog.openai.com/learning-to-cooperate-compete-and-communicate/?source=hn
======
jboggan
This is great. I've always thought intelligence can only be defined as an
emergent property of self-replicating systems operating under stress, and this
provides a good framework for that "stress".

~~~
TeMPOraL
> _intelligence can only be defined as an emergent property of self-
> replicating systems operating under stress_

That's a prescription for making one, but it isn't a good definition of
intelligence. It doesn't tell us what to expect from it.

Personally, I prefer intelligence defined as extremely strong, cross-domain
optimization power.

------
deepnet
Peter Abeel ( OP link author with Igor Mordatch ) explains his groups work [1]
as a guest lecturer for Berkeley's cs294-112 Deep Reinforcement Robotics.

[1]
[https://www.youtube.com/watch?v=f4gKhK8Q6mY&list=PLkFD6_40KJ...](https://www.youtube.com/watch?v=f4gKhK8Q6mY&list=PLkFD6_40KJIwTmSbCv9OVJB3YaO4sFwkX&index=25)

This great talk starts with his work at OpenAI on neural net safety and
adverserial images, then the OP research paper Emergence of Grounded
Compositional Language in Multi-Agent Populations[2] concluding with his work
( with Andrew Ng ) reinforcement learning helicopter flight and stunt
controllers from human pilots.

The OP multi-agents divide labour and apparent collaborative plans appear.
That the goal seeking agents split up, appear to dance to and from from their
goal, distracting the predators from their kin is to all appearance
coordinated and clever.

Dawkin's Selfish Gene espouses alturism as an inevitability of genetic
relatedness, the individual sacrifices but the genes persist in siblings.

In this work alturism emerges purely memetically.

The Nash equilibrium of cooperation jumps the local minima of selfishness in
this prisoners dillemma.

Multi agent enviroments are difficult to learn with many false minima for the
learners.

This work hints that the loose coupling of language, rather than direct
sharing of memories or genes, is noisy enough to find more global solutions
that appear complex or 'plan-like'.

Maybe these AI's should be considered as planners, yet in a bottom-up
immediate-heuristic emergent way.

[2] [https://arxiv.org/abs/1703.04908](https://arxiv.org/abs/1703.04908)

------
EGreg
Am I the only one that finds this scary?

As I have repeatedly said, so far the intelligence we have been producing is
NOT the kind that applies abstract logic rules to figure out meaning.

It is the kind that takes full advantage of computers' strengths: perfect
copying and speed.

So good ideas are copied and propagate. Neural networks are just this on
steroids... basically extracting signals from noise and doing a search in a
space to maximize something, and storing the results.

Humans were able to transmit knowledge, then produce books etc. Now bits can
be perfectly copied with checksums.

This isn't general intelligence in the human sense. But that's what makes it
scary. It can solve these problems with brute force. Resistance soon may
really be futile. Not just in running, avoiding capture and death. But also in
ANY human system we rely on, including voting, due process of law, trust,
reputation, family, sex, humor, etc.

~~~
hacker_9
Still, better this is done in the open. Otherwise Russia and China will just
continue in secret, which is definitely a world I don't want to live in.

~~~
justicezyx
Not sure what you want to say. US can do it open, China and Russia can still
do it in secret.

While, as a matter of fact, you will be naive if you believe that there is no
secret research happening inside US government agencies.

~~~
hacker_9
My point is there is no stopping this research now, open or not neural nets
just have too much potential. So I'm happy there is an open initiative that
the big players support.

~~~
justicezyx
I think you are being too optimistic.

Big companies are just equally a problem as big governments. They are even
more effective and efficient in collecting user data and reaching privacy
sensitive information.

AI in general has become a game for big players, that's in itself a worrying
trend.

~~~
lowpro
Big companies are not like government. Government is the law and what everyone
adheres to; companies simply aren't. A private company can't force you to do
anything you didn't agree to (agreement = contract/interaction), governments
force people to do things they can't opt out of. Corporations are bad, but
governments are much worse because they can use information they have to force
people. If you are thinking that lobbying means corporations are basically
government and therefore it's all the same, this isn't true either as lobbying
doesn't always happen, and even through lobbying corporations normally lobby
for favorable non-restrictive legislation, it doesn't allow them to force
people as a government does.

~~~
TeMPOraL
Nah, you're thinking too directly. Private companies can absolutely force you
to do anything you don't want to, but they rarely do this - too costly in a
system with a somewhat working rule of law - and if they do this, they do it
pretty subtly. Examples from all across the spectrum:

\- Strong companies can and do force weak governments to adopt whatever
policies they want; see e.g. tobacco companies and their fight against health
warnings on cigarette boxes. Or private companies and their private armies -
happened in the past (e.g. East India Company), happens today (why do you
think people kill each other in Africa over tantalum they don't have
technological capability to use?).

\- Lobbying you mentioned. Doesn't always happen, but happens frequently.

\- Various low-level deals and bribery on e.g. city scale.

\- Low-skilled jobs. Just talk to people who are stuck with those, especially
in smaller cities. Employers can, and routinely do, make them do anything
because they basically own them.

So yeah, private companies are _not_ like governments. They can't just go and
coerce you to do something in the name of law. But they have lots of less
direct ways to coerce people if they really want to. The job of the government
is usually to make them not want to.

------
tomcam
I was honestly hoping this would be about communications within the openai
organization itself.

------
mgummelt
> a multiagent environment has no stable equilibrium

It does.
[https://en.wikipedia.org/wiki/Evolutionarily_stable_strategy](https://en.wikipedia.org/wiki/Evolutionarily_stable_strategy)

~~~
ilya_su
True in the general case, but the idea is to set things up (possibly by
constantly changing the environment) so that there isn't one. This has
happened at least once, when the brains of our ancestors have tripled in size
relatievly quickly.

