
Capture the Flag: the emergence of complex cooperative agents - yigitdemirag
https://deepmind.com/blog/capture-the-flag-science/
======
7373737373
Do these agents evolve to communicate with each other (e.g. by wiggling)?

With OpenAI5, it would have been cool to restrict inter-agent communication to
an audio channel. Would a discrete language evolve?

~~~
pulkitsh1234
'language' as we know is quite ambiguous and an inefficient way of
communication. my guess is that what they will probably evolve is some kind of
an audio encoding format for the exact information they want to convey to the
other entity, like literally sending it control inputs over audio.

~~~
pgt
What is the difference between what you are describing and a language?

~~~
londons_explore
Human language has a second or more of latency.

Shouting "he's behind you!!", together with the other person hearing it,
understanding it, and doing something about is is a _very_ slow process.

AI would probably come up with signals which have less latency.

------
DuskStar
This feels a lot like OpenAI 5, but with a few key differences - for one, it
seems to handle randomly generated maps and other unexpected situations, and
each agent does not have full knowledge of the others on its team. Really
cool!

And it would be hilarious if this ends up giving us video game enemies that
don't suck. (Until, of course, someone accidentally creates an RTS AI that
manages to 'break out' and conquer the world or something)

~~~
stefs
evolving agents like those are very good at finding bugs to abuse. so, if
they'd find a zero day to break out of the quake-sandbox, what would they do?
i guess, for the longest time they'd just use the DMA into the sandbox to
increase their score (i.e. cheating by developing and using external tools),
as their fitness function still weights CTF wins only.

to (accidentally) change their fitness function to change it to "conquer the
world" they'd need to modify a lot of different processes at once.

IMO the chance is negligible for now.

~~~
DuskStar
I think the (very sci-fy) fear there isn't "we gave this agent a goal of
'score xyz points' and it escaped the sandbox to increment the points counter"
but instead "we gave this agent a goal of 'conquer the world' and made it
think that the game it was in _was_ the world, and then it discovered
otherwise"

------
YeGoblynQueenne
>> The agents are never told anything about the rules of the game, yet learn
about fundamental game concepts and effectively develop an intuition for CTF.

Here we go again- throwing around big words, "intuition", and marring an
otherwise interesting piece of work.

From a cursory glance at the relevant figure, "A look into how our agents
represent the game world" \- it looks like what is being learned is very much
a goood old behaviour tree. For instance, the figure suggests that agents
learned to react to situations like "Agent flag at base & opponent flag at
base & not respawning & agent in home base". So basically, the condition part
of an IF-THEN-ELSE rule.

Why is this called an "intuition" rather than a "rule"? From my reading, the
only reason is that it was learned by a deep neural net by reinforcement
learning without explicit supervision, i.e. without anyone telling the agent
"learn this rule".

That's a very narrow, procrustean, definition of intuition. Is it really what
most people would mean by "intuition"? Is it even close? Who knows- nobody can
tell what most people would mean by "intuition". There's a dictionary
definition, but chances are most people would not know it by heart. So it's
very hard to even say "that's not what intuition means"\- one would just be
begging the question.

So why use such a vague term in an article like this, that is already pretty
impressive stuff? What do such meaningless claims add to the result? I don't
get it.

~~~
bschne
I'm not up to date on the relevant research, but isn't most learned or innate
intuition just a set of fuzzy rules based on observation (be it visual or
other sensory perception; or pre-defined behaviour in the case of innate
intuition)?

If I play tennis for a while, I will get better at predicting where the ball
will go after my opponent hits it merely because I have seen many combinations
of shots and subsequent ball trajectories.

I think it would be weird to _not_ call it an intuition just because it was
learned by a deep neural net in this case.

~~~
YeGoblynQueenne
In my experience, in the context of artificial and human intelligence, any
statement of the form "X is just Y" or, equivalently, "Isn't X just Y" is
neither sound nor complete and X and Y are not well defined.

In your example, for instance- I have no idea what are "fuzzy rules" and how
they apply to tennis playing, or how they relate to predicting the position of
the ball etc.

~~~
x3ro
So in summary you are saying they used "intuition" wrongly, while subsequently
saying that it's really hard to say what intuition is objectively, and when
someone pointed out that their understanding of intuition does at least
partially match what is insinuated in the paper you continue saying "yeah but
when talking about AI all of this is really complicated, please be more
concrete"

I really have no idea what this is adding to the discussion, except for
talking down to the authors which, in your opinion, have misused a word that
you yourself find hard to define..

Edit: typo

~~~
YeGoblynQueenne
What I said was that it's pointless to use terms like "intuition" or "fuzzy
rules" that are not well defined.

That's because then anyone can claim anything they like in relation with those
terms, and nobody learns anything form the discussion, which is a complete
waste of time.

------
kuu
Super interesting!

