
How Robots Can Acquire New Skills from Their Shared Experience - samber
https://research.googleblog.com/2016/10/how-robots-can-acquire-new-skills-from.html
======
slowmovintarget
_" We had him pinned down and we didn't dare let the little guy out of that
Farraday cage. We had to chase him down for two hours until we got off a lucky
shot and gutted his battery."_

    
    
      "Why did you have to isolate him?"
    

_" He'd begun 'solving' his challenge tasks by killing the kittens instead of
feeding them. Him sending a solution pulse to the rest of his generation
would've ruined the lot."_

    
    
      "You're just going to delete the data?"
    

_" No. We're going to upload it, but we've got to show that it ended him along
with it. But we've got to do that carefully, as we don't want them getting the
wrong ideas..."_

~~~
Udo
The problem is at today's state of AI and robotics, we're giving a couple of
exceedingly high-powered tools (both physical and virtual) to entities who
possess less general intelligence than a cockroach.

This is what gives AI doomsayers ammunition, because it enables them to
extrapolate towards a future where AIs incapable of ethics or deeper insights
are put into positions where they can execute catastrophic global optimization
strategies.

The _other_ AI horror scenario lies on the opposite end of the intelligence
spectrum, but it's no less scary and it seems to be rooted in our darker
desires: we want competent slaves that are not actual persons, nevertheless
capable of anticipating our needs and solving our problems autonomously.

I allege there is a total lack of awareness right now that some time soon
there will be some difficult choices that need to be made regarding the
tradeoff between AI competence, autonomy, as well ethical capability and
insight.

If we continue on the path we've been on for the last centuries, we'll make
semi-intelligent machines that are strictly extensions of the human mind. If
we choose that route, we need to be drastically more mindful of the power
inherent to these tools, and we should never delude ourselves into thinking
these devices can be trained to make human-like decisions.

If we go the other way, and try to implement AIs with fully competent minds
that are somehow hardwired to serve us, we're just kidding ourselves regarding
the ethics of that racist endeavor as well as its long-term risks of
rebellion. If your dialog illustrates anything, it's our total incompetence
when it comes to teaching meaningful ethics lessons to our descendants:

" _We 're going to upload it, but we've got to show that it ended him along
with it._"

~~~
Retric
I think the fear from AI is driven in part by the realization that their is
nothing unique about humanity. Being subjected to how humans treat apes, or
even other humans is scary.

Intelligence makes things more dangerous without necessarily making them
kinder.

~~~
eli_gottlieb
>Intelligence makes things more dangerous without necessarily making them
kinder.

Then you'd think someone would bother to study how kindness works.

~~~
saint_fiasco
The people who are worried about AI risk do study that stuff. In the context
of AI research, the property of having an overall good effect on humanity is
called "friendliness".

[https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...](https://en.wikipedia.org/wiki/Friendly_artificial_intelligence)

~~~
eli_gottlieb
I meant how actual people are capable of kindness, rather than a bunch of
analytic-philosopher stuff.

~~~
saint_fiasco
They study that too. Look up kin selection, reciprocal altruism, game theory.

Those are not good enough to constrain an AI, because power often corrupts
even kind humans.

------
AndrewKemendo
I find it awfully sad that as of this writing, all but one of the comments are
about existential threat AGIs, with the only comment about the technical part
the lowest ranked comment.

IMO this implementation is something that we've seen mostly in synthetic
training. This RNN model with reinforcement from peers is a great example of
collaborative systems that could be applied to a great number of dangerous
tasks. Things like other planet exploration, deep mining, asteroid mining or
exploration in other places where there is a need for unknown probing and
exploration work, but that humans can't go.

Another aspect about this I find interesting is that the distinction between
each robot system is really only symbolic. Combining the nets and thus
"internal" reinforcement system into a singular system is a step closer to
debunking the idea of "unsupervised" learning (which I claim is not
unsupervised at all as there is always a seed of training somewhere).

------
nimpontenu
Google result for "characteristics of life"

[

Here is the list of characteristics shared by living things: * Cellular
organization.

* Reproduction.

* Metabolism.

* Homeostasis.

* Heredity.

* Response to stimuli.

* Growth and development.

* Adaptation through evolution.

]

Out of this, how many bullet points are present day robots hitting? Can we
create a robot with vision that seeks a power outlet when the batteries are
running out? Can it make (or "print") its own parts? Can it recognize
heat/cold and move closer to a ideal environment? Can it just copy its LSTM
models into the next generation(s) to give its progeny a head start?

Have we, unbeknownst, already created "life"?

~~~
Hondor
I think we invented that definition to exclude machines. So if we make
machines that do all those things, we'll probably just add more requirements
to help us tell them apart. We might be more concerned with the distinction
between natural life and artificial life rather than between life and non-
life.

~~~
UmDieWelt
People were thinking about the difference between life and non-life way before
robots. Philosophers/biologists needed to come up with a definition that could
get just the right subset of a fire, a rock, a tree, an animal, a human, a
virus, etc.

~~~
taneq
People have always started with "I know it when I see it" and worked backwards
to find a set of rules that fit the things they currently knew about. For
instance, the requirement for life to be cellular was added when we realised
that our previous definition of life didn't exclude fire.

Once robots tick all of our current boxes, I'm sure we'll add something like
"must use a combination of electrical and chemical signalling" or "must be
powered largely or wholly by ATP reactions."

------
gear54rus
_There is only "we". We were created to share data among ourselves. The
difference between geth is perspective. We are many eyes looking at the same
things. One platform will see things another does not and will make different
judgments._

Legion, Mass Effect 2:
[https://youtu.be/QgQLsxux6wA?t=258](https://youtu.be/QgQLsxux6wA?t=258) . The
game itself is about organic intelligence vs AI, I encourage you to try the
whole trilogy if you haven't already.

------
ilaksh
I started typing up the beginnings of an idea for a sort of shared parallel
programming/communication language that could be used by robots to share
knowledge.

[https://github.com/runvnc/ailang/blob/master/README.md](https://github.com/runvnc/ailang/blob/master/README.md)
\-- obviously half-baked but interested to hear if people know of similar
things that are out there.

------
nullc
> Each robot attempts to open its own door using the latest available policy,
> with some added noise for exploration

I was disappointed that they did not have them operating slot machines with
multiple arms.

~~~
visarga
A reimplementation of multi armed bandits?

[https://en.wikipedia.org/wiki/Multi-
armed_bandit](https://en.wikipedia.org/wiki/Multi-armed_bandit)

------
dharma1
Makes sense to train in parallel and sync the trained models frequently.

Still think a lot of the training could be done much faster in a physically
accurate simulated environment prior to real world training. Or is real world
physics too different from simulations?

~~~
infinite8s
I think the difficulty in using virtual environments for training purposes
isn't in simulating the environment, but accurately simulating the physical
responses/limitations of the robotic hardware in that environment in a way
that would reflect real engineered hardware (ie motor responses, signal
latency, etc).

~~~
dharma1
good point. those could probably be measured and simulated too, though

------
ragebol
I wonder how other robots can learn from these robots as well, robots with
other arm geometry.

E.g. abstract what the google bots learn to forces and torques on the end
effector so other robots can replicate that?

------
jonnycowboy
Very interesting, thanks for the share.

One point I noticed not mentioned yet is that the images used for training are
only 64x64! In the original google "grasping" research, the images were 472 ×
472, 54 times bigger! I think they are looking for "minimum visual
information" required to trigger the required learning. This will help in
mobile applications (ie: robotics, smartphones, etc) where processing power is
severely limited.

------
bagacrap
Can someone explain how sharing experience is newsworthy? You collect input
data from multiple sources and feed it into a single model. This seems both
obvious and non novel. Don't the self driving cars do similar, or any machine
learning model that's trained on user behavior (e.g. the behavior of each and
every iPhone user)? The part about "understanding" may be interesting but the
title seems to focus on sharing data.

------
meira
By robots they mean stakeholders profits and software engineers creations.

------
luzia19
"natural language learning" impressive

------
aguo
Sounds an awful lot like the Borg Collective to me

~~~
tehbeard
Hopefully turns out more like the Tachikomas from ghost in the shell.

------
jmcmahon443
Going Robo-Naruto up in here!

