
How robots are grasping the art of gripping - lainon
https://www.nature.com/articles/d41586-018-05093-1
======
dragontamer
Its always funny to realize how "easy" beating human-intelligence is (Chess
AI, Go AI, even Mathematical Proofs), but how hard beating human-simple
behaviors are.

IE: Stand up on two legs, walk forward, and move these 10 objects from this
bin to that bin without breaking any of those objects. Or fold these clothes
(surprisingly difficult to get a robot to do that).

We can build super-advanced Chess and Go AIs that beat out the grand summation
of theoretical human knowledge (with Chess theory going back hundreds of
years, and Go theory going back maybe thousands!!), but we still can't move a
variety of objects from one bin to another successfully and repeatably.

\-----------

Another funny example: AIs (or various algorithms at least) have solved the
object camera tracking problem, but still basically fail at depth perception
and figuring out if something is "still" or "moving" based on sight alone.

~~~
yathern
A fun way to think about this is by analyzing the 'training-time' for these
tasks.

The things we think are easy, we've been trained to do through millions of
years of evolution. Running with two feet through a forest is pretty easy for
us. Not so much so for computers.

The things that we think are mentally taxing, we've only been doing for a few
thousand years or so.

~~~
abhiminator
>The things that we think are mentally taxing, we've only been doing for a few
thousand years or so.

It's kind of fascinatingly serendipitous in the sense that computers/AI are
super quick at performing the tasks that we humans find extremely taxing
(complex mathematical computation, visual simulation) while struggling with
tasks that comes naturally to us humans (gripping objects, running with two
feet--as you described) owing to evolution.

I wonder how much of that is because of constituent 'building block' materials
that make up the basic structure of homo sapiens and computers -- carbon and
silicon, respectively.

------
Animats
That's not a very good article on gripping and manipulation. Nature was once a
serious scientific publication, not PR Newswire.

The state of the art is still bad, and it's been bad for a long time. Watch
these two videos: 1960s: [1] 2012: [2] Note how little progress there has
been.

Here's the actual winner of the 2018 Amazon picking challenge.[3] The system
mentioned in the article did not win.

Throwing deep learning at the problem helps, but not all that much. It's
amazing how hard this is.

[1]
[https://archive.org/details/sailfilm_pump](https://archive.org/details/sailfilm_pump)
[2]
[https://www.youtube.com/watch?v=jeABMoYJGEU](https://www.youtube.com/watch?v=jeABMoYJGEU)
[3]
[https://www.youtube.com/watch?v=AljePt7Mh6U](https://www.youtube.com/watch?v=AljePt7Mh6U)

~~~
psychometry
This is a Nature Briefing, not an article/letter. Nature is still a serious
scientific publication, on par with Science and NEJM.

~~~
Animats
Live by the brand extension, die by the brand extension. They put their name
on it, they have to take the hit for it being crap.

~~~
psychometry
Christ... It's not a "brand extension" (whatever that means); it's a blog post
intended for the general public, not researchers. If you want research, look
at the journal archives.

~~~
Animats
A "brand extension" means you take your famous name with a good reputation and
paste it on something else to boost the new thing. This risks the reputation
of the famous brand if the new thing is a dud. AdWeek has a list of
embarrassing brand extension flops.[1]

Even if it's not a total flop, it can degrade the brand. Holiday Inn Express
and Hilton's low-end product lines are examples. The overall result was to
move the brand downscale.

[1] [https://www.adweek.com/brand-marketing/best-and-worst-
brand-...](https://www.adweek.com/brand-marketing/best-and-worst-brand-
extensions-146966/)

~~~
psychometry
Great, thanks. Anyway, I assure you that the journal Nature will not suffer
any loss of status by having blog posts summarizing recent research for
laypeople.

------
harperlee
My pet theory is that this is going to be way easier when tactile/pressure
sensors drop their cost/resolution by orders of magnitude.

Animals basically have a feedback loop that works a little bit with
proprioception / position measures and way more with expectations of tactile
sense (to see this, reflect on how you move in the dark: not thinking about
your joint angles, but about that not to kick!).

As a hobbyist, for example, there is no way I can have a "video" of pressure
maps on an artificial skin, so I can only hack my way with a one-dimensional
finger-mounted pressure sensor. How on earth can I program a robot to use his
hand only by looking at it? I would magine that most researchers are in a
similar way restricted to work with subpar feedback loops.

~~~
candiodari
While it's true that animals, including humans have a long list of tricks
built into their bodies to increase their chances of a correct grasp, to
enable force application in the first place, and in some cases to deliberately
damage whatever they're grasping (thus wounding and/or killing prey), I think
you'll find that the dexterity required to jump a cat body at all is very,
very hard.

Moving a cat body, at 130 kph, on an angle that will take the front claw (they
always use the same one) across the throat of a wildly jumping around animal
(with enough force to allow the claw to rip it open) while keeping that same
cat body away from any hooves (one hit from even deer hooves will at the very
least end the hunt of any puma, and can do damage up to killing it) is ...
it's not just ridiculously hard. It's almost absurd what sort of control you'd
need.

I'm also not very sure how tactile sensors help, even if they could not just
register contacts but air speeds as well, that does not seem like it would
make the problem much easier.

Plus cats are a very "general" body. The number of ways it can move, because
it lacks a fully connected inner bone structure (so almost any cat joint can
move remarkably far in almost any direction). I'm pretty sure this adds insult
to injury and makes it a lot harder yet again.

------
joe_the_user
Well, it has taken years but seems like Boston Dynamics and similar companies
have moderately agile walking robots. Yet somehow these don't seem to be
appearing anywhere.

Now we hear there's progress in gripping.

The thing about both these steps is walking and grasping are extremely
challenging tasks to robots but even more challenging is the easy integration
of these and other movements that humans achieve when doing "simple tasks" and
especially cooperating in simple tasks - in the "unstructured environments"
referenced in the article.

~~~
tlb
It's hard to find practical applications for walking. The vast majority of
environments in which mobile robots can be useful also have smooth floors.
Legs have higher cost, weight, failure rate, and power consumption than
wheels.

Better gripping, though, has immediate practical applications in manufacturing
and distribution. We're much more likely to see that in use in the near
future.

~~~
hugs
Next month I'll be going on a weeklong backpacking+camping trip. I don't
believe in the notion that modern camping is "roughing" it (tents and shoes
and cooking gear are essential tech). So on that note, I'd _love_ to have a
low-cost solar-rechargable "pack mule" robot that could carry more gear. (I'll
survive without it, of course, but it would be nice.) Wheels on a robot
wouldn't work where I'm going, but legs would.

~~~
sgillen
Yup, the military has this exact same use case, I think that's where all the
funding for Boston dynamics big dog (and probably others) came from in the
first place.

~~~
hugs
I'll know I'm living in the future when I can buy a BigDog at REI.

------
abhiminator
>"Some researchers are using machine learning to empower robots to
independently identify and work out how to grab objects. Others are improving
the hardware..."

Software is the main bottleneck in ML at the moment, imo. That and data. The
hardware problem (computation), which the author discusses briefly, has been
more or less solved in the industry as newer software require less
miniaturization of ICs and peripheral components[0] -- this despite Moore's
law effect wearing off. [1]

It's all about striking a fine balance between the hardware's raw
computational capability and designing compatible software nimble enough to
adapt rapidly to colossal influx of data.

[0] [https://www.forbes.com/sites/quora/2017/01/27/has-moores-
law...](https://www.forbes.com/sites/quora/2017/01/27/has-moores-law-held-up-
over-the-last-five-years/#10ed1c821d26) [1]
[https://www.technologyreview.com/s/601441/moores-law-is-
dead...](https://www.technologyreview.com/s/601441/moores-law-is-dead-now-
what/)

~~~
stcredzero
_Software is the main bottleneck in ML at the moment, imo. That and data. The
hardware problem, which the author discusses briefly, has been more or less
solved in the industry_

I'm only a casual long time observer, but I think there's an opportunity here
for haptic teleoperation. Data could be gathered from haptic teleoperators in
much the same way that Tesla is gathering data for self-driving AI. Back in
the 90's and early 2000's, Sarcos corp's website used to have a crappy
realmedia video of a full-arm haptic rig operating an over-sized hydraulic arm
casually holding an anvil like a beer mug. So evidently, we've been able to do
full-body haptics and human-level agility manipulators for quite awhile.
(Research in that stuff actually started in the 1970's!) On-orbit
teleoperation from the ground could have interesting applications.

~~~
greglindahl
NASA appears to do quite a bit of on-orbit teleoperation from the ground with
the arm on the ISS.

~~~
stcredzero
The arm on the ISS functions more like a mini space-crane and less like an arm
on a worker.

------
bytematic
Interesting how we have designed the world for humans and now the optimal way
for many robots to work is to be designed like humans. I always hoped for some
more optimal design, maybe even something that can morph.

~~~
joe_the_user
Well, it's natural for humans to design the world for humans (though a good
part of it now is designed for humans inside machines, ie cars).

Note, however, that most robots today actually operate in spaces designed for
machines - ie, factories. Getting robots to operate in the "unstructured
spaces" the article mentions is extremely difficult and hasn't been cost
effective.

A large amount of gains in productivity have been achieved by substituting
machines for humans, with "robots" just being the most programmable of
machines. But very few of those gains involve direct substitution of machine
movement for human movement - rather it's involved imposing structure on the
whole productive environment so the rigid motion of a machine has a
predictable result.

~~~
GabeWeiss_
Also of note, the societal backlash of replacing humans with robots has been
pretty massive. So it's not just cost-effectiveness that's being balanced, but
care with the workforce not to implement too many replacements too fast. The
places where we've seen massive substitution are all industries where humans
were in physical danger from the work being done. Risky manufacturing lines,
etc.

I feel like that's starting to shift a bit where folks are realizing that more
automation by robotics isn't actually reducing the potential workforce, it's
just shifting it. Again though, it's up to the employers and manufacturing
companies to really drive that point home.

It's an exciting time. :)

~~~
joe_the_user
_The places where we 've seen massive substitution are all industries where
humans were in physical danger from the work being done. Risky manufacturing
lines, etc._

That doesn't really reflect the situation. Machines have been substituted for
humans since at least Eli Whitney's Cotton Gin. Substituting machines for
humans has had nothing to do with providing safety, rather it has been driven
by the desire for increased productivity and thereby profits. Certainly, this
increased has provided massive benefit to society along with various
drawbacks. That's literally the history of the "industrial revolution".

[https://en.wikipedia.org/wiki/Cotton_gin](https://en.wikipedia.org/wiki/Cotton_gin)

~~~
GabeWeiss_
No no, you misunderstand me. I'm not saying it's FOR human safety...it's for
profit. But the pushback on "losing jobs to machines" is less when the end-
result is replacing human jobs that have a high degree of danger associated
with them. Which is why, throughout history as well, we've seen larger
adoption of automation via machines in those areas. The friction to do that
replacement is lower, so we see it more often.

~~~
joe_the_user
_But the pushback on "losing jobs to machines" is less when the end-result is
replacing human jobs that have a high degree of danger associated with them._

Well, OK, except for the long history of industrial automation that doesn't
bear this out at all. There has resistance to automation from the English
Luddites to Detroit Black Workers Union and beyond and arguments that
automation is only OK safety is involved have not entered the picture that
entire period.

Basically, there's never been a situation where public complaints stopped
automation. It has proceeded as fast as technology allowed to the present.

EP Thompson is the reference for early automation and the luddites but it's
hard to give a reference for a negative.

If you have any reference for situation where public sentiment limited
automation, I'd love to see them. Because it sounds like you are mistakenly
thinking the world actually works according to the vague headlines one see
sees periodically.

------
carapace
> Many roboticists think that there is unlikely to be a universal solution to
> grasping.

Already found: "Universal robotic gripper based on the jamming of granular
material"
[http://www.pnas.org/content/107/44/18809](http://www.pnas.org/content/107/44/18809)

> Here we demonstrate a completely different approach to a universal gripper.
> Individual fingers are replaced by a single mass of granular material that,
> when pressed onto a target object, flows around it and conforms to its
> shape. Upon application of a vacuum the granular material contracts and
> hardens quickly to pinch and hold the object without requiring sensory
> feedback. We find that volume changes of less than 0.5% suffice to grip
> objects reliably and hold them with forces exceeding many times their
> weight.

~~~
glup
That's a cool appendage, but there are still a lot of problems in grasping /
planning with materials like fabrics, right? e.g. this doesn't solve the
notoriously hard laundry folding problem

~~~
ggm
Fingernails and Fingerprints. I can't fold laundry after I cut my talons, and
I can't fold laundry when the friction is absent.

Training a robot to fold laundry would be like training an astronaut to fold
laundry in Apollo era gloves (which btw, I have read were so bad, the lunar
walkers lost fingernails inside 'em)

------
mbym
“When you pick something like a pen up off a table, the first thing you touch
is the table.” I stopped reading for a good two minutes of experimentation.

------
YeGoblynQueenne
>> “The world is designed for anthropomorphic hands,” says Brock

If I may just interject, totally calmly like, the world was not designed, and
certainly not for our hands.

------
tedyoung
Did anyone else read this as the robots are grasping the art of "griping"? As
if we need them to learn how to complain. :)

~~~
sgillen
A robot being able to complain that something is very difficult for it (using
more energy for something than it needs to for example) would be really cool!

