
Closing the Simulation-to-Reality Gap for Deep Robotic Learning - indescions_2017
http://research.googleblog.com/2017/10/closing-simulation-to-reality-gap-for.html
======
nobbis
Trust Google to look at their c. 1993 gouraud shaded, untextured renderings
without shadows and conclude "let's use deep learning to make it look
realistic!"

~~~
tehsauce
This seems like an unreasonable approach given the quality of their graphics.
They might as well train an GAN to replace the entire rendering pipeline if
they are using just simple images! Train it to generate images directly from
pose information. They really should focus on improving their rendering beyond
a 1993 level if they want a robust solution.

~~~
candiodari
That is too indirect a problem. Neural networks will find answers to problems
where there is a relatively direct hierarchical connection between input and
output. Symbolic information would require different and very big
architectures. Also doing that is still state-of-the-art and you don't try to
solve 2 state-of-the-art problems at the same time.

Firstly, using the better rendering approach has been tried and failed. These
networks exploit things that perfect rendering has but reality doesn't. For
instance, in a rendered simulated environment it is perfectly reasonable to
estimate exactly where an edge is and then follow that edge with zero margin
with the robot. In reality you need a minimum margin if you expect it to work
(or your object will be damaged), and a decent one.

Photorealistic rendering tries to emulate optics that are incredibly good,
with no distortion, and the robots have cheap webcams with plastic lenses,
with both consistent distortion near the edges and random distortions
everywhere because they're webcams. Not supercheap ones, but this isn't a RED
camera either. You could fix those lenses, but why would you ? It would be
more valuable if it works with cheap lenses, and human and animal eyes don't
really try to be distortion free. Rather they have the "software" behind it
compensate.

Second, photorealistic rendering requires modeling accuracy, and material
modeling on a scale we just can't do for real datasets. These datasets have
millions of objects, and to make the robot more tolerant of the general
weirdness of the real world, they are randomly perturbed, to turn millions of
examples to billions. But those perturbed objects are not photorealistic
anymore. Nobody has a toothbrush with a triangle in the bottom sticking out to
one side.

Besides, keep in mind that the resolutions used here are quite small. It's not
like they're trying to run a renderer at 4k resolution. Think 128x128, or
256x256. Not exactly a big issue.

~~~
nobbis
Photorealism (or "better rendering") doesn't mean perfect imaging. You can
model the radial distortion, rolling shutter, Bayer filtering of cheap webcams
in 5 lines of code.

A better simulation is closer to reality, not closer to an ideal.

