
Covariant.ai and applying deep learning to robotics - wojtczyk
https://www.indexventures.com/perspectives/rebirth-robotics-how-covariant-unlocks-power-deep-learning-robots/
======
xpe
Here are two example sentences that catch my attention:

1\. "Most robots these days make use of some form of Deep Learning." This is
not obvious. What is the basis for it?

2\. "Robots themselves have been around forever, but, with a few exceptions,
have been disappointments." In historical context, this is hardly true. Look
at assembly lines, at automation, just to start.

~~~
smilekzs
Venturing a guess, a possible way for #1 to hold true (in some sense) might be
the maturity of camera-based object detection and classification, which even
the most risk-averse industrial robot builders might have justified to adopt
in their latest offerings.

~~~
unishark
Maybe if they have a narrow definition of robot that requires it to have
computer vision, as opposed to just a machine that repeats a repetitive task
exactly.

~~~
ragebol
Most industrial robots are relatively simple machines that repeat a task
described in details. Only input may be to wait until a new part came in or to
move down the stack of parts until the gripper runs into something to grip.

------
xiaolingxiao
Just for some context from someone who is involved in robotics, both Google X
and Samsung Research have research teams working on robotics arms. I would
expect to see a lot more of these companies in the coming years, weaving a
narrative of RL ( currently getting hyped a lot in academia, again ) and
factory automation.

Manipulation is another task that appear deceptively simple, but is actually
very complex for machines, similar to autonomous driving. Personally, any
solution involving manipulation with _fingers_ cannot be viable. Thankfully
their approach appear to use a simple gripper. Most of their publication is
around general RL ([https://covariant.ai/our-
approach](https://covariant.ai/our-approach)). And again similar to AVs, the
sim to real gap is pretty big here too.

One good thing is that warehouses is a more constrained environment and can be
further structured around specific robots. And Amazon has internal robotics
teams and have deployed robotic arms in limited settings. It works there
because the entire warehouse is structured around robots, that's what it
takes.

~~~
skolman
Just curious, do you think OpenAI's approach
([https://openai.com/blog/solving-rubiks-
cube/](https://openai.com/blog/solving-rubiks-cube/)) will work? They were
able to use fingers in a non-trival setting. It has a long way to go before it
can be deployed in any useful capacity, but to me it challenges the idea that
robot fingers won't ever be viable.

~~~
xiaolingxiao
Interclass variation amongst rubik cubes is far lesser than intra-class
variation amongst different objects. I would put that in the same category as
dialogue systems generating responses using RL ( I've actually worked on this
), incidentally also on top of the page today. These things are mostly dog and
pony shows. RL's application in practice is limited, even for Covariant .ai
I'm not convinced that most of the paper they have posted is not just
marketing. Academics are quite good at playing this game, see this paper on
their site[1]. It has nothing to do w/ grippers. In practice once you touch
hardware, all that policy gradient goodiness goes out the window, hardware
considerations, domain specification, etc will dominate how good your solution
is. So "domain spec and design" means you need to work closely w/ your
customers, and have a say in how their warehouse is designed. Amazon doesn't
run into this issue because everything is done in house. But if you try to
deploy the same system at Walmart w/o strong institutional support, the system
will fail.

Thus companies such as this is a pump and flip play. There's a direct comp
that's Amazon's internal division, everyone is trying to copy Amazon's supply
chain efficiency these days, so the best outcome is strategic investment from
Walmart and such, and then followed by an acquisition.

You saw similar companies come out in the nascent days of the deep learning
hype, socher from Stanford comes into mind. His company MetaMind was sold to
Salesforce for xx million amount, all the engineers got a fine pay day, they
didn't really release a product. But they certainly published some nice papers
along the way.

[1]
[https://openreview.net/attachment?id=ByeWogStDS&name=origina...](https://openreview.net/attachment?id=ByeWogStDS&name=original_pdf)

~~~
KKKKkkkk1
They posted their double-blind submission on arxiv? Seems very shady. Is this
common in AI conferences?

[https://arxiv.org/abs/1906.05862](https://arxiv.org/abs/1906.05862)

~~~
xiaolingxiao
Yeah in CS ML it’s pretty common. Many papers are on arxiv before submission.
With changes in between.

------
krasin
Another impressive startup in this area is nomagic.ai. From what I know, they
are more advanced than covariant, had been in production for more than a year
and recently raised a decent Seed round.

Good luck to both teams!

~~~
kornish
Osaro and Dexterity have also both raised Series Bs to work on the same
problem.

------
Koshkin
Meta/off-topic: I think that for many people the use all these cool-sounding
"mathy" names - covariant, differential, tensor (flow), etc. may in fact be
more irritating and confusing than justified to any meaningful degree.

~~~
sgillen
What’s wrong with using mathy names to describe mathy products?

~~~
whatshisface
It's annoying when the products aren't what the names describe. It would be
like naming a brand of facial tissues "Motor Vehicle" or a brand of computers
"Apple." ;)

------
tmoney1818
From my research, maybe the most interesting characteristic is that all these
companies seem dependent on the sucker gripper. I haven't worked in this
field, but you'd think it'd be easy getting other gripper to work well,
especially since covariant is combining simulated training with non-simulated
training.

~~~
sgillen
I think that the communities experience has led them to the belief that the
suction gripper is simply the most effective gripper currently available. It’s
possible to get other grippers working in theory, but I think if you want
reliability suction on constrained packages is the way to go.

------
amelius
I was under the impression that reinforcement learning already tackled the
"picking" problem sufficiently well.

~~~
sgillen
I believe your impression is wrong. I believe that amazon shut down its
picking challenge in 2018 because it was clear to them the technology wasn’t
ready to replace humans in there distributions centers yet.

I’d be happy to be proven wrong though if you have examples.

------
canada_dry
> _the technology was shockingly advanced ... we were blown away_

This is a very impressive step on the road, but this kind of hyperbole always
sets off my _Segway early-warning-system_.

