> I suspect we’re at least five years away from those rather ordinary capabilities in robots.
Maybe. Here's a robot at Berkeley folding towels in 2010.[1] A Willow Garage robot folding jeans in 2012.[2]
Foldimate in 2017.[3] Even boring old Chicago Dryer had this working by 2021.[4]
They're all really slow. That's because they have no understanding of dynamics. The item has to come to a full stop between operations. Chicago Dryer got past that problem with a sequence of steps at different stations, each station taking about one second. That yields a useful commercial machine for large laundries.
It's just a demo problem, though. The approach is interesting. They're trying to use LLM technology on a completely different kind of problem. For that, you need a lot of training data.
If you're going to do things in this way, you need data from the inside of doing it. That's hard to acquire, but not impossible. They claim to use "robotic training data". Not sure if this is from robots being operated by humans as teleoperators. Others have tried that. There's a Stanford project that looks very similar.
Something like this has been used to train quadrotor drone controllers.
There's no obvious cheap way to acquire lots of data of this type, though. You have to run your own experimental setup and log.
Motion tracking from vision on a squirrel colony would be interesting as a data source. Squirrels are very agile and easy to observe. Then run the skeleton movement data back through a simulator and try to extract the forces the muscles are exerting. Now you have something usable for training an agile robot. Maybe sports videos could be used for training.
Maybe. Here's a robot at Berkeley folding towels in 2010.[1] A Willow Garage robot folding jeans in 2012.[2] Foldimate in 2017.[3] Even boring old Chicago Dryer had this working by 2021.[4]
They're all really slow. That's because they have no understanding of dynamics. The item has to come to a full stop between operations. Chicago Dryer got past that problem with a sequence of steps at different stations, each station taking about one second. That yields a useful commercial machine for large laundries.
It's just a demo problem, though. The approach is interesting. They're trying to use LLM technology on a completely different kind of problem. For that, you need a lot of training data.
If you're going to do things in this way, you need data from the inside of doing it. That's hard to acquire, but not impossible. They claim to use "robotic training data". Not sure if this is from robots being operated by humans as teleoperators. Others have tried that. There's a Stanford project that looks very similar. Something like this has been used to train quadrotor drone controllers. There's no obvious cheap way to acquire lots of data of this type, though. You have to run your own experimental setup and log.
Motion tracking from vision on a squirrel colony would be interesting as a data source. Squirrels are very agile and easy to observe. Then run the skeleton movement data back through a simulator and try to extract the forces the muscles are exerting. Now you have something usable for training an agile robot. Maybe sports videos could be used for training.
[1] https://people.eecs.berkeley.edu/~pabbeel/papers/Maitin-Shep...
[2] https://www.youtube.com/watch?v=MOtbcYE4Z4o
[3] https://www.youtube.com/watch?v=C76osXtpLeM
[4] https://www.youtube.com/watch?v=YpTuwKu5fY0