
Ask HN: What are the modern approaches to robotic control? - bistable
I&#x27;m eager to find references on modern approaches to robotic control.<p>The coordinated robots from Boston Dynamics (acquired by Google) impress the most, with their ability to react quickly to perturbation and move competently over difficult terrain.<p>I assume they are not solving inverse kinematics problems continuously, and also assume that much of their progress was made before the deep learning revolution, so maybe they don&#x27;t even use neural networks either.<p>So are they using any reinforcement learning?  I have no idea so would appreciate any quick summaries or pointers to relevant information.<p>What other companies are working in robotics at the same level as Boston Dynamics?<p>Approaches to robot perception, path planning (e.g. A*), and environmental mapping (e.g. SLAM) are also very interesting, but not the topics I&#x27;m interested in for this question.
======
ModernMech
You say you're not interested in path planning, but path planning and control
are really on the same spectrum, which is motion planning. For instance, where
to next put big dog's leg is a control problem, but also a path planning
problem (how do I get the leg there? Where is the best place to put the leg?
Does the leg collide with anything?). If you can't answer these questions and
are blindly putting your leg wherever, you're not in control of anything.

Here's a good paper on RL in robotics: [http://www.ias.tu-
darmstadt.de/uploads/Publications/Kober_IJ...](http://www.ias.tu-
darmstadt.de/uploads/Publications/Kober_IJRR_2013.pdf)

RL is not widely used for control, but it has yielded some impressive results.
In my experience I had a highly dynamic system for which I built a hand-tuned
model for motion planning. I also built a RL model and trained it using the
hand-tuned model. The RL model performed more than 50% better than my very
best efforts.

Also, some of my favorite textbooks:

Principles of Robot Motion by Choset et. al.

Statistical Robotics by Fox, Burgard, and Thrun

Linear Systems Theory by Hespanha

~~~
antiuniverse
Not my field, but in case this link is helpful:

"Planning Algorithms" by Steven M. LaValle

[http://planning.cs.uiuc.edu/](http://planning.cs.uiuc.edu/)

>This book presents a unified treatment of many different kinds of planning
algorithms. The subject lies at the crossroads between robotics, control
theory, artificial intelligence, algorithms, and computer graphics. The
particular subjects covered include motion planning, discrete planning,
planning under uncertainty, sensor-based planning, visibility, decision-
theoretic planning, game theory, information spaces, reinforcement learning,
nonlinear systems, trajectory planning, nonholonomic planning, and kinodynamic
planning.

------
falcolas
For more generalized information on how you bring a machine from state A to
state B, you may want to read up on Control Systems [0], one of which is the
PID controller [1]. It's used in everything from industrial water heaters to
quadcopters. It wouldn't surprise me to see them also being used at some level
with more advanced robotics.

[0]
[https://en.wikipedia.org/wiki/Control_system](https://en.wikipedia.org/wiki/Control_system)
[1]
[https://en.wikipedia.org/wiki/PID_controller](https://en.wikipedia.org/wiki/PID_controller)

~~~
stevendhansen
PID is a generalized control algorithm that performs poorly unless your system
is entirely linear. It is overwhelmingly used in industry, however, because it
is simple, and anyone can drop in a PID block and adjust three constants to
get semi-OK response without understanding anything about the dynamics of the
system they are controlling.

For serious control systems you can look up state feedback control, model
predictive control, and nonlinear systems control (to name a few very broad
categories out of many possible options).

~~~
dwpdwpdwpdwpdwp
Yes, and I'm compelled to add that robots are typically actuated by servo
motors and/or hydraulics, which are very commonly under the control of PID
loops. Those PID loops allow the actuators to reach their set-points (whether
position, velocity, etc.). To obtain 'smooth' motion, those set-points are
usually streamed to the actuator at 60+ Hz, and it is the generation of those
set-points that can become very complex; the stream of set-points is not
commonly generated by the output of a PID controller.

~~~
stevendhansen
PID control is rarely used for professional servo applications, with the
exception being amateur and lower performance systems (radio controlled, etc).
This is especially true with PM servo motors because these machines are
inherently nonlinear and cross-coupled. PID doesn't deal with either of these
issues (nonlinearity or cross coupling) and therefore offers extremely limited
performance.

The industry standard update rate for servo systems is ~1kHz (although it
depends on the application), and I have seen systems with >5kHz torque
bandwidths. The torque dynamics associated with a typical PM machine used for
servo systems are easily in the 10's or 100's of microseconds, so 60Hz control
would not cut it.

~~~
dwpdwpdwpdwpdwp
Again, yes, but to clarify, the 60+ Hz I was referring to was not the servo
loop update rate, but the rate at which (loosely speaking) the servo loop
setpoint is updated. (1)

For example, KUKA robot arms can operate in a mode where a motion path is
planned and a sensor on a tool tip can make slight adjustments to the motion
path on the fly. The points on those motion paths (as well as corrections) are
updated every 4 or 12ms (83.3 to 250 Hz).

My point is that servo motor drives do indeed implement (sophisticated) PID
controllers for current, position, velocity control loops. (2)

Obligatory Defensive Writing:

(1) Typical industrial servo motors implement sophisticated control over the
velocity and acceleration profiles of the point to point moves. Anyone
interested can look up the DS402 standard and take a look at the motion
profile modes of operation.

(2) The PID loops implemented in servo drives are way more sophisticated than
the canonical PID control loop equation
([https://en.wikipedia.org/wiki/PID_controller](https://en.wikipedia.org/wiki/PID_controller)).
Nonetheless anyone who finds themselves manually tuning the servo loops for an
industrial servo motor will surely find themselves setting proportional and
integral gains.

Source: Commissioning servo motor drives is a part of my job.

~~~
stevendhansen
Just because controllers have proportional and integral terms does not mean
they are PID controllers!

I'm not just being pedantic here, there is a world of difference between a
modern servo controller and a simple (or advanced) PID controller. Good servo
controllers include completely different topologies including state feedback
decoupling, disturbance decoupling, state tracking feedforward terms, advanced
notch filters, sliding mode gains, and many other techniques. This is not just
an advanced PID loop, but a different controller design method altogether.

Source: I design servo (and other motor) controllers for a living

------
pumpikano
There is a surge of interest in learning control policies end-to-end. Many of
Sergey Levine's recent papers are relevant:
[http://homes.cs.washington.edu/~svlevine/](http://homes.cs.washington.edu/~svlevine/)
(there are also some talks linked there).

------
srlake
It's been a few years since studying in this area, but one approach we used to
use for modelling and control of manipulators (mostly industrial robotic arms)
was Screw Theory.

Here's a recent paper using one of these approaches:
[http://www.intechopen.com/books/international_journal_of_adv...](http://www.intechopen.com/books/international_journal_of_advanced_robotic_systems/improved-
inverse-kinematics-algorithm-using-screw-theory-for-a-six-dof-robot-
manipulator)

One approach that could be used for a system like Big Dog would be a feed-
forward control loop with kinematic/dynamic modelling of the robot. These
approaches use knowledge of the system dynamics to predict the output based on
changing inputs or disturbances.

------
glial
You'll really be missing out if you don't look into the work of Emo Todorov:

[http://homes.cs.washington.edu/~todorov/](http://homes.cs.washington.edu/~todorov/)

He's done some amazing work recently in helping solve problems surrounding
movement planning (not really the same as path planning like A*), so you can
tell a robot to do something general like "stand here", and it treats the
movement planning as an optimization problem and the dynamics of its 'body'
and physics as constraints. The resulting behavior is eerily lifelike.

------
sharemywin
they use something called "force control" which I think uses a spring to act
as some kind of tendon.

[http://news.nationalgeographic.com/news/2003/03/0311_030313_...](http://news.nationalgeographic.com/news/2003/03/0311_030313_dinobot.html)

[http://web.mit.edu/spotlight/archives/troody.html](http://web.mit.edu/spotlight/archives/troody.html)

Mentions some of the engineers at Boston Dymanics including the inventor of
troody. [https://prezi.com/tn5kji7ehk3u/prezi-
project/](https://prezi.com/tn5kji7ehk3u/prezi-project/)

~~~
sharemywin
more info here:

[http://news.mit.edu/2001/dinosaur](http://news.mit.edu/2001/dinosaur)

We use series elastic actuators, or springs connected to motors, and low-
stiffness control, so the joints are looser, and more biological."

HOW IT WORKS Troody, which weighs about ten pounds, has 16 joints and 36
sensors. "Every joint has a position and force sensor," Mr. Dilworth said. The
robot also has a vestibular system -- the equivalent of an inner ear -- that
it uses for balancing, and an onboard computer that automatically runs a
walking control algorithm.

~~~
gene-h
We use series elastic actuators not so the joints are more biological, but
mainly so that motors don't get creamed when the robot bumps into something
and so that the robot doesn't cream whatever it bumps into. The second reason
why we do this is to get force/torque control. Series elastic actuators
measure force or torque by measuring the deformation of a spring. You get
increased force bandwidth at the expense of decreased position bandwidth

------
npmanor
If you're interested in walking, the Florida Institute for Human and Machine
Cognition (ihmc: [http://robots.ihmc.us/](http://robots.ihmc.us/)) came in
second place at the DARPA Robotics Challenge using force control and capture
point dynamics. They do primarily software, but their walking is quite stable.

------
orasis
Sample efficient methods using Gaussian Processes are interesting.

[http://mlg.eng.cam.ac.uk/pilco/](http://mlg.eng.cam.ac.uk/pilco/)

------
dbcurtis
Google "underactuated"

~~~
jamessb
And see the MIT "Underactuated Robotics" book (Free online) [1] and associated
MITx course [2].

[1]:
[http://underactuated.csail.mit.edu/underactuated.html](http://underactuated.csail.mit.edu/underactuated.html)

[2]: [https://www.edx.org/course/underactuated-robotics-
mitx-6-832...](https://www.edx.org/course/underactuated-robotics-
mitx-6-832x-0)

