
An Introduction to Mathematical Optimal Control Theory [pdf] - mindcrime
https://math.berkeley.edu/~evans/control.course.pdf
======
wenc
A note about this: much of the optimal control theory work surrounding
Pontrayagin's Principle are theoretical building blocks whose primary utility
is for mathematical analysis.

In practice, they result in boundary-value problems that are impractical to
solve for systems of non-trivial size, and are almost never implemented in
practice. Occasionally they are used to construct parameterized solutions for
extremum control (e.g. NCO tracking) for very small systems, but these tend to
be rarer cases. One runs into dimensionality issues very quickly.

In industrial control systems, optimal control models are almost always
discretized and the optimization is done on algebraic systems of equations.
Linear algebra dominates there.

~~~
westoncb
> ... are theoretical building blocks whose primary utility is for
> mathematical analysis.

When you say 'mathematical analysis' here, is that just a broader category
that encompasses e.g. real and complex analysis? Or something else?

~~~
wenc
Yes, that is what I meant [1]. For example, you can use ideas from
differential equations/analysis to determine say, the existence and uniqueness
of solutions for continuous ODEs. Pontryagin's Principle and the calculus of
variations in general gives you theoretical machinery for working with models
in analytic form. Some problems such as minimum time optimization are more
tractable in continuous time form than in discrete time. You can also in some
simple cases derive the set of closed-form optimal solution trajectories
(unconstrained case) and analyze that directly.

Once the models are transformed into discrete form for numerical solution, the
tools used lie more in the realm of linear algebra (positive definiteness of
Hessians, etc.)

[1]
[https://en.wikipedia.org/wiki/Mathematical_analysis](https://en.wikipedia.org/wiki/Mathematical_analysis)

------
CrazyCatDog
Craig Evans (the author) is the most selfless mathematician I’ve ever studied
under—-hands down a life-changing teacher. These notes, as are all his
teaching notes, are magnificent.

------
foxes
>Those comments explain how to reformulate the Pontryagin Maximum Principle
for abnormal problems.

Please tell? I have yet to see anyone give a satisfactory approach on how to
deal with the abnormal case.

For context there are sometimes optimal solutions which are not given by
Pontryagin's Maximum Principle (PMP). An analogous situation can occur with
Lagrange multipliers. The necessary conditions given by the Lagrange
multipliers are not related to the maximization of the object functional. I
surely think the situation is worse with the PMP because you are now in a
continuous setting. I think [0] offers some good discussion for the abnormal
case in Lagrange multipliers.

I would be interested if anyone has made any recent progress in dealing with
the abnormal case for the PMP.

[0] Optimality Conditions: Abnormal and Degenerate Problems By A.V. Arutyunov

------
mindcrime
Trying to dig into Optimal Control Theory a bit, after realizing that - in
many ways - OCT and (certain aspects of) Machine Learning are just opposite
sides of the same coin. Reinforcment Learning in particular shares a lot of
concepts with OCT.

~~~
rocketflumes
for more on that subject - check out this recent RL and OCT survey by Ben
Recht, also from UC Berkeley:
[https://arxiv.org/abs/1806.09460](https://arxiv.org/abs/1806.09460)

~~~
pjrule
Ben Recht also has an excellent series of blog posts (very related to this
survey on arXiv, but broader) on the intersection between reinforcement
learning and optimal control. An index is available here:
[http://www.argmin.net/2018/06/25/outsider-
rl/](http://www.argmin.net/2018/06/25/outsider-rl/)

~~~
mindcrime
I was just reading those last night. Definitely good stuff.

------
jkabrg
Is this a very crude summary of Pontryagin's principle? Basically, you use
Lagrange multipliers to solve a constrained optimization.

In contrast, dynamic programming is based on stitching together optimal sub-
solutions.

