
Kalman filters and functional programming - edwintorok
https://www.johndcook.com/blog/2016/07/14/kalman-filters-and-functional-programming/
======
pjc50
Kalman filters are important in advanced control systems. One of the things
the Apollo computer system did was run Kalman filtering on trajectory data, in
order to continuously verify its course.

[https://www.technologyreview.com/s/602287/how-an-inventor-
yo...](https://www.technologyreview.com/s/602287/how-an-inventor-youve-
probably-never-heard-of-shaped-the-modern-world/?set=602297)

[https://pdfs.semanticscholar.org/b7da/dbff2c53bc30bd910fc0db...](https://pdfs.semanticscholar.org/b7da/dbff2c53bc30bd910fc0db00e5071a37acfd.pdf)

------
mturmon
The first paper linked (page 5 of
[http://vixra.org/pdf/1606.0328v1.pdf](http://vixra.org/pdf/1606.0328v1.pdf))
makes a cute connection that was new to me.

It starts with the familiar formula for on-line computation of the sample mean
(e.g.,
[https://en.wikipedia.org/wiki/Algorithms_for_calculating_var...](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm))
--

    
    
      mu_n <- mu_n-1 + (1/n) (z_n - mu_n-1)
    

This corrects the mean "mu" based on the new observation "z_n".

The paper then uses this simple expression to motivate the Kalman filter
update equation, in which a new observation z_n is used to compute an
innovation (or error signal) y:

    
    
      y = z_n - H x_n-1
    

The innovation is then used to update the state-estimate x,

    
    
      x_n = x_n-1 + K y
    

The two updates are formally identical, with the Kalman state "x" being
analogous to the running-mean "mu". I had never noticed the connection. There
is a similar connection to the on-line variance.

------
Seanny123
Great. I have another form of mathematics that I need to learn. I'll put it on
the pile on top of geometrical algebra and stochastic processes.

~~~
astine
Sometimes I wish I could quit my job for a few years and just catch up on all
of the math I'd like to know. There is an amazing amount of stuff to learn
that I just have no clue about and ultimately is much more interesting than
yet another HR tool.

~~~
moron4hire
My advice is to just start a side programming project where you _have_ to
learn the concept. I think it's a much, much faster way of learning, to have
to implement the technique, than to go through textbooks and exercises. I
don't think most people who aren't career mathematicians will truly understand
linear algebra until they write a few Matrix and Vector libraries in support
of some other thing they are deeply interested in building.

~~~
cossatot
Seconded. The slightly better option, if you can get away with it, is working
it into a project at work (rather than a side project) so you can devote a few
of your fresh-brained 9-5 hours to it each week and not feel too guilty.

------
taeric
This looks really interesting. The idea of splitting the accumulator from the
data seems a clear win. Would love to see evidence confirming it works.

~~~
davidrupp
Brian Beckman works for Amazon Prime Air (drone delivery service)[1]. I submit
the video "First Prime Air Delivery"[2] as circumstantial evidence that it
does indeed work.

[1] [https://twitter.com/lorentzframe](https://twitter.com/lorentzframe) [2]
[https://www.amazon.com/Amazon-Prime-
Air/b?ie=UTF8&node=80377...](https://www.amazon.com/Amazon-Prime-
Air/b?ie=UTF8&node=8037720011)

~~~
taeric
I'm not sure I follow. Would you let me submit as evidence an old piece of
software that is littered with GOTOs all over the place as evidence that that
is a great way to write software?

I fully believe this can work. Especially with a talented practitioner.
However, the question is if it will lead to more maintainable (e.g., by
needing fewer iterations?) and faster (e.g., by lots of people using it)
output with fewer bugs (e.g., by surviving the lindy effect).

That is, my question is entirely empirical and will hinge on repeated success.
Not initial success. I am personally hopeful for it, but I have reasons not to
trust my hopes.

------
hota_mazi
I wish the various papers linked in that page used a different language than
Wolfram Alpha to illustrate the various concepts, especially since that
language doesn't support even basic functional constructs such as pattern
matching.

~~~
krastanov
I am very confused by this comment. While I do find the Wolfram language
difficult to read and generally obfuscated, it definitely has a ton of
advanced functional constructs. If anything, simple pattern matching is the
main way I used to write code in Mathematica.

Among the typical "scientific computing" languages, Wolfram is the only one
that has "cool" functional programming tricks as the standard way to solve a
problem in it.

1:
[https://reference.wolfram.com/language/guide/RulesAndPattern...](https://reference.wolfram.com/language/guide/RulesAndPatterns.html)

P.S. Seriously! The amazing functional style of Mathematica is pretty much the
one thing I like about it.

------
pvitz
As those papers are hosted on vixra.org, how credible is the content? Has
somebody with deeper knowledge on the subject had a look into it?

~~~
Confusion
How about you go through a tiny bit of trouble and figure out for yourself
that John Cook is legit?

~~~
pvitz
The papers are written by Brian Beckman as you have found out by yourself
certainly. However, this doesn't answer my question.

------
ww2
Kalman filter is a natural example of iterative algorithm. Writing it in Fold
does not gain much.

------
digler999
glad to see this guy on here again. I hadn't seen him on HN in a while and I
forgot his name so I couldn't google. he's great at explaining advanced math
concepts.

~~~
edwintorok
You can add his blog to your RSS feed, or bookmark the articles you like and
tag them (I do both).

In fact I think bookmarks are underrated, I tried various other ways of
organizing information (Zim, Org, etc.) and bookmarks are by far the simplest
and most effective. For example just tag this blogpost with 'math, concepts'.
Then you just type 'math concepts ' in your Firefox address bar and see all
related pages. If you setup firefox sync then you can bookmark pages (even
from other apps by sharing the URL with firefox and using the 'add bookmark'
action), and tag it later on the desktop.

------
visarga
So, is it like error compensation in interpreting sensor data?

~~~
dbcurtis
It does sensor fusion. The google term you want is "optimal estimation".

A Kalman filter isnt really a filter. It keeps a state estimate of a system.
The system state is updated on a regular basis using control inputs. That runs
open loop, with a model of the drift gradually reducing your confidence in the
open loop estimate. As sensor inputs arrive, they are incorporated in the
state to refine the state estimate. The sensor noise model is your confidence
in the measurement.

The Baysian arithmetic comes into play when incorporating measurements. The
confidence in current estimate and confidence sensor are used to decide how
much of each to average into the new estimate.

~~~
abstrakraft
Ah, but it is a filter - it filters out high frequency noise. One
interpretation of the Kalman filter is a variable cutoff low-pass filter. The
cutoff is determined by the Kalman gain. Note the similarity between the
Kalman state update equation and a first-order low-pass IIR filter:

x_post = (1-KH)x_prior + K*z

~~~
dbcurtis
Well, yeah. It has a smoothing effect on the sensor data. But you don't use it
because you want a filter. You use it because you want to build a controller.

