
Using A Kalman Filter To Make Sense Of Noisy Data - jack7890
http://seatgeek.com/blog/dev/using-a-kalman-filter-to-predict-ticket-prices
======
sytelus
If you want to understand Kalman filter in simplest term, check out the
article at [http://credentiality2.blogspot.com/2010/08/simple-kalman-
fil...](http://credentiality2.blogspot.com/2010/08/simple-kalman-filter-
example.html)

------
dredmorbius
Yet another blog which, if visited without JS enabled, displays a rather
disconcertingly unrelated page (ticket search).

Noscript -> seatgeek.com -> Temp

(For anyone else confused by the apparent topic/content discrepancy).

And for those interested in a more readable technical explanation, there's the
Wikipedia article (linked from the seatgeek blog):
<https://duckduckgo.com/lite>

~~~
jack7890
Good point, we will absolutely fix this.

If you don't mind me asking, why are you visiting websites without JS?

~~~
bhrgunatha
There are a variety of reasons:

* Safety - not everyone protects their website from XSS attacks

* Speed - Downloading 30 or more script files from all over the planet slows down browsing

* Tracking/privacy - it doesn't make your browsing private, it cuts down the more obvious things - <https://www.youtube.com/watch?v=f_f5wNw-2c0>

* Principle - browsers render HTML - nobody should be forced to require javascript ONLY to read the content or just to have access to the correct page. If you have fancy effects or some additional functionality or if it's required to provide core functionality (e.g. maps) then fine, but JUST rendering text?

------
jack7890
If there are other folks interested in using this approach, drop us a line.
We're happy to try to help other startups make sense of their data.

~~~
jianshen
I was working with Kalman filters for a hardware project and had serious
trouble wrapping my head around the topic.

I ended up confusing a Kalman Filter with a plain old Low Pass Filter at first
(and you can reduce a Kalman filter to that if you don't have enough inputs)
but it really is quite a powerful tool.

It's neat to see it applied to a different problem that might make it easier
for novices (like myself) to understand. Thanks for posting!

~~~
psykotic
Kalman filters are a simple idea when viewed from the right angle, at least
for someone with a good grasp of linear algebra.

You have a linear dynamical system with normal-distributed dynamical noise.
You then take linear measurements of that system over time, and again those
measurements are subject to noise that is normally distributed. The question
is then, given the series of measurements, what is the best estimate of the
state of the dynamical system? When written out like this, it's just a least
squares problem. What makes it efficiently solvable is that the matrix
structure is block tridiagonal. If you apply the block version of Gaussian
elimination to that block tridiagonal system, you get the Kalman filtering
equations. That's all there is to it.

~~~
plafl
The best introduction I have seen so far is the Udacity course on robotic
cars, although it touches very slightly the Kalman filter. I think the
exposition, starting with the general case of bayesian filtering and then
deriving from it several types of filters is very clear.

If you want to dig more I think the best book, by far, is "Applied Optimal
Estimation" by Arthur Gelb. (I have not read yet Probabilistic Robotics,
though).

Finally, a finer point of Kalman filtering which is not normally mentioned is
that you don't need your distributions to be gaussian. If your distributions
are gaussian the Kalman filter is optimal, if they are not gaussian then the
Kalman filter is not generally optimal, but it is still the best linear
filter.

------
essayist
Useful.

I'd love to find FAAS - Kalman (and other) filtering as a service.

For instance: I run daily backups on various databases. I expect the backup
size to increase roughly linearly, but I'm just going to look in on the
backups at random, likely ignoring them for months at a time.

It'd be great to be able to run the backup size series through a filter that
would alert me when something unexpected happened, e.g. slope changes
significantly or some unusual step change.

------
jnazario
great post, and thanks for sharing it. i came across the kalman filter after
developing an algorithm to detect worm propagation. it applies equally well in
that scenario, basically predictions.

------
derekja
Very nice post, thanks!

