
Gaussian Processes Are Not So Fancy - ajay-d
https://planspace.org/20181226-gaussian_processes_are_not_so_fancy/
======
btown
For anyone wanting to dig deeper, GPML is the bible on Gaussian Processes:
[http://www.gaussianprocess.org/gpml/chapters/](http://www.gaussianprocess.org/gpml/chapters/)

GP's are awesome. Usually, if you've got a few points, your first approach to
"fitting a curve through them" would be to choose some parametric form and
hope that you're not overfitting with e.g. too high a degree of polynomial.
But what if you've got something piecewise, with unpredictable pieces? And
what if you don't need a writable parametric form for your curve, but all you
need is to answer the question "given my data, what's the probability
distribution over the y value at x=5, 6, 7, 8, weighted over all likely curves
that might fit my data based on how well they fit?"

Then a GP fit on your data will work wonders for you, as essentially an oracle
for those kinds of queries. (And you can then just use the means of those
distributions sampled at small intervals, if all you want is to throw a single
curve on the screen.) Depending on your choices of priors and kernels, you can
have it do magical things.

------
ajschumacher
Thanks! I wrote that post! Another friend pointed out yesterday this other
post about Gaussian Processes: [https://www.jgoertler.com/visual-exploration-
gaussian-proces...](https://www.jgoertler.com/visual-exploration-gaussian-
processes/) I think that post has some fun visualizations for showing how
different kernels work, but I tend to prefer my explanation. Would love to get
more eyes on it and feedback specifically about whether I have any mistakes in
there!

~~~
plouffy
Hi,

Great post. A little typo I found was in the covariance equation (copy pasted
when you copy-pasted the equation), there's an extra bracket at the end.

I loved the sentiment at the end:

A Gaussian Process might be useful for you. But please don't assume that it is
sophisticated just because the language around it often is, or that its
results are automatically true just because they have error bars.

~~~
ajschumacher
Thanks! Really appreciate the feedback, and thanks especially for the typo
catch! It should be fixed now!

------
jacobolus
_

~~~
jey
No, that's just a common choice. You can choose a covariance function (kernel)
corresponding to some other function space.

[http://www.gaussianprocess.org/gpml/chapters/RW4.pdf](http://www.gaussianprocess.org/gpml/chapters/RW4.pdf)

