
Numerical Optimization: Understanding L-BFGS - aria
http://aria42.com/blog/2014/12/understanding-lbfgs/
======
jamessb
This is a nice introduction, but it looks there is a problem with the
formatting: some of the math is not rendered properly (in the source, some
expressions are wrapped in dollar-signs rather than script tags).

~~~
aria
What browser are you in? It just using MathJax and appears to work in WebKit
and Firefox.

~~~
jamessb
Firefox 33.1.1 on Mac OS 10.10.1. Initially it would get stuck looking like:
[http://imgur.com/DhXAmI7](http://imgur.com/DhXAmI7)

After refreshing the page a few times, all the formulae rendered correctly. In
Safari things rendered correctly immediately.

------
howeman
Gonum has an LBFGS implementation
[https://github.com/gonum/optimize/blob/master/lbfgs.go](https://github.com/gonum/optimize/blob/master/lbfgs.go)

~~~
howeman
See the "NextDirection" method for the update.

------
aria
Author here, happy to answer any questions.

~~~
stephencanon
I hate to be that guy, but "Raphson", not "Rhapson"
([http://en.wikipedia.org/wiki/Joseph_Raphson](http://en.wikipedia.org/wiki/Joseph_Raphson)).
It's also worth noting that no one _ever_ actually forms the inverse of H,
even if H is dense. At worst you would compute some factorization of H and use
that to solve for the update d.

~~~
aria
Typo is fixed, thanks!

I think that's explicitly mentioned in the Quasi-Newton section that you only
need to implicitly multiply and not form the matrix.

~~~
tjl
I still see it misspelt throughout.

~~~
aria
Want to try a refresh, I think it ought to be fixed now. Thanks

