
Ensmallen – Flexible C++ library for efficient mathematical optimization - forrrealman
http://www.ensmallen.org
======
montecarl
I've written several frameworks for optimization and the approach here of
defining special objective functions works quite well. I like the look of this
and would love to try it out soon.

This is a small thing, but I'm really happy to see that their L-BFGS line
search algorithm supports a maximum step size. Many numerical optimization
libraries do not offer this simple feature, which is critical if your goal is
to find a minimum that is close to the initial starting location. This is
important in atomistic geometry optimization.

------
cozzyd
In terms of c++ matrix libraries, I've found eigen3 to be better than
armadillo (although armadillo is easier to use).

------
nolps
Github is
[https://github.com/mlpack/ensmallen](https://github.com/mlpack/ensmallen)

~~~
nolps
There's also a paper:
[https://arxiv.org/abs/1810.09361](https://arxiv.org/abs/1810.09361)

------
svantana
This is looking very nice, can't wait to try it out. But I feel it's a missed
opportunity to not provide some simple examples [1], as well as a benchmark to
quickly estimate the amount of flops that can be expected. The main reason to
use a library like this one over pyTorch or autograd/scipy is speed, after
all.

[1] The test cases can be seen as examples, obviously, but they are not
written in an easily accessible form IMHO.

EDIT: Looking more closely, it seems you need to provide an explicit gradient,
no autodiff included, which makes it a complete nonstarter for me.

------
mempko
Looked at the documentation, had to do a double take "Is this mlpack??? a
fork??" then checked, yup, it's a header only version of the mlpack's
optimization library. Very cool.

------
glalonde
Anyone know how this would compare to IPOPT for constrained, sparse nonlinear
optimization?

------
ipunchghosts
How does this differ from Ceres?

~~~
Mr_P
Ceres is great for sparse, structured non-linear-least-squares problems (that
is, problems where levenberg-marquardt is applicable). I believe it was
initially designed for bundle-adjustment problems arising from structure-from-
motion applications.

Ceres also does work for dense problems and general convex optimization, but
it doesn't provide as much of an advantage over other packages (that is, you
could just call into any existing l-bfgs implementation and wouldn't benefit
much).

For some other problems, Ceres doesn't scale or can get in the way. For
example, if you have a single value to maximize and a large number of
variables, then ceres' auto-diff system won't work (ceres::Jet is based on
"forward-mode" automatic differentiation, storing partial derivatives on the
stack. If you have a high-dimensional gradient, it can exhaust stack space).
On the other hand, Ensmallen doesn't seem to come with any auto-diff
functionality.

------
hughes
Sounds like the name is based on the perfectly cromulent word "embiggen"[1],
but somewhat confusingly uses the first part of the word "enlarge".

[1] [https://www.merriam-webster.com/dictionary/embiggen](https://www.merriam-
webster.com/dictionary/embiggen)

~~~
hellepardo
I don't think "emsmallen" can be said reasonably :)

~~~
hughes
Huh, turns out you're right, which is entirely why em- and en- both exist and
mean the same thing.

[https://en.wiktionary.org/wiki/em-#Prefix](https://en.wiktionary.org/wiki/em-#Prefix)

