
Comparative Study of Caffe, Neon, Theano, and Torch for Deep Learning - mindcrime
http://arxiv.org/abs/1511.06435
======
argonaut
For those interested in learning about the field: focus on the concepts, not
the frameworks. This is true especially of machine learning, which is highly
conceptual. I don't think these frameworks are suitable for people not
familiar with the concepts. For example, I think it's a good exercise for
beginners to implement a basic (no convolution) two layer neural net in plain
Python using matrix operations in numpy.

Regardless, these frameworks are more similar than different (one exception is
Caffe, which is very focused on computer vision and makes it really easy to do
stuff in that area, but makes it really hard to do anything else). In the
course of exploring the area, you will probably touch all of them at some
point (when you're trying to use a researcher's code).

~~~
mziel
This very much. Understanding the underlying concepts is crucial. Many people
use out-of-the-box tools just plainly wrong, because they didn't bother with
fundamentals. I know a guy who reinvented Naive Bayes and Laplace smoothing.
Kudos for him, but he wasted about 3-4 months of company's time on it.

That said, I would start with Python based frameworks:

\- Caffe is more for production, and Theano and family for experimentation.

\- You can play with Theano or Tensorflow using Keras which abstracts them

\- Python scientific stack makes it easier to visualize the results later

------
p1esk
It's a bummer they published this just before TensorFlow release...

~~~
mindcrime
True. But new "stuff" is coming out so fast in ML these days, that pretty much
whenever you publish a comparison, a month later something new is going to
drop.

------
frik
Torch, the scientific computing framework for LuaJIT, is based on the great
Lua language. This makes it a good choice - LuaJIT per se is a faster than any
Python interpreter/VM.

Caffe is C++, and the rest are Python (Neon, Theano and TensorFlow).

~~~
argonaut
This is a bit misleading. No framework is doing computation in the language
you write your model in. The code you write in any of these frameworks just
calls wrappers around the underlying C/C++/Cuda implementation.

~~~
frik
You are right, my statement was a bit misleading. These Lua and Python based
frameworks call the high performance implementation which is coded usually in
C/C++/Cuda/Fortran/etc. Nevertheless the Lua <-> C interface is lightweight
and it certainly helps if the overhead is low.

I know a guy who wrote a fast SAT solver in C, though called it with various
slow bash scripts. I asked him why he uses bash, he mentioned it doesn't
matter. In the end Microsoft's SAT solver beat him, exactly because they cared
about such details too.

