Keras is pretty much the best way to do almost anything these days. If you are starting out learning, use ConvNet JS, but after that switch to Keras.
TFLearn is really nice if you are already using Scikit.
There's lot of frameworks on there: TensorFlow, Caffe, CNTK (that's a lot of stars for something no one outside MS uses!) Theano, Torch etc. But I think the sleeper there is MXNet. I haven't used it, but I hear good things about it, and DLMC have good track record in producing some pretty nice software (XGBoost!).
Also, DeepDetect! I keep trying to find that and never can remember the name.
Top tip: If you use the matrix and vector classes in math.net then you can optionally configure it to use optimised version of e.g. matrix multiplication, that map through to one of the providers, such as Intel Math Kernel Lib, OpenBLAS, and I think there's a CUDA provider too.
I tried the Intel MKL one and the dense matrix multiplication was about 60x faster than a plain C# version.
Most of the time is usually spent in the convolution layers. Convolution is not a matrix multiplication in the current implementation. I guess it would be a matrix multiplication in frequency domain or by using a Toeplitz matrix.
I've implemented a CPU Parallel version and gave a try at GPU implementation. But I'm not satisfied at all by the GPU version :)
Pull requests more than welcome!
I figure there's a code re-organisation task since propagating node activations through a layer of weights is essentially a matrix multiplication (fully connected => fully dense matrix).
The optimised routines make use of vectorised CPU instructions and the FMA instruction (fused multiply and add), all of which are perfect fits for [dense] matrix multiplcation. Not so great for sparse matrices, but they help, usually unless it's very sparse it's faster to use a dense matrix format with zeros for the missing weights.
> Pull requests more than welcome!
Duly noted :)
What makes Keras take the advantage?
Compare how you build some vaguely comparable models in Keras and raw TensorFlow. Keras uses TensorFlow (or Theano) underneath, so there is no performance penalty.
It's like in Python machine learning, most people use Scikit instead of implementing things in numpy.
Ah, thanks for the reference, DD author here, the funny fact about the name is that translated to my mother tongue (French), it sounds like porn ;)
The network description language (now 'BrainScript') is a far nicer way to specify networks than the approaches used by any other network. Especially for recurrent networks. In CNTK you can just say `X = FutureValue(Y)` or `X = PastValue(Y)`. It's so convoluted in TensorFlow I actually never worked out how to do it.
It also has their fancy 1-bit SGD stuff, but I doubt many people use that, and it has a more restrictive license anyway.
Most of these frameworks are Python-oriented: Keras, Theano, Caffe, TF, neon, Mxnet, etc. The space is saturated. If you look at deep-learning projects by language, then Torch stands out -- it has a Lua API. And Deeplearning4j is the most widely used in Java and Scala. You don't have to crowbar it into a Spark integration, like you do with TensforFlow. http://deeplearning4j.org/
MXnet is not talked about a lot, but it's growing fast. It was heavily used by Graphlab/Turi, recently bought by Apple, so the question is what will happen with it now.
He does a pretty good job of showing actual activity on github:
I point this out because drive by stars of a project are easy...but it doesn't reflect actual user activity.
"Popular Deep Learning GitHub Projects" would have been a more accurate title. It is mentioned in the text but the title mentions "Top" which gives a different impression than the word "Popular".
It takes maybe 15 minutes in Python / numpy for something basic like a 2-layer net, but backpropegation is a little annoying to get right. Thus, tensorflow (or theano or caffe or whatever).
This is much older than the 9th of August.
It is 20/11819 = 0.16% stars off. It isn't old, just not automatically updated.