
Leaf: machine intelligence framework in Rust - mjhirn
https://github.com/autumnai/leaf
======
halflings
It's unfortunate that just because Tensor Flow was released, you felt the need
to replace references to deep learning (which is a somewhat well-defined
application) by "machine intelligence" [1], which could mean so many things.

Otherwise, it's great to see such initiatives in Rust. I'm unsure if there's
an equivalent that implements more traditional ML algorithms (logistic
regression, decision tree, random forest, SVM, etc.)

[1]
[https://github.com/autumnai/leaf/commit/58bfa92abd5721eae116...](https://github.com/autumnai/leaf/commit/58bfa92abd5721eae116b44e86103cbd850aee23#diff-04c6e90faac2675aa89e2176d2eec7d8)

~~~
hobofan
We wanted to make clear that we are not "just" doing deep learning with Leaf.
A lot of other ML algorithms that don't fall under the deep learning label can
benefit of the tools provided (e.g. solvers) and can be implemented as layers.
While not too popular, there are papers showing this being done with e.g. a
SVM[0].

[0] [http://deeplearning.net/wp-
content/uploads/2013/03/dlsvm.pdf](http://deeplearning.net/wp-
content/uploads/2013/03/dlsvm.pdf)

------
rryan
Great work -- very exciting to see a Rust ML framework!

Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is)
but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I
think in particular the layer-based approach makes it awkward to represent
more complicated models since not every model can be decomposed into
backward/forward passes.

Do you worry that by adopting a layer-based paradigm you are boxing yourself
in? A graph-based approach does seem more flexible (and distributable across
machines).

~~~
hobofan
I am very optimistic concerning our plans to make Leaf work in a distributed
fashion. It's not quite finalized yet, but the rough outline consists of
abstracting other parts of the Network away as a special type of layer and
using capnproto + zeromq for transportation. Sadly I haven't come across a
good implementation of model parallelism to learn from, so that might become a
bit tricky.

I think there is still a lot of room for improvement for layer-based networks,
but even if the graph-based approach will become more prelevant I think the
Rust trait system gives us great tools to migrate and reuse a lot from our
layer based approach.

~~~
DjvuLee
I just curious about is there any good Actor model library in Rust, just like
Akka for Scala/Java, so there is a better model for transportation.

------
cstpdk
I am very excited to finally see this type of library in a modern (strongly
typed) language, let alone my current passion language. In my opinion your
pitch (ok, Readme), should put emphasis on the productivity and usability gain
of types and a generally "smarter" language moreso than the not-offloaded-to-C
performance (do people care what they code gets compiled to?)

I was pretty excited about the tensorflow announcement but I'm actually
saddened that it might cast a shadow over this library, which in my opinion
brings more to the arena (e.g. improvements on the current scripting approach
of theano, torch et. al)

Your product, Autumnai, sounds exciting as well

~~~
hobofan
Thanks for the feedback!

We love Rust for a lot of reasons, but couldn't really fit them all in the
Readme ;)

------
philippnagel
Since there seems to be an uptick in submissions related to machine
intelligence projects lately (or rather since Google released
[http://tensorflow.org](http://tensorflow.org)) (e.g.
[https://news.ycombinator.com/item?id=10537697](https://news.ycombinator.com/item?id=10537697))
does anyone dare to make a comparison?

I am currently just getting started in this area due to my interest in AI and
I am not quite sure which route to choose.

~~~
mjhirn
MJ here - founder of Autumn and creator of Leaf.

I can give a quick comparison on Leaf vs. TensorFlow - although this might
seem somewhat biased.

We like what the Google engineers did on TensorFlow and share a lot of similar
ideas on how future Machine Learning should be structured and implemented.
Especially that in the end it is just a performant pipeline for numeric
information processing.

We feel that the biggest difference (besides the different stages of the
project) is the language and the ecosystem that it embraces. We are strong
believers in Rust and think that it might have good chances to succeed in the
long run - which we outline in more detail in the Q&A[1].

Due to your question I would recommend picking up on the basic concepts of
machine intelligence and then going with whatever ecosystem fits the
task/stack. Might become similar to choices of web frameworks in the next
couple of years. I think Neon[2] - as a Python framework - seems to be easy
and performant as well.

[1]: [https://github.com/autumnai/leaf#why-
rust](https://github.com/autumnai/leaf#why-rust) [2]:
[https://github.com/NervanaSystems/neon](https://github.com/NervanaSystems/neon)

------
cwyers
> Leaf is a open source framework for machine intelligence, sharing concepts
> from TensorFlow and Caffe.

That was fast. (Like, I know that's a documentation change, but stil.)

------
semisight
Exciting to see this! I haven't taken a look at TensorFlow code yet but it
looks very similar (method names and all) to Caffe. I've wanted a CNN library
for Rust for a while.

It sounds like performance is important to you. Are there any plans to add GPU
paths soon? If it's CUDA, are you going to use cuDNN?

~~~
hobofan
Yes, GPU support has high priority but we are still investigating what
library/tools we want to go with, since the current options in Rust are not
that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would
require writing Rust binding) or maybe something else.

~~~
koute
If you're going for GPU support then _please_ do not use CUDA. It's non-
portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which
is very awkward to install. I highly recommend OpenCL which is portable,
requires no SDK, is open and for properly optimized code is just as fast as
CUDA.

~~~
nl
While this sounds attractive, OpenCL performance just doesn't seem to be there
at the moment.

In CNN benchmarks[1] the CL based implementations consistently finish at the
bottom. I _think_ this is mostly because of the weak nVidia OpenCL
implementations, but that doesn't help, since most people use nVidia (eg,
Amazon GPU instances are nVidia based)

[1] [https://github.com/soumith/convnet-
benchmarks](https://github.com/soumith/convnet-benchmarks)

------
DjvuLee
I like your idea very much, especially implement in Rust. As we need more
powerful computation, the native low-level language is a very good choose, we
can control the memory, and call the native library to run on CPU/GPU.

Even the development of Rust may not very high enough compared with Java/Go(I
do not sure), but we can keep the core as simple and steady, and use high-
level language such as Python/R to implement the API.

Hope to hear more wonderful news about this Project.

