
Snips Uses Rust to Build an Embedded Voice Assistant - albi_lander
https://blog.mozilla.org/blog/2018/02/21/snips-uses-rust-embedded-voice-assistant/
======
oulipo
Hi, I'm the co-founder of [https://snips.ai](https://snips.ai) and we are
building a 100% on-device Voice AI platform which runs on the Raspberry Pi 3
It is free to use for makers, and we will start open-sourcing the components a
few weeks from now.

We are using Rust because it is safe and very fast!

We would love to tell more about what we are building if you have questions.

The whole platform runs on-device which makes it ideal for privacy, cost, and
to allow it to run when there is no network

We are available in English, French, German, and soon Japanese and Korean and
we are working on other European languages!

We would love to see what you build with our platform to feature it on our
website

Take a look at what some people have built with it:
[https://github.com/snipsco/awesome-snips](https://github.com/snipsco/awesome-
snips)

and a few tutorials to get you started: [https://medium.com/snips-ai/building-
a-voice-controlled-home...](https://medium.com/snips-ai/building-a-voice-
controlled-home-sound-system-using-snips-and-sonos-2aaf16523ce9)

~~~
squarefoot
This sounds _very_ interesting. Being capable of working unconnected is the
only way to get true privacy from these devices. Having them open sourced
(hopefully 100% though I can understand the reasons for keeping some parts
closed) would be even better. I have always been strongly opposed to networked
voice assistants because the risk of them being used as spying tools by design
is simply too high, And surely no Google, Amazon, Apple etc. assistant will
never enter my home, but that one is a different beast. Keep up the good work!

ps. Please add other European languages soon :)

~~~
oulipo
We are working on this! What language would you want?

~~~
squarefoot
I'm Italian. There are applications in the automotive field for a mostly
disconnected device such as this one. If it can deal with a noisy environment
like a car that would make it even more interesting.

~~~
oulipo
We are also working on car environment, and we expect to start italian this
year :) Subscribe to our newsletter!

------
coldtea
> _How can Snips embed all the code for a voice assistant onto a single
> device? They wrote it using the Rust systems programming language._

And that's what we call an "non sequitur".

~~~
Jyaif
And it taints the whole piece.

"Instead of having to write and then rewrite for each platform, he used Rust’s
cross-compilation capability" also does not make a lot of sense.

~~~
Yoric
Well, to be fair, Rust's toolchain "speaks" cross-compilation natively. I
haven't found any other toolchain with such a well-designed cross-compilation
experience.

The piece seems to have been a bit over-simplified for non-technical readers,
though.

~~~
luciocorrea
Go has superb support for cross-compilation too.

------
quadrature
Great project, definitely a worthy goal. I know several people who are
skeptical of voice assistants because they aren't fully on device.

The blog article was very lean on the technical details apart from some basics
about rust's safety and portability features.

could you elaborate on where the gains from rust come into play ?, is your
model built on a DL framework that is written in rust ?.

~~~
oulipo
The DL framework is classical, but the models can then be reimplemented in
pure Rust for efficiency

~~~
quadrature
So you're taking the parameters from something like tensorflow and then
exporting them to be executed in Rust ? and that is more efficient than the
c++ backend of tensorflow ?.

~~~
throwawayendy
And also removing all of the overhead of the machine learning abstraction.
Although I am skeptical that anyone would ever actually do that

~~~
danieldk
I used to train my parser (before switching to Tensorflow) using Caffe, dumped
the parameters using a small program, and loaded them up in Go arrays slices
and applied the network using simple C BLAS operations. This works fine,
especially when you are using simpler networks. As a bonus, you don't have the
overhead of Tensorflow session runs.

It does become a bit of a drag when you are building more complex networks
(e.g. with multiple RNN layers, batch normalization, etc.). In that case there
are two straightforward options for Rust. There is a Tensorflow Rust binding
against the tensorflow C API [1] with which you can just load and run a frozen
Tensorflow graph. This is the approach that I am currently using, though I am
running graphs on workstations/servers.

Another option is compiling the graph with Tensorflow's XLA AOT compilation,
which compiles the network to C++ classes (that you could bind from Rust).

[1] [https://github.com/tensorflow/rust](https://github.com/tensorflow/rust)

~~~
albi_lander
There seems to be a third option provided by someone, kali, working at Snips:
[https://github.com/kali/tensorflow-deploy-
rust](https://github.com/kali/tensorflow-deploy-rust)

~~~
danieldk
This only supports a small subset of ops. It is pretty much corresponds to the
first option that I mentioned - train with Tensorflow, extract the parameters
and provide implemenations of ops in your native language.

------
t_fatus
I really love the idea of an off-line voice assistant, however most of my tech
friends in France and I feel that Snips (which is very visible on the press
here) still has to deliver something tangible ( _). This might be the first
time I want to try one of their product though.

_ they had announced a keyboard app for iOS and Android which did seemingly
the same thing as the swipe keyboard.

~~~
altharaz
I agree.

Their IA assistant seems really interesting, and I would really love to give
it a try, but I don't want to invest energy if there is a high risk that the
service shuts down soon.

I remember the Tranquilien app [0], which was also developed by Snips. There
was a lot of media talking about it, but it has been quickly stopped :(.

[0]
[https://www.digital.sncf.com/store/applications/tranquilien-...](https://www.digital.sncf.com/store/applications/tranquilien-0)

~~~
albi_lander
Actually the good thing is that if Snips disappears, you will still be able to
run your snips assistant as everything runs locally.

~~~
StavrosK
Yeah, but that's until some library makes a backwards-incompatible change and
the whole things stops building. Sure, it's better than a completely
proprietary service, but support and active maintenance are very important
too.

~~~
Yoric
The Rust ecosystem and build system, is designed to prevent this. If you're
using a library and want to code to keep compiling in 10 years, you can (and
should) just lock the dependencies.

Since the libraries are all mirrored on crates.io and, by design, cannot be
removed once they are published, you should be good, at least until the day
the owners of crates.io stop being able to pay the server costs.

I believe that you can also host local mirrors, but I haven't checked that.

------
hardwaresofton
Could you expand on why CMUSphinx[0]/Julius[1] not used?

I'm all for using rust for the speed and safety it allows, and then FFI-ing to
CMUSphinx (this is exactly what I want to do, given more free time in the
future), could you explain why the available open source libraries (CMUsphinx
is just one, there's also Julius) couldn't be used?

[0] - [http://cmusphinx.github.io/](http://cmusphinx.github.io/)

[1] - [https://github.com/julius-speech/julius](https://github.com/julius-
speech/julius)

~~~
oulipo
We are using deep-learning models which have much better accuracy for the
speech recognition

~~~
hardwaresofton
I understand that the project is using AI, but why not feed that learning into
sphinx, or some other tool? Couldn't this product have just been essentially
an extension to make one of those other open source, research-backed efforts
_smarter_?

How does any other project benefit from the models you build? Or is that the
business model -- produce open source software that no one else can really
extend or use with anything else, but hopefully people will then buy into your
modelling strategy + tooling?

I do realize that you have absolutely no obligation to any other voice
recognition effort, but I wonder how easy it is for anyone else to use the
model you're building.

~~~
oulipo
Sphinx has its own models, it is not easy to extend it with the frameworks we
are using

We will be open-sourcing more of the platform over time and give back to the
community, this will start with the NLU in the coming weeks

~~~
hardwaresofton
Thanks so much for being open about it, I see why you didn't go with trying to
extend it.

Again, I want to express that you don't owe me anything (and it was entitled
of me to imply that you did) -- but I wanted to know. Maybe in the future
writing that thing that can enrich other models is possible.

------
dustindriver
Does the voice recognition AI use all of the Pi's processing power while
running? I mean, can the Pi do anything else while the voice recog is running?
Or does it context switch after a voice command? Sorry if this is a silly
question...

~~~
oulipo
No, the Pi can still do computations while the speech recognition runs

------
bfrog
This sounds amazing. Looking forward to trying it out on my rpi

------
ziftface
> It had the traits Snips needed

Well done.

