
Introducing TensorFlow.js: Machine Learning in JavaScript - runesoerensen
https://medium.com/tensorflow/introducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db
======
rationalthug
For anyone interested, here is a sampling of available ML libs in JavaScript.
The first two, including tensorflow.js, are sub-libraries of the major ML
projects, and the others are custom JS libs:

[tensorflow.js]([https://github.com/tensorflow/tfjs](https://github.com/tensorflow/tfjs))

[mxnet.js]([https://github.com/dmlc/mxnet.js/](https://github.com/dmlc/mxnet.js/))

[propel.js]([https://github.com/propelml/propel](https://github.com/propelml/propel))
- this uses tensorflow.js under the hood

[ml5]([https://github.com/ml5js/ml5-library](https://github.com/ml5js/ml5-library))
- also uses tensorflow.js

[webdnn]([https://github.com/mil-tokyo/webdnn](https://github.com/mil-
tokyo/webdnn))

[brain.js]([https://github.com/BrainJS/brain.js](https://github.com/BrainJS/brain.js))

[mljs]([https://github.com/mljs/ml](https://github.com/mljs/ml))

[synaptic]([https://github.com/cazala/synaptic](https://github.com/cazala/synaptic))

edit: added the ml5 lib mentioned in a subsequent comment.

~~~
murukesh_s
brain.js if I remember correctly is a very old project and is no longer
maintained. But it was great to learn the basic ANN concepts. What I would
love to see is a Node.js implementation of deep learning and ML libs (those
listed above are targeted for browsers). With Node.js supporting most of es6+
features it's been a fun coding in it. Can't wait to get hands on ML with
server side JavaScript as well..

~~~
robertplummer
It is maintained now: [http://brain.js.org/](http://brain.js.org/)

...I'm the maintainer

------
z1mm32m4n
I think my favorite part of this is that inevitably people will write type
definitions for TensorFlow.js in TypeScript, Flow, ReasonML, Purescript, and
more.

It has always seemed like data science and machine learning tasks have always
been most popular in dynamic languages like Python, Julia, and R. I really
hope this can be a bridge over to typed machine learning APIs!

(Of course, I’m also interested to hear about your favorite typed machine
learning library if you have some that you already use)

~~~
wwwigham
> I think my favorite part of this is that inevitably people will write type
> definitions for TensorFlow.js in TypeScript

Actually, it already ships with typescript definitions - it's written in
TypeScript![1]

[1]
[https://github.com/tensorflow/tfjs/blob/master/src/index.ts](https://github.com/tensorflow/tfjs/blob/master/src/index.ts)

------
jashkenas
For folks who want to immediately play around with this — I've ported over the
sentiment analysis example to an Observable notebook, so you can just edit the
code directly:

[https://beta.observablehq.com/@jashkenas/sentiment-
analysis-...](https://beta.observablehq.com/@jashkenas/sentiment-analysis-
with-tensorflow-js)

... or just have fun watching it analyze random Metacritic reviews of ”The
Last Jedi”.

~~~
sarreph
Thanks for this! As a non-ML'er (for lack of a better term), I enjoyed seeing
how simple it could be to port this kind of amazing stuff into a JS app :)

------
m0meni
This is really cool. I was wondering what the difference between this and
deeplearnjs[0] were, and turns out it's the same thing. Between this, and
propelml[1] (from the creator of node.js), it seems like ML in the JS
ecosystem is starting to pick up.

It'd be cool to see more demos like the ones on the
[https://tenso.rs/](https://tenso.rs/) website. Seems like they were using
their own hand rolled library though.

[0]: [https://deeplearnjs.org/](https://deeplearnjs.org/) [1]:
[https://github.com/propelml/propel](https://github.com/propelml/propel)

~~~
3laab
deeplearn.js has been rebranded as tensorflow.js

There are already some cools demos built with
tensorflow.js([https://deeplearnjs.org/demos/performance_rnn](https://deeplearnjs.org/demos/performance_rnn)
or
[https://teachablemachine.withgoogle.com](https://teachablemachine.withgoogle.com))

There's also a high-level library built on top of tensorflow.js that makes it
even easier to use some common ML methods:
[https://github.com/ml5js/ml5-library](https://github.com/ml5js/ml5-library)
(disclosure: I'm part of the team developing it)

~~~
thedangler
Those sites aren't working on google chrome Mac Version 65.0.3325.162
(Official Build) (64-bit)

I don't know what the issue is it tells me to download chrome.

------
lmeyerov
If you are interested in using production-grade ML from JS -- and the full
data science and big data eco-system -- a bunch of us in the Apache Foundation
and GPU world have been joining under Arrow and GOAI:
[https://devblogs.nvidia.com/goai-open-gpu-accelerated-
data-a...](https://devblogs.nvidia.com/goai-open-gpu-accelerated-data-
analytics/) .

The Graphistry team and ccri have been investing on zero-copy nodejs bindings
to the full ecosystem, eg, leverage full TenserFlow, AWS GPUs, CuDNN, SciPy,
pandas, dask, multigpu, MapD, etc. If you're a node hacker, this is an open
effort -- contributors welcome! You can reach me at leo@[ graphistry], and
I'll be publishing some JS roadmaps summarizing recent meetings and release
directions.

------
paultopia
Dumb question: Does this:

"TensorFlow.js automatically supports WebGL, and will accelerate your code
behind the scenes when a GPU is available."

imply that you can get all the benefits of GPU-accelerated machine learning on
any old GPU---not just CUDA-compatible Nvidia GPUs?

For example, if you've got one of those 15 inch Macbook pros with a GPU, can
you use TensorFlow.js to train a model at comparable (even if not identical)
speed to a custom deep learning box?

~~~
modeless
Yes, it will work on almost any GPU (even mobile!). No, it will not be
anywhere near the speed of CUDA.

~~~
murukesh_s
But I guess it will be as fast or faster than native CPU code.

------
rkp8000
This looks very cool.

Does the WebGL integration mean it doesn't require installation of CUDA or
cuDNN on the machine it's running on? That's currently the most difficult part
by far of getting GPU-enabled TensorFlow up and running in Python.

Additionally, can it work with non-NVIDIA GPUs? That is another fairly
stringent current limitation for those that don't want to cough up the dough
for an NVIDIA card or pay for cloud service.

~~~
anonfunction
It does not require CUDA or cuDNN on the client machine, it should work on
most GPUs (even mobile) but won't be as performant as CUDA / cuDNN.

------
diefunction
why ppl want to run ML in the browser ? Sorry I can not think of a use case.

~~~
lovelearning
One important reason is that it helps democratize ML to make it accessible to
everybody. Even a casual hobby programmer can write awesome applications, and
_anybody_ \- especially non-technical users - can use them. It makes ML -
especially inference - as easy as opening facebook.

Right now, just to consume ML models - whether as a application developer or
an end user - requires some combination of special skills that fall in a
spectrum of complexities - from something relatively simple like installing a
system package or an environment like Anaconda or a pip package, to something
much more complex and time consuming like building TF or Caffe.

ML in browser bypasses all of that.

~~~
bo1024
I still don't understand -- do you have an example application in mind?

You seem to be conflating producing ML models -- i.e. doing machine learning
-- with "consuming" ML models -- i.e. asking the learned models to make a
prediction. You don't need any ML in the browser to do the latter. And I can't
see why you'd do ML in the browser to do the former...

~~~
lovelearning
[Update: Oh boy, I realized this is a gigantically long post after posting it.
Sorry about that, but I hope my step by step explanation convinces you.]

Not conflating - the benefits apply equally to both learning and inference,
but since there are magnitudes more potential consumers of inference than
learning, I emphasized it.

It's true that one doesn't _need_ any of this, but my point is _not_ having
them in browser means there are barriers put up - of complexity, of costs, of
privacy, of effort - to developers and end users.

I'll use face recognition as a walkthrough example, but this applies to
absolutely _any_ ML use case if you think about all the steps involved in
taking it from idea to development to deployment to end use.

Take a problem I've worked on a bit - intelligently searching through personal
photos and videos. Most people have atleast a few hundreds of GBs of photos
and videos - family photos, pets, travels - in aggregate across all their
devices. Some may feel the need for search software that can answer questions
like "find me that photo with Alice (user's daughter) playing with
Scooby(user's dog) from 10 years ago".

In a world without browser ML, how would a developer design, develop and
deploy this with maximum convenience for both development and end use? Maybe
like this...

\- dev starts off by deciding they don't want to mess with any of that ML
stuff. Their skills lie in front-end design and usability. They decide to go
with Amazon's or Google's face recognition service (IDK if AMZ/GOO actually
have such a service, but if they did, it's reasonable a dev would look at them
as the first option).

\- But they soon find out it's just shifting the complexity elsewhere. Now,
they have to provide a way for users to upload their hundreds of GBs of media
to S3 or GCS. Which means more APIs to learn and integrate. More costs for
storage. Usability barriers and privacy suspicions for users. Security aspects
have to be looked into. Looks like it'll have to become a paid service now.

\- The service by itself is not enough. Dev still has to provide the front-end
(which they are skilled at) for users to select photos, crop faces, apply
labels, and send it all to the service's transfer learning API.

\- After all that, some users complain that accuracy is not good enough
because it couldn't find many photos. Dev has no way to tweak the models
because those are behind another company's opaque service. It's increasingly
looking like a custom backend is necessary.

\- So version 2. Dev learns some ML. Then downloads a pre-trained model that
can do face detection and recognition - say FaceNet or OpenFace.

\- They have to deploy it server-side for training and indexing. They learn a
bit of Nginx and WSGI, and deploy it. They don't know how many users will use
and how much data will be uploaded - have to plan automated scaling for that.
EC2 or GCE? More stuff and more APIs to learn, and more costs.

\- Dev still has to provide the front-end for users to select photos, crop
faces, apply labels, and upload to their learning service. Dev has to
implement per-user transfer learning and store per-user transfer data and
models.

\- Dev has to implement all the required provisioning for inference and
transfer learning - be it raw GPU servers or docker or K8s or whatever. More
costs.

\- For an end user, the need to upload hundreds of GBs of personal media to a
3rd party is also a barrier - takes time, loses privacy and likely incurs
bandwidth costs.

\- So version 3. Dev says forget the server-side. User already has GBs of
photos in their hard disks. Instead of bringing their photos to us and
managing it, let's take the software to them. Let's just package up everything
and allow user to download and use the entire thing on their local machines.
Maybe as platform-specific installables. Or as platform-neutral docker image.
Reduces costs and complexity for developer. Can even be free since there are
no costs incurred by developer. Android's still a problem since it can't do
docker, and dev doesn't know Android app development.

\- The end user too benefits with far better privacy and usability. However,
they still have to install a package - sounds easy, but in a world of "user
does not have administrative privileges" and "sudo", there are still potential
barriers to cross. And Android is still a no-go because the dev doesn't know
it.

Now in a world with browser ML, you can see how those remaining problems too
can be solved. Javascript ML is write once, run on any browser - even
Android's. User does not have to install anything. Dev does not have to write
anything specifically for a different platform. All the transfer learning and
inference can happen in user's browser.

The browser environment still presents some barriers - such as not being able
to access local photos directly without user selecting them, and limited local
storage for models. But both can be solved with some creative batching and
using solutions like emscripten's virtual file system in memory (I'm not sure
if TF.js uses the latter, but other frameworks like OpenCV.js do). User pays
some cost of reduced usability, which they may be ok with since they may see
the alternative options as being worse. And the privacy is matchless.

All this is applicable to any ML use case. Anything involving user's private
data such as speech recognition or document scanning/OCR too get the exact
same benefits for both developers and users.

~~~
bo1024
Sorry for the late response, but I want to thank you for the in-depth post! I
agree with you that version 3 is way better, but I'm very cautious about
advocating for browsers as de-facto operating systems. If we want better
cross-platform systems for sandboxing and running programs, I'd prefer to
develop those directly instead of giving more power to browsers and browser
vendors.

------
ralusek
For everyone's reference, I had remembered there was already a library that
existed that claimed to do this.

I was digging around to find it, it's called deeplearn.js, and upon checking
it looks like that project has officially moved to the one referenced in this
very post.

[https://deeplearnjs.org/](https://deeplearnjs.org/)

------
rwieruch
For anyone interested in using ML in JavaScript, checkout this status quo
article [0] about it. Tensorflow.js has to be added to it still.

\- [0] [https://www.robinwieruch.de/machine-learning-javascript-
web-...](https://www.robinwieruch.de/machine-learning-javascript-web-
developers/)

------
batmansmk
Super cool, I hope it will help getting more people around ML + bring better
ergonomics to ML overall.

