
TensorFlow: open-source library for machine intelligence - jmomarty
http://tensorflow.org/
======
kragen
This is really significant. At this moment in history, the growth of computer
power has made a bunch of important signal-processing and statistical tasks
_just_ feasible, so we are seeing things like self-driving cars, superhuman
image recognition, and so on. But it's been very difficult to take advantage
of the available computational power, because it's in the form of GPUs and
clusters. TensorFlow is a library designed to make it easy to do exactly these
things, and to scale them with your available computing power, along with
libraries of the latest tricks in neural networks, machine learning (which is
pretty close to "statistics").

As a bonus, it has built-in automatic differentiation so that you can run
gradient descent on any algorithm — which means that you can just write a
program to evaluate the goodness of a solution and efficiently iterate it to a
local maximum. If you do this enough times, hopefully you'll find a global
maximum. There are a variety of other numerical optimization algorithms, but
gradient descent is very simple and broadly applicable.

And it runs in IPython (now Jupyter), which is a really amazingly powerful way
to do exploratory software development. If you haven't tried Jupyter/IPython,
google up a screencast and take a look.

I'm just repeating the stuff that's on the home page in a different form, but
this is a _really big deal_. Most of the most significant software of the next
five or ten years is going to be built in TensorFlow or something like it.
(Maybe Dan Amelang's Nile; see Bret Victor's amazing video on the dataflow
visualization he and Dan built for Nile recently at
[https://youtu.be/oUaOucZRlmE?t=24m53s](https://youtu.be/oUaOucZRlmE?t=24m53s),
or try the live demo at
[http://tinlizzie.org/dbjr/high_contrast.html](http://tinlizzie.org/dbjr/high_contrast.html),
or download and build the Nile system from
[https://github.com/damelang/nile.](https://github.com/damelang/nile.))

Apparently the Googlers think that too, because among the 39 authors of the
white paper are several shining stars of systems research, including some who
may be up for a Turing Award in the next decade or two.

~~~
Fede_V
One small note on the paper itself. I love that the paper's author are in
alphabetical orders, without any stupid jostling over first/last authorship.

I think that's fantastic and wish academia went that way too (yeah right).

~~~
sytelus
Why is alphabetical order great? It just only means that people with last
names starting with z will get penalized every single time because of choosing
wrong parents! I would think randomized order would be chosen - especially if
you were mathematician.

~~~
infinii
"Choose" parents with a surname starting with A then.

~~~
Swizec
You can also legally change your last name to anything you want at any point
in time. If you care enough.

Or use a pseudonym.

~~~
ant6n
Meet Mr. Aaberg.

~~~
mcnawoh
Interestingly, with this sounding like a somewhat realistic Norwegian/Danish
name, it should actually show up at the bottom by the sorting rules of those
two languages.

The digraph "aa" is an archaic representation of the modern character "å", the
last letter of the Norwegian and Danish alphabets, and should be sorted
accordingly.

------
Fede_V
This is awesome, thanks so much for sharing this - and I love how whimsical
the docs are:

Our pond is a perfect 500 x 500 square, as is the case for most ponds found in
nature.

A feedback on the PDE aspect: is there any nice way to write your model and
grid using TensorFlow, and still obtain the free derivatives/optimized
computation graph if you use a different solver to actually integrate the
equations? The example shows off a very simple toy first order Euler solver,
but in practice this is grossly inadequate. I suspect that it's very difficult
if not impossible - especially since most solver libraries are very old
Fortran libs, and carrying forward the reverse mode differentiation there is a
nightmare. Still, this makes for very nice building blocks.

Another question - would you consider adding an example of using this together
with Google Ceres - to make the two autodiffs play nice together?

------
Nitramp
Maybe of interest: this is (amongst others) by Jeff Dean, famous for Bigtable,
Map Reduce, Spanner, Google Web Search, Protocol Buffers, LevelDB, and many,
many other things. He's a Google Senior Fellow, but also doubles as the patron
saint of Google Engineering.

~~~
zeroxfe
He is also a ninja, an astronaut, a master chef, and a damn good fellow!

------
Schiphol
> The TensorFlow Python API requires Python 2.7

The whole scientific Python stack is available for Python 3. This seems like a
somewhat backwards thing to do -- or perhaps the requirement is intended to
mean _at least_ 2.7?

Edit: added context

Edited again: More careful wording

~~~
vrv
We're looking to support Python 3 -- there are a few changes we are aware of
that are required, and we welcome contributions to help!

Tracking here:
[https://github.com/tensorflow/tensorflow/issues/1](https://github.com/tensorflow/tensorflow/issues/1)

~~~
Schiphol
Thanks for a constructive reply to a (well-meaning, but) not very nice
comment!

Congrats on the library, which looks awesome.

~~~
witty_username
Criticism is considered "not very nice"? Your initial comment wasn't impolite
or anything.

------
malmsteen
So, i have a very simple question.. :

I want to start deep learning now, to implement RNNs, autoencoders, Q-learning
on a very specific application (not images).

I've read a lot of papers, but not done any DL implementations yet (although
many classical ML so far), my question is very simple :

Where do I start ????

Should I use Torch ? Theano ? Theano + pylearn2 ? Theano + pylearn + Lasagne ?
Caffe ?

Or should I just switch to this magical new library directly ?

I feel confused.. any advice ?

~~~
varelse
Methinks embrace the magical library written by engineers and researchers at
Google instead of by a random goulash of machine learning grad students with
no investment in the outcome other than graduating.

~~~
rhaps0dy
> with no investment in the outcome other than graduating

Come on. Grad students are motivated by advancing the field.

... are they not? Maybe you can also be saying that the ones who write
libraries are motivated only by graduating.

~~~
varelse
Sure, we all start out that way. But by the 3rd or 4th year, you frequently
just want to write that thesis and get on to the next thing because you've
already advanced the field as much as you're going to as a grad student.

I have a large package of code on sourceforge that consists of my grad school
and postdoctoral efforts. If I compare its style to what 2 decades in the
industry since has taught me, it's laughable. That said, it is C code, so I
occasionally plunder it for algorithms I figured out way back when.

~~~
benanne
Nobody forced us to open-source Lasagne, so I think that remark was a bit
unfair. If we really didn't care about anything but graduating, why would we
bother going through the trouble of sharing the code in the first place?

But I do see your point. Google obviously has a lot more manpower to spend on
this, so it might be a better bet in the long run.

It's also worth comparing this to a few similar projects that have been
announced recently: MXNet
([http://mxnet.readthedocs.org/en/latest/](http://mxnet.readthedocs.org/en/latest/)),
Chainer ([http://chainer.org/](http://chainer.org/)) and CGT
([http://rll.berkeley.edu/cgt/](http://rll.berkeley.edu/cgt/)). And Theano of
course, which has been the flag-bearer of the computational graph approach in
deep learning for many years.

~~~
varelse
Actually lasagne is pretty good, I wasn't targeting you (but then you're
pretty good at winning kaggle competitions so perhaps there's a connection
there, no?)...

I'm thinking mostly of Theano, which, from a performance standpoint, appears
to have died the death of a thousand inexperienced cooks in the kitchen. The
~1000x performance regressions that it invokes when a junior data scientist
goes off the rails and ends up with a python inner loop amidst GPU kernels is
just depressing and seemingly unfixable. Hopefully, TensorFlow will be better
if only because it was written in a world now very aware of Pixel's Law.

Mxnet is awesome, but perhaps a little too parameter servery for my personal
tastes, and I'm now wondering what the point of CGT is now other than to be
Coke to Google's Pepsi. I also think the whole deep learning framework
business model just took a torpedo amidships (and not long after the layoffs
at Ersatz).

Finally, I had never heard of Chainer until today, thanks! That said, without
autograd functionality, the people I work with would probably stick with Caffe
+ cuDNN.

~~~
robbyetor
Pixel's Law? what s that? can' t find it on google...

~~~
varelse
Interesting, it does appear to be gone...

Basically, old NVIDIA marketing material used to state that GPUs double in
performance every year or so whilst Moore's Law is slowing down w/r to CPU
performance because clock speeds have been the same for a long time

This strictly isn't true because core count, SIMD width and vector unit counts
in CPUs have all been increasing. However, from the perspective of a single-
threaded C application, this is indeed so. CUDA/OpenCL OTOH automagically
subsume multi-core, multi-threading, and SIMD into the language itself so the
hypothetical "single-threaded" CUDA app just keeps getting better(tm).

The reality though (IMO of course) is that Intel promises and delivers
backwards-compatibility at the expense of free performance beer. In contrast,
NVIDIA delivers performance at the expense of 100% backwards-compatibility
beer for optimized code (but read each GPU's whitepaper and spend a week
refactoring your code per GPU generation and you get both, also IMO and of
course experience). Of course, to be fair, if you refactor your code every
time they improve AVX/SSE, CPUs are a lot mightier than what
Python/Javascript/R usually imply.

------
an4rchy
I tried going through the site and also the comments but couldn't wrap my head
around what this library actually is. It sounds awesome based on the
response/comments. Can anyone explain this to a layman?

~~~
levesque
It is a library for the optimization of machine learning algorithms, similar
to Theano. You write a model you want to optimize as a graph of symbolic
expressions, and the library will compile this graph to be executed on a
target platform (could be CPU, GPU). This is really neat because you
essentially wrote your program in Python and now it's going to be running as
optimized C++ code (or as optimized kernels on your GPU).

The fact that you defined your model as a symbolic graph means you can derive
gradient equations automatically, hence ridding you of a rather tedious step
in optimization. With such tools, the workflow for a practitioner is much
simpler: 1-code models as graphs of symbolic expressions, and 2-define
objective functions (ex. for a regression problem, your objective would be the
root mean squared error, plus some regularization). Disclaimer: I did not look
at the code, but from what I understand it is pretty much the same as Theano
(which I've been using a lot lately).

------
superfx
Does anyone know how this compares to Twitter's recently released torch
autograd?

[https://blog.twitter.com/2015/autograd-for-
torch](https://blog.twitter.com/2015/autograd-for-torch)

EDIT: To be a bit more specific, autograd just requires that one writes the
forward model, and it can involve any program logic like conditionals, etc, so
long as the end result is differentiable. It then takes care of all the
gradient calculations.

------
jheriko
i really wish people would stop calling multidimensional arrays tensors.

there is already well defined language for this. multidimensional array is
fine, psuedotensor is fine. tensor is confusing if you have any previous
background with tensor calculus before the word became machine learning
flavour of the month.

still. this does look pretty cool overall. processing large volumes of data is
becoming increasingly less specialised and more generic, which can only be
good. :)

~~~
zer0nes
Tensor is a much better term than multidimensional array:

\- It's not as verbose

\- The concept of tensor has linear algebra operations associated with it
while multidimensional array is just a programming term without any attached
Math operations.

~~~
jordigh
> The concept of tensor has linear algebra operations associated with it while
> multidimensional array is just a programming term without any attached Math
> operations.

But that's the precise problem. The multidimensional arrays that programmers
call "tensors" do not generally have the operations defined on them that you
would expect from a tensor, such as index contraction, tensor products, or
differentiation. At the same time, a tensor is different from an array, just
like a number is different from its decimal representation. These distinctions
are important when you want to change the basis, a very useful and frequent
operation for both tensors and numbers.

Then again, this battle was already fought and lost for vectors, which
programmers specialised to mean "arrays" instead of meaning "elements of a
linear space".

------
pvnick
François Chollet, the author of keras, made the following tweet this morning:
"For those wondering about Keras and TensorFlow: I hope to start working on
porting Keras to TensorFlow soon (Theano support will continue)."

------
cogware
Here's the TensorFlow whitepaper
[http://download.tensorflow.org/paper/whitepaper2015.pdf](http://download.tensorflow.org/paper/whitepaper2015.pdf)

------
andyjohnson0
_" TensorFlow is an Open Source Software Library for Machine Intelligence"_
and then later _" TensorFlow™ is an open source software library for numerical
computation using data flow graphs."_

So it seems to be a dataflow computation library that is being _used_ for
AI/learning. Since I know almost nothing about either, I'm wondering if this
(dataflow) approach has other applications unrelated to deep learning. Any
comments?

Also, i wonder if this could be implemented in hardware - say on a FPGA?

~~~
varelse
FPGAs are not measuring up to GPUs yet for these tasks. There's too much
floating-point math (and yes, Altera is addressing this) for forward
prediction, and way too much intermediate state to save for training (GPU
memory controllers are 'da bomb for this). Finally, OpenCL compilation time on
FPGAs is measured in hours as compared to seconds for GPUs.

That said, address all of the above and maybe NVIDIA's stock won't hit 50 next
year.

~~~
dharma1
did you see this?

[http://arxiv.org/pdf/1510.03009v1.pdf](http://arxiv.org/pdf/1510.03009v1.pdf)

~~~
nickpsecurity
Now THAT has some potential. Both for FPGA's and SIMD/MIMD architectures.
Thanks for the link.

------
peterjliu
For those looking for more context, here's the Google research blog post:

[http://googleresearch.blogspot.com/2015/11/tensorflow-
google...](http://googleresearch.blogspot.com/2015/11/tensorflow-googles-
latest-machine_9.html)

------
argonaut
Honestly, the raving that's going on in this thread is unwarranted. This is a
_very nice_ , well put together library, but it does not do anything
fundamentally different from what has already been done with Theano / Torch.
It is not a "game-changer" or a spectacular moment in history as some people
seem to be saying.

------
obstbraende
Are there any major conceptual differences to Theano? Not that I wouldn't
appreciate a more polished, well funded competitor in the same space.

It looks like using TensorFlow from Python will feel quite familiar to a
Theano user, starting with the separation of graph building and graph running,
but also down into the details of how variables, inputs and 'givens' (called
feed dicts in tensorflow) are handled.

~~~
albertzeyer
I'm looking at the RNN implementation right now
([https://github.com/tensorflow/tensorflow/blob/master/tensorf...](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/rnn/rnn.py)).
It looks like the loop over the time frames is actually in Python itself.

    
    
        for time, input_ in enumerate(inputs): ...
    

This confuses me a bit. Maybe the shape is not symbolic but must be fixed.

I also haven't seen some theano.scan equivalent. Which is not needed in many
cases when you know the shape in advance.

~~~
obstbraende
I think this loop actually still only builds the graph -- what `scan` would
do. The computation still happens outside of python. That is, in tensorflow
they perhaps don't need `scan` because a loop with repeated assignments "just
works"... Let's try this:

It seems like in TensorFlow you can say:

    
    
        import tensorflow as tf 
        sess = tf.InteractiveSession() # magic incantation
    
        state = init_state = tf.Variable(1) # initialise a scalar variable
    
        states = []
        for step in range(10):
             # this seems to define a graph that updates `state`:
             state = tf.add(state,state)
             states.append(state)
    
        sess.run(tf.initialize_all_variables())
    

at this point, states is a list of symbolic tensors. now if you query for
their value:

    
    
        print sess.run(states)
        >>> [2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
    

you get what you would naively expect. I don't think that would work in
Theano. Cool.

~~~
benanne
Why wouldn't this work in Theano?

    
    
        >>> import theano
        >>> import theano.tensor as T
        >>> state = theano.shared(1.0)
        >>> states = []
        >>> for step in range(10):
        >>>     state = state + state
        >>>     states.append(state)
        >>> 
        >>> f = theano.function([], states)
        >>> f()
        [array(2.0),
         array(4.0),
         array(8.0),
         array(16.0),
         array(32.0),
         array(64.0),
         array(128.0),
         array(256.0),
         array(512.0),
         array(1024.0)]

~~~
obstbraende
Thanks! When I tried this before, I thought compilation was stuck in an
infinite loop and gave up after about a minute. But you're right, it works.
Though on my machine, this took two and a half minutes to compile (ten times
as long as compiling a small convnet). For 10 recurrence steps, that's weird,
right? And the TensorFlow thing above runs instantly.

~~~
benanne
Agreed. Theano has trouble dealing efficiently with very deeply nested graphs.

------
ertand
Geoffrey Hinton and Jeff Dean will give separate tutorials at NIPS.
[https://nips.cc/Conferences/2015/Schedule?day=0](https://nips.cc/Conferences/2015/Schedule?day=0)

------
mrwilliamchang
Google says they are working on a distributed version and will release when it
is ready.
[https://github.com/tensorflow/tensorflow/issues/12#issuecomm...](https://github.com/tensorflow/tensorflow/issues/12#issuecomment-155150681)

------
auston
Many many thanks in advance to anyone who answers:

I've futzed around with ML docs (dataquest, random articles, first 3 weeks of
andrew ng course on coursera) & what I don't get about this
([http://tensorflow.org/tutorials/mnist/beginners/index.md](http://tensorflow.org/tutorials/mnist/beginners/index.md))
is how to actually make a prediction / classification on something that is not
"test" data. Can anyone point me in the right direction of how to "use" a
model once I have it validated, like we do at the end of that tutorial?

------
dryginmartini
How does TensorFlow compare to Torch7 ([http://torch.ch](http://torch.ch))?
They look very similar to me.

~~~
albertzeyer
Actually, this looks much more like Theano to me. It's all symbolic, it has
symbolic differentiation, it's Python and it even seems to have a similar API
like Theano.

In the Whitepaper
([http://download.tensorflow.org/paper/whitepaper2015.pdf](http://download.tensorflow.org/paper/whitepaper2015.pdf)),
it's compared to Theano, Torch and others.

~~~
nickpsecurity
That might be a strong advantage in its favor, though. Building a better
version of a proven concept with similar usage/API's often gets more market
uptake. Might get more use.

------
aconz2
> This open source release supports single machines and mobile devices.

Can someone clarify if "single machines" means the Apache license only applies
to single machines and not distributed systems?

Sidenote: I wish every open source project had a release video.

~~~
sudhirj
There's a note on the site about being able to move computation to a process
on a different machine, but it does mention copying the model. If I understand
correctly, I think models tend to be quite small, though.

Does it actually make sense to train a model simultaneously on different
machines? I can understand running the same model on lots of machines, of
course.

~~~
aconz2
I was interested in whether the _license_ only permits a single machine.

~~~
Sanddancer
Nope, nothing in the license prohibiting it. Seems more a case that they just
haven't released the glue for it to run on multiple machines, but hope to have
it added one way or another.

------
m00n
Minor nitpick: The authors are not the first ones to recognize that 'tensor'
makes a great (brand-) name. As someone who is paid to think about tensors
this is a bit annoying, because the first thing in my mind when I read a page
like OP, is "where are the tensors?". Alas, there are none.

For a mathematician, a "multidimensional data array" is not a tensor. It is a
tuple. A tensor is a tuple with much more structure, associated to linear
actions on its components. Said differently, if you can't tell me, what
7*[Alice] means, you have no right to call [Alice] a tensor.

------
musesum
I'm kinda floored, by this. Have been developing a flow based language to do
this. Even started with the approach of using Python and then moving to
scripting directly in C++. Maybe I tweak my own project. Anyone playing with
low level GPU deployments, like Metal?

[EDIT] Ok, so there is some GPU - should probably play some more before asking
obvious questions. Really keen on scripting out flow directly.

~~~
pilooch
link to your project ?

~~~
musesum
Sorry, not ready for prime-time. Closest is a visualization of the previous
version of the script:
[https://www.youtube.com/watch?v=a703TTbxghc](https://www.youtube.com/watch?v=a703TTbxghc)

That version wasn't a NN graph for ML; more of an executable flow graph for
visual music. Will post something on
[https://github.com/musesum](https://github.com/musesum) in a couple months.

------
mark_l_watson
I can't wait to try this tonight. I have a fun "messing around" project (I am
using GnuGo to generate as much training data as I need for playing Go on a
very small board, and I am almost to the point of starting to train and test
models).

BTW, Ruby has always been my scripting language but because of the wealth of
libraries for deep learning I am thinking of switching to Python.

~~~
mkorfmann
Why switch? I use Ruby, when I can and Python when I can't.

Would really love to see ruby forks of popular ml libs.

Or is it because Python is better perfomance-wise?

~~~
mark_l_watson
Really good question since I really enjoy using Ruby, more so even than
Clojure and Haskell.

I too would like to see Ruby ports of some of the popular ML libraries. The
classifier gem provides an example.

------
cdnsteve
AWS has GPU clusters already available FYI:
[http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_clu...](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cluster_computing.html)

Not sure how much effort is required to use them with this library.

------
eddd
2:21 in the video you can see a link to: corp.google.com

I love their UI for single sign on.

~~~
andy_ppp
Hmmm, the video appears to only be 2:17 long now...

~~~
eddd
1:52 actually, sorry I was mistaken.

------
syllogism
Any reason you wrote the Python wrapper in SWIG instead of Cython?

The nice thing about Cython is that you can wire up two C extensions to talk
to each other directly, without going through Python. Cython also gives you
2/3 compatibility out of the box.

~~~
pacala
SWIG supports many other languages besides Python.

------
RivieraKid
What kind of tool can be used to create that animated illustration? Looks
really neat.

~~~
croddin
It looks like it is TensorBoard, included with TensorFlow
[http://tensorflow.org/how_tos/summaries_and_tensorboard/inde...](http://tensorflow.org/how_tos/summaries_and_tensorboard/index.md#)

------
scott_s
The programming model here is very similar to that of the language I work on,
Streams Processing Language, normally just called SPL. A paper we have
submitted on the language itself: [http://www.scott-
a-s.com/files/ibm_tr_2014.pdf](http://www.scott-a-s.com/files/ibm_tr_2014.pdf)

Our domain is online stream processing for, generally, the big data space. I
do think, however, that describing computations in this manner gives enormous
flexibility to runtime systems to actually exploit all available parallelism
dynamically.

------
samuell
Seems like Ben Lorica's call for open source tensor libraries for data science
[1], were answered!

[1.a] [http://radar.oreilly.com/2015/03/lets-build-open-source-
tens...](http://radar.oreilly.com/2015/03/lets-build-open-source-tensor-
libraries-for-data-science.html) (March 2015)

[1.b] [http://radar.oreilly.com/2015/05/the-tensor-renaissance-
in-d...](http://radar.oreilly.com/2015/05/the-tensor-renaissance-in-data-
science.html) (May 2015)

------
samuell
Interesting! I wrote in a draft post back in late 2013 [1], that asked:

"What if one could have a fully declarative “matrix language” in which all
data transformations ever needed could be declaratively defined in a way that
is very easy to comprehend?"

I'm now pondering whether TensorFlow isn't quite an answer to this question?

[1] Posted the draft now for reference: [http://bionics.it/posts/matrix-
transformation-as-model-for-d...](http://bionics.it/posts/matrix-
transformation-as-model-for-data-flow-operations)

~~~
davmre
The idea of representing a program by a declarative computation graph of
matrix transformations has been big in deep learning research for a few years.
Theano [1] is the canonical example, though more recently the space has gotten
bigger with other frameworks including CGT [2] and now TensorFlow. The
computational graph abstraction is really nice in part because it gives you
very straightforward automatic differentiation, which dovetails very nicely
with the trend in machine learning of casting a wide range of algorithms as
(stochastic) gradient ascent on some loss function.

[1]
[http://deeplearning.net/software/theano/](http://deeplearning.net/software/theano/)

[2] [http://rll.berkeley.edu/cgt/](http://rll.berkeley.edu/cgt/)

~~~
samuell
Thanks! Yea, though in fact my idea is more about wiring the _actual (matrix
/tensor) operation_ with dataflow, rather than just the dataflow _between such
operations_.

But it might be bit of a different problem area :)

------
doczoidberg
I am a newbie to machine learning.

How does it compete to Azure ML besides it is open source?

~~~
dswalter
It's not really directly comparable. This is a computational framework which
you can use to implement a variety of ML algorithms (or in general numerical
computation), so the interface to this is lower-level than Azure ML.

If Azure ML is a bunch of premade sandcastle molds, TensorFlow is a more
accurate, faster way to pour sand. You make the mold.

~~~
mohamedmansour
TensorFlow built a language, AzureML uses a bunch of languages of your choice
to build any machine learning on Azure. Try it
[https://studio.azureml.net/](https://studio.azureml.net/) follow the tutorial

------
irq-1
This looks similar to Orange. Orange has a visual designer, easy data access
(SQL, arbitrary files) and lots of widgets, but it's a desktop tool not a
library and doesn't standardize the data structures.

[http://orange.biolab.si/](http://orange.biolab.si/)

[http://docs.orange.biolab.si/widgets/rst/index.html](http://docs.orange.biolab.si/widgets/rst/index.html)

~~~
infinite8s
The newest version of Orange (version 3) is standardizing on the scientific
python stack (numpy, scipy, scikit-learn)

------
mjw
Does anyone know if TensorFlow can apply algebraic simplifications and
numerical optimisations to the compute graph, in the way that Theano does with
its optimisations?

Sounds like it doesn't suffer from the (alleged) slow compile times of Theano,
but I wonder if the flipside of that is that you have to implement larger-
scale custom Ops (like torch's layers) in order to ensure that a composite
compute graph is implemented optimally?

------
vmarkovtsev
This is very similar to what we did in Samsung several years ago:
[https://velesnet.ml](https://velesnet.ml)

------
HockeyPlayer
Anyone experienced with machine learning want to try this out on some high
frequency trading data? I'm in the processing of preparing a data set for more
traditional analysis and would be willing to share.

We are in Colorado, but I'm happy to work with someone remotely.

------
Dowwie
Something tells me that Facebook's AI group will release its React to their
Angular

------
datashovel
Didn't read the paper, but when I saw that a main focus is on distribution I
started thinking about how awesome it would be if people could start
crowdsourcing compute resources to solve some big problems.

------
benmathes
Found a mistake in the docs:

[http://tensorflow.org/get_started/basic_usage.md](http://tensorflow.org/get_started/basic_usage.md)

In the "Variables" example, looks like a variable name got changed but not
updated everywhere:

    
    
        # at the start:
        var = tf.Variable(0, name="counter")
    
        # then:
        one = tf.constant(1)
        new_value = tf.add(state, one)
        update = tf.assign(state, new_value)    
        
    

The variable "state" should be the variable "var", or vice-versa.

------
devit
Does it support OpenCL and thus Intel integrated and discrete AMD GPUs?

~~~
chimtim
Appears to be only CUDA from the code. Also no distributed systems support
released. Also, I cannot get it to work since this morning.

~~~
happycube
Requires nvidia capability >=3.5 at that (GK11x and Maxwell) which rules out
anything I've currently got (a couple of 760's/GK104's)... which is the chip
Amazon EC2 uses as well.

They're probably using Dynamic Parallelism.

------
jefurii
The diagram at least looks a lot like Max/MSP and PureData, which are data-
flow tools for processing MIDI and audio data. Could this be used to implement
something along those lines?

------
sandGorgon
what would be really awesome is if Andrew Ng or Norvig build a course around
Tensorflow. It is really not useful to a beginner to be learning everything in
matlab.

~~~
davmre
There are already a few university ML courses out there using Theano (for
which Tensorflow is essentially a drop-in replacement), and I think this will
be a much bigger trend over the next few years. IMHO for a first course it's
useful to do some work at the Matlab/numpy level just so you get experience
with deriving/implementing gradients yourself, but for larger (deep) models
automatic differentiation is an amazing productivity boost that should make it
possible to cover a lot of interesting topics that you'd otherwise not have
space for.

~~~
sandGorgon
actually numpy would be a brilliant starting point - but I'm not able to find
any popular ones that dont use matlab (or some dialect thereof).

disclaimer: I have no idea what I'm talking about, but I do know that coursera
and stanford courses are the oft cited ones and they use matlab/octave.

------
joanfihu
Great stuff to see that Google is releasing their own tools for research and
production systems. Does anybody have a grasp of the main differences between
the triple Ts: Tensorflow, Theano and Torch?

As far as I can see Theano and Tensorflow support dataflow-like computations,
automatic differentiation and GPU support via CUDA.

Finally, why didn't Google, The Lisa Lab and Facebook worked together on
building a unique library instead of three?

------
aminorex
Quite an ugly set of dependencies to build. In particular, bootstrapping bazel
is excruciating.

------
rcthompson
Is there a part of TensorFlow that generates that slick diagram on the front
page?

------
Houshalter
This seems really similar to Brain Simulator: [http://www.goodai.com/#!brain-
simulator/c81c](http://www.goodai.com/#!brain-simulator/c81c)

------
fscherer
can anybody tell me if this can calculate the gradients of a conditional
graph? (something that is not implemented in theano which gives me a huge
headache right now)

------
blazespin
GPU support, woohoo!

~~~
varelse
And just 4 years after their insistence that GPUs would never have a role at
Google... How 'bout that?

~~~
pacala
Google is a big place, with diverse opinions. Even better, it encourages
updating one's opinion in face of new evidence. GPUs have proven themselves
cost-effective in tackling a number of vision problems [and others], Google
has those problems, so the opinion was updated and the problems were solved in
a cost-effective manner.

~~~
varelse
That... Was not... My experience there...

But kudos to the deep learning guys for overcoming that potential energy
barrier NVIDIA couldn't surmount on their own...

~~~
Florin_Andrei
To me, processing lots and lots of data with "relatively simple" algorithms
spells GPU, pretty much. Machine learning seemed destined to bump into GPUs
sooner or later.

~~~
varelse
Why yes...

[https://github.com/BIDData/BIDMach](https://github.com/BIDData/BIDMach)

------
Omnipresent
I was going to dive into Theano. Is this much different than Theano? Or
better?

------
orasis
Could this system be used to implement a Gaussian Process?

------
MoOmer
I'm really excited to dig in and read the source, actually.

Thank you for open sourcing this.

------
ppoutonnet
Great job guys!

------
thadd
Just found what I'm going to do today

------
fitzwatermellow
Great job, guys! The Docker of Neural Compute..

------
fibo
For those interested in a minimal data flow JavaScript Engine see my project
dflow:
[https://www.npmjs.com/package/dflow](https://www.npmjs.com/package/dflow)

------
swah
OTOH code completion is still only mediocre for most programming languages! Of
course, those things aren't really related, but it feels easier than self-
driving cars.

------
nye2k
If we could stop using the term Neural Networks in relation to computing, that
would be great. Computers don't have neurons.

~~~
BinaryIdiot
There are artificial and biological neural networks. The artificial one is
based upon the biological one. Therefore dropping the 'artificial' or
'biological' seems fine to me.

~~~
nye2k
This is practical, however I strongly disagree with this semantically. The
term should always be accompanied with 'artificial' or another name should be
invented as they are vastly different fields. Search Neural Network, I assure
you that you will no longer find biological information; it is littering.

