
Jeff Dean explains TensorFlow [video] - quantisan
https://www.youtube.com/watch?v=90-S1M7Ny_o&t=21m2s
======
argonaut
Just wanted to repost this from the other thread on TensorFlow, since I joined
the party a bit late:

I think some of the raving that's going on is unwarranted. This is a _very
nice_ , very well put together library with a great landing page. It might
eventually displace Torch and Theano as the standard toolkits for deep
learning. It looks like it might offer performance / portability improvements.
But it does not do anything fundamentally different from what has already been
done for many years with Theano and Torch (which are standard toolkits for
expressing computations, usually for building neural nets) and other
libraries. It is not a game-changer or a spectacular moment in history as some
people seem to believe.

~~~
mailshanx
I think the fundamental differentiator might be how "production ready"
TensorFlow is - the promise of simply declaring an ML pipeline and have that
run in very heterogeneous compute environments from mobile phones to GPU
blades to plain-old clusters, if fulfilled, can indeed be a huge game changer.
The promise is that you literally do not have to write _any_ new code when you
are done with a research project / a series of experiments and are ready to
deploy your pipeline at a large scale. None of Theano / Torch etc make that
promise.

~~~
argonaut
That's not really a _fundamental_ differentiator. Torch/Theano are definitely
production ready. I think the portability is definitely an advantage, though.

~~~
albertzeyer
I think you would not use Theano in an end-user product. It's made for
developers to run on developer machines. It's very fragile. It has a very long
start-up time, might be in the order of several minutes at the first start.

Maybe it would work good enough in a service backend. But even there it would
not scale that well. For example, it doesn't support multi-threading (running
a theano.function from multiple threads at the same time).

~~~
argonaut
Good point. Torch, then.

------
narrator
FYI, Jeff Dean is the inventor of most of Google's distributed computing
infrastructure including MapReduce. Definitely up there with the likes of John
Carmack and Fabrice Bellard as one of the great software engineers of all
time.

~~~
fmela
The Jeff Dean Facts are worth reading: [https://www.quora.com/What-are-all-
the-Jeff-Dean-facts](https://www.quora.com/What-are-all-the-Jeff-Dean-facts)

Personal favorite: "Jeff Dean once shifted a bit so hard, it ended up on
another computer."

~~~
vijayr
"Jeff Dean's resume lists the things he hasn't done; it's shorter that way."

------
shostack
Is there anything an "early" programmer like myself can do to play around with
this stuff without a background in the related math?

I'm dying for this stuff to be dumbed down enough where Joe WebUser can feed
in arbitrary data in a csv or point an app at a data source and get some sort
of meaningful results.

It truly seems like an area where once the barrier to entry is greatly
reduced, the creativity of laymen will lead to some truly amazing executions.

~~~
chm

        I'm dying for this stuff to be dumbed down enough[...]
    

It kind of already is. Have you read the docs/examples? I don't think your
mentality is fruitful. Having argued with people who shared your point of
view, it seems there will always be _something_ too difficult that prevents
them from being good at X.

There's no substitute for sweat. Have fun with the code they gave you and see
where you end up!

~~~
shostack
I think we have different definitions of what "dumbed down" means.

I never said I could/would never put in the work to learn it. I'm saying that
the place it is in right now is still too advanced for someone with my
background to pick up and play around with without sitting down to seriously
study the underlying concepts that are objectively fairly dense subject matter
that can require advanced math and CS backgrounds.

To be clear, I'm not advocating that everything should be dumbed down for the
sake of it. My point was largely that when the barrier to creation gets low
enough, more creative types that don't have the heavy technical backgrounds
can pick it up and create things that more technical users may never have
imagined.

Not everything is best served as remaining elusively complex for the layman.

Also, for the record, I will probably read up on some of this stuff because I
find it interesting and enjoy learning. I just wish it was a step more
accessible than it is today, even with this development.

~~~
chm
Hi Shostack,

I wasn't trying to bring you down. Maybe what you're looking for is a visual
programming environment, where you can drag and drop functions, data, etc?

~~~
shostack
Not sure I even need a visual programming environment as I'm comfortable using
a command line and hacking stuff together in Sublime.

However I am highly visual and visualizing the impact on the results would be
really helpful. I deal with a lot of analytics and data as part of my day-to-
day managing digital media. I often find that I can easily spot trends just by
glancing at data visualizations, and infer insights from them.

Further, being able to visualize the nature of the functions/data/etc. would
also be very helpful. I tend to need to visualize something to fully grok it.

If you have any suggestions for a more visual take on machine learning that is
beginner friendly, I'd love a link.

------
vonnik
TensorFlow looks amazing, and the Google team deserves huge kudos for open-
sourcing it. It may well become one of the best supported OS DL frameworks out
there.

People have been asking about its fundamental differentiators. I'm not sure
there are any. Theano and Torch already set a pretty high standard.

We know what good tools look like, and those tools exist even if they're
getting incremental improvements.

Now it's just a matter of building really cool things with them.

------
oiduts
This is from ~22nd of October, and he is being asked whether it is an internal
thing or going to be released - his answer is that it is internal and there
are no plans to release it. Did they change their mind in a couple of days
(not likely)? Was he not in the loop (also unlikely)? What else?

~~~
ozgung
Maybe touching to lips gesture is a body language sign of not telling the
exact truth:

[https://youtu.be/90-S1M7Ny_o?t=2043](https://youtu.be/90-S1M7Ny_o?t=2043)

He says "I don't have anything to announce" so technically not lying.

------
amelius
It would be great if one could automatically dispatch this to a commercial
cluster. So you could say: I want this network to be trained in 1 day, and the
system would say: that would cost $X, and it would instantiate some
AWS/Azure/Google instances, and run the task.

------
thearn4
> TensorFlow™ is an open source software library for numerical computation
> using data flow graphs.

> ...

> Gradient based machine learning algorithms will benefit from TensorFlow's
> automatic differentiation capabilities. As a TensorFlow user, you define the
> computational architecture of your predictive model, combine that with your
> objective function, and just add data -- TensorFlow handles computing the
> derivatives for you.

Interesting, it kind of look like the machine learning focused version of
NASA's OpenMDAO (also a graph-based analysis and optimization framework with
derivatives, but for engineering design).

------
Kiro
Jeff Dean has an amazing resume. He designed and implemented MapReduce,
BigTable and much more.

OT but how much does a super engineer like him get paid at Google?

~~~
EwanToo
Largely it'll come down to how much money he wants.

His salarly will probably be in the 6-figures, but he'll be a millionaire many
times over. He joined Google in 1999 (IPO was in 2004), so his stock will have
made him a very rich man.

~~~
argonaut
Way off. He's a Senior Google Fellow and his bio used to be on the executive
leadership page of Google (I can't find that page any more). He is being paid
on the order of 10MM per year, base, easily.

~~~
EwanToo
Perhaps, I doubt his salary is that high, but he'll have significant amounts
of of stock grants, options, etc.

Executive compensation is a very odd area, as I said, it pretty much depends
how much money he wants.

~~~
argonaut
Yeah I should have been more clear. But I'm pretty sure his equity alone is
more than $30M / year.

------
alttab
This was a great watch. It made me want to apply to the residency program.
Edit: wasn't sarcasm.

------
codyguy
Yay!! An open sourced voice engine in the future? That would really shake
things up.

~~~
IshKebab
No.

a) Stuff similar to this has been available for ages and there are no (good)
open source voice recognition packages.

b) It requires absolute mountains of training data which we don't have.

c) It requires designing a suitable network, which I'm not sure if we have,
but I would doubt it.

d) It requires training a network on the mountains of training data using an
immense computing cluster, which we requires money that we don't have.

Don't hold your breath.

~~~
ambiate
Sometimes creativity is the only thing holding people back from exploiting the
natural insects of the web.

Case in point, ever wonder why those captchas include street addresses or
'pick the shape with a hole in it?' Spoiler: you're building training data and
validating training data.

How else can we silently retrieve training data?

------
hooloovoo_zoo
Who cares? Gradient descent is bandwidth limited not software limited. (edit:
for ANNs)

~~~
Houshalter
Only for some applications, I think.

