
Keras will be added to core TensorFlow at Google - jonbaer
http://www.fast.ai/2017/01/03/keras/?href=
======
chrisfosterelli
> because TensorFlow’s API is verbose and confusing, and because Keras has the
> most thoughtfully designed, expressive API I’ve ever experienced

This doesn't feel fair to say, just like it wouldn't if you replaced
"Tensorflow" with "C" and "Keras" with "Python". They operate on fundamentally
different levels and provide a similar trade off in control/ease of use.

We started using Keras for a project a few months ago, and it was great while
it supported what we were doing. Once we needed to go outside of the box a
little bit we essentially had to rewrite it in just Tensorflow.

This is great news though! Hopefully it will make the barrier to entry much
lower for getting started with Tensorflow.

~~~
taion
I strongly recommend checking out TensorFlow-Slim if you're looking for a
lightweight abstraction for using TensorFlow – at least the layers, anyway.

The API looks similar to something like Keras or Lasagne, but the layers are
just simple operations on tensors, so it integrates seamlessly with vanilla
TensorFlow in a way that something like TF-Keras won't.

At the same time, though, the layers are the same foundation that TF-Keras
look like they'll use, so you get much of the same expressive power, but
without sacrificing flexibility.

~~~
AlexCoventry
> it integrates seamlessly with vanilla TensorFlow in a way that something
> like TF-Keras won't.

Google also released prettytensor, which is designed to address the same
problem.

[https://github.com/google/prettytensor](https://github.com/google/prettytensor)

------
agibsonccc
Disclaimer: Very biased player in the space.

I'd like to personally see keras as the industry standard as an interface. We
are starting to give classes with it for our customers and the feedback has
been positive.

CNTK is adding one:
[https://github.com/Microsoft/CNTK/issues/797](https://github.com/Microsoft/CNTK/issues/797)

Mxnet is there:
[https://github.com/fchollet/keras/issues/1313](https://github.com/fchollet/keras/issues/1313)

TF being there obviously helps.

//Begin blantant self promotion :D

No one will care about ours as much (research crowd,python,etc etc,..), but
I'll throw our hat in the ring here too, keras has been amazing for our use
case and we are personally pushing more python people towards that:
[https://github.com/deeplearning4j/deeplearning4j/pull/2562](https://github.com/deeplearning4j/deeplearning4j/pull/2562)

^^ that will bridge the spark ecosystem and friends so you can run on
yarn/mesos/spark etc from a familiar interface.

//End blatant self promotion

------
captainarab
Somewhat off topic -- HN introduced me to fast.ai and I have found their
hands-on/practical approach to teaching particularly useful!

~~~
jph00
I'm glad you liked it :)

As it happens, I literally just now posted the curriculum for part 2 of the
course -
[http://www.fast.ai/2017/01/17/curriculum2/](http://www.fast.ai/2017/01/17/curriculum2/)
. If you're near SF, you may want to join us. Either way, I'd love to get
feedback on the curriculum - anything you'd like to see added? Anything from
part 1 that you'd like to know more about?

~~~
captainarab
I am just getting started actually, so I will provide you more feedback as I
complete part 1!

For starters, I found the pairing of the lecture, the code, AND the
documentation to be particularly useful. The setup in anaconda really enables
to you compare/understand inputs and outputs, which at least for me, is very
helpful! Big fan of learning through practical application, which the
aforementioned combo is well suited for, imo.

Kudos and thank you (and the team) very much for all of the hard work! I am
not sure I'll be able to attend part 2 in person but I will be sure to follow
along online. If I am ever in SF at the time of a course, I will certainly
apply!

All the best.

------
Ralfp
This looks very interesting to me. Being responsive for campaign sites
allowing people to upload images, I would love to be able to use neural
network in some of my projects, even if for simple "flag this picture as
suspicious" because people are supposed to upload pictures of their cakes, but
they are uploading something else instead. Over few years of running the site
we've accumulated human-curated database of few dozens of thousands of images
of cakes, and with traffic ever growing, I would love to be able to include
neural network to prioritize suspicious submissions for moderators.

As person without any eduction in the field of maths or neural networks,
examples from Keras docs and blog reasonated with me far more than Tensor Flow
ones.

------
ma2rten
The blog post links to a tweet and that tweet has a couple more details. The
new keras is going be tensorflow-only and build on top of tf.contrib.layers
(so a complete rewrite).

[https://twitter.com/fchollet/status/820746845068505088](https://twitter.com/fchollet/status/820746845068505088)

~~~
mgreg
It actually sounds like Keras as we know it will continue to exist and support
both Theano and TF.

Theano support will continue for as long as Keras exists. This integration
changes nothing for existing Keras users, only for TF users. -
[https://twitter.com/fchollet/status/821090410659344384](https://twitter.com/fchollet/status/821090410659344384)

Rather than a "new" Keras it sounds (I could be wrong) as if the Keras API
will be included now with TF as an alternative way of interacting with TF (as
it has been for some time) and simple bundled with TF.

~~~
attractivechaos
I wonder how Keras will be integrated into TF and how keras-in-tf will evolve
with the separate Keras. More specifically, how many more APIs will keras-in-
tf have in addition to the current keras? Wouldn't this hurt keras as a
backend-agnostic library? Will the keras-in-tf APIs always keep in sync with
standalone-keras? I will be curious to see how these pan out before getting
too excited.

------
kyloon
This is amazing news as using Keras is my opinion the best way to start
building models in TensorFlow for most people (and also having support for
Theano is definitely a plus).

------
fauigerzigerk
That is potentially very bad news.

I fear that from now on the Python API will be the best documented API used by
most example code, and the low level API will over time become something
obscure that is hard to approach and badly documented, just like Torch's C
API.

I don't like this split between a core system written in C or C++ and a high
level API written in a language that's too slow and memory hungry to write the
core system in.

This architecture is probably meant to reflect an existing split between users
and implementors of the system and I can understand the arguments in favor.
But I think it also creates and reinforces that split, which isn't a good
thing at all.

We were actually looking into moving to TensorFlow from Torch because the
Torch C API is considered an internal thing not meant for public consumption.

~~~
cinquemb
I don't really understand why people want to funnel themselves into these ml
frameworks when you can have the freedom to implement arguably more
interesting things in c++/fortran with libs like armadillo, openblas, superlu,
etc? I mean, i get it if people just wanna copy paste the functions they see
used in blogs/press/toy demo releases and get on with things, is that it?

~~~
fauigerzigerk
I have an uneasy feeling about these frameworks as well, but I'm not an expert
in this field. So if I'm told that it's not worth the effort to recreate all
of that stuff from scratch I'm going to have to believe it.

What I do insist on is that we keep the capability to selectively dig deeper
where we need to and combine different libraries with our own code.

That's why I'm always trying to reduce the frameworkness and increase the
librariness of any third party code we use. Scripting language runtimes are a
major roadblock in that approach.

~~~
cinquemb
> I have an uneasy feeling about these frameworks as well, but I'm not an
> expert in this field. So if I'm told that it's not worth the effort to
> recreate all of that stuff from scratch I'm going to have to believe it.

I wouldn't consider myself an expert in the "ml" field either, and only really
informed because of my academic/contract research work in neuroimaging wrt DSP
and beamforming and having to rely on performative code involving what at the
end of the day comes down to manipulating matrices with some set of
mathematical/physics constraints; I wished reviewers gave us an easier time
with our paper and just believed our results without taking months before
accepting it haha.

But if you are able to use these frameworks/hack around the quirks for your
use case with minimum effort and solve your problems to satisfactory degree, I
completely understand that they can be good enough.

>…we keep the capability to selectively dig deeper…

>That's why I'm always trying to reduce the frameworkness and increase the
librariness of any third party code we use. Scripting language runtimes are a
major roadblock in that approach.

I don't disagree with your sentiments at all, but it seems like you are
already at the intersection where you just need to go down your own road
because its probably unrealistic to expect Google's needs to be aligned with
yours in the long term. Then again, I'm the type of person who would fork the
nightly version of firefox and remove and add the stuff I want out of the box
so maybe this is not the best suggestion >.<

------
Smerity
I'm glad this is happening and honestly surprised this didn't happen earlier.
fchollet, the Keras main author, works at Google and had integrated TensorFlow
into Keras very early on (given he'd known about the existence long before the
general public). Even though Keras is backend agnostic, you don't really lose
any of the flexibility that TensorFlow would give you - it's a pretty
transparent abstraction[3]. Best of all, if a new underlying tensor library
appears tomorrow that's better than TensorFlow, you'll be getting support for
that too!

Keras goes beyond simply being a concise API, a variety of examples, or a
strong community with the "best practices included" philosophy. Opinionated
but quite useful.

As an example, the settings for an LSTM are complex and require reasonably
thorough understanding of many topics. There's dropout (and the many debated
ways one could apply it), there's the forget bias, there's weight
initialization, there's ... You get the idea.

If you use `keras.layers.recurrent.LSTM` however, bam, you get an opinionated
version of these for free. Initialization is Glorot uniform for most of the
weights but then orthogonal[2] for the inner weights. The forget bias is set
to one - as I hope every library has by default now but wasn't the case for
some time. Dropout is variational inference based dropout - recent, likely
what you want, and zero complexity.

At some point you'll likely want to learn about all the details - and this
provides a smooth easy transition for that as you go "wait, what's a Glorot?"
\- but for getting your feet wet and/or solving a specific task, "best
practices included" seems the best combination. I've successfully recommended
this to high school students and they've been up and running with neural
networks in short order!

Given all of this, whilst I'm a researcher who works on fiddly novel
architectures that require some pretty specific features so use Chainer at
work, I turn to Keras for my fun side projects as it keeps me sane and happy
:)

Full disclosure, I've committed examples to the Keras codebase and know
Francois in person.

[1]:
[https://keras.io/layers/recurrent/#lstm](https://keras.io/layers/recurrent/#lstm)

[2]:
[https://smerity.com/articles/2016/orthogonal_init.html](https://smerity.com/articles/2016/orthogonal_init.html)

[3]: [https://blog.keras.io/keras-as-a-simplified-interface-to-
ten...](https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-
tutorial.html)

[4]: [http://chainer.org/](http://chainer.org/)

~~~
agibsonccc
What does chainer have that keras is missing?

~~~
Eridrus
Chainer has a "define by run" architecture for defining computation graphs, so
you can have very dynamic computation graphs. TF/Theano have static
computation graphs, though an optimized define by run library is coming to TF
in ~February, not sure if there are any plans to make it play nice with Keras
though.

~~~
agibsonccc
Ohh that's very similar to ours actually. Our computation graphs just work
relative to their context. We've found it very easy to add arbitary vertices
because of it. Not that you care, but someone might: We ended up using this
style because we opted for not having auto diff in our layers due to speed as
well as the state of our framework at the time (layers being hand implemented
already).

We got the use case for auto diff but not being focused on research we just
decided it was easier to hand implement the layers and just have the graphs be
defined at the layer level.

Thanks for clarifying.

------
amelius
But what will happen to support for the Theano backend?

------
ilaksh
Is there any way to do Keras or Tensorflow on AMD or will there be? I know AMD
was pushing something new called ROCm or whatever. But I looked at the OpenCL
thread on Tensorflow and they seem to be pretending ROCm isn't relevant.

I am considering just using WebGL or something like Turbo.js to experiment
with my Radeon hardware for AI.

Maybe AMD should hire some people to work full time on Tensorflow. Anyway at
this point next thing I buy will probably be an nvidia card and chalk up the
amd graphics card purchase to ignorance. At least I played a few games on it.

~~~
transcranial
Have you seen Keras.js? It uses WebGL to run certain computations on the GPU.
[https://github.com/transcranial/keras-
js](https://github.com/transcranial/keras-js)

~~~
attractivechaos
Pretty cool! Just to make sure: keras.js so far only supports
inference/prediction, but not training (including autodiff, SGD, etc). Is that
correct?

------
chenzhekl
For me, `tf.contrib.layers` is high-level enough. Besides, it is easier to be
integrated to other TensorFlow functionalities than Keras.

------
dzhiurgis
Can you use Keras to train on GCP?

------
gooshan
For what I see, TensorFlow is getting what Torch users have had for awhile
with NN.

------
bbctol
I like this news; as someone coming from an abstract, non-programming
background, Keras is gorgeous.

------
pantulis
Is this bad news for TFlearn?

------
ktamiola
Excellent news!

------
du_bing
Great News！

