
Google’s self-training AI turns coders into machine-learning masters - rbanffy
https://www.technologyreview.com/s/609996/googles-self-training-ai-turns-coders-into-machine-learning-masters/?utm_source=twitter.com&utm_medium=social&utm_content=2018-02-26&utm_campaign=Technology+Review
======
YeGoblynQueenne
>> “We need to scale AI out to more people,” Fei-Fei Li, chief scientist at
Google Cloud, said ahead of the launch today.

Problem is, when Google says "AI" they mean deep learning, on ginormous
datasets with humongous clusters of GPUs. That don't scale.

Accordingly, when Google says "we need to scale AI out to more people" what
they really mean is "we need to make more people use our services".

Sure- but most developers would be most happy with an AI system with the same
predictive power as deep nets (or thereabouts) that used let's say 1/1000th of
the data and could be trained on a cheap laptop in a couple of minutes.

That capability means either abandoning deep learning as inherently impossible
to train small, or spending major resources to make it possible to train deep
learning on small datasets with little compute.

Is Google working on this problem? Is anyone?

~~~
vidarh
Frankly I think we'd benefit more, faster even from just better standard
libraries of _really_ simple methods in ways that makes it more obvious how to
take advantage of them.

E.g. there are so many scenarios where even really basic statistical
approaches like bayesian models can provide drastic improvements over what
people tend to do, but most developers I've worked with don't know how to take
advantage of even that and/or don't know when they can use them or how to
communicate to stakeholders what capabilities are available.

There will be ridiculously many low hanging fruits in that area for years to
come.

~~~
nolroz
I'm trying to start simple on my journey towards thinking more like a
statistician and have long suspected this would be the case at many companies.
Besides Bayesian models, do any other common methods seem to be ignored as
often? Any suggestions resources that a newbie on the scene might use to
educate themselves?

~~~
yorwba
One simple method that is often good enough is k-nearest neighbors. Basically
computing the _k_ most similar training values to a test value and then
averaging their outputs to get a prediction.

~~~
nolroz
Thank you! I'll be sure to dig into k-nearest more as well.

------
agibsonccc
They picked such a horrible name for this. Yes it's catchy but please
understand what it actually is.

You have every ML vendor (disclaimer: My competitors) following google with
this name trying to get a piece of the hype pie spreading more confusion in
their marketing material. It drives me nuts.

You have folks doing everything from claiming hyper param search is "automl"
to transfer learning + grid search + "insert random architecture search" here
is magic that will save us all from needing to understand how this stuff
works. People it takes more than that. Please read the papers for yourself
down below and try to understand the limitations of these techniques.

Granted, it's great that we are attempting this, it's a real step forward, but
please call it for what it is.

Here's the papers referenced in the original blog
post([https://www.blog.google/topics/google-cloud/cloud-automl-
mak...](https://www.blog.google/topics/google-cloud/cloud-automl-making-ai-
accessible-every-business/)) :

Learning Transferable Architectures for Scalable Image Recognition, Barret
Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le. Arxiv, 2017.

Progressive Neural Architecture Search, Chenxi Liu, Barret Zoph, Jonathon
Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin
Murphy, Arxiv, 2017.

Large-Scale Evolution of Image Classifiers, Esteban Real, Sherry Moore, Andrew
Selle, Saurabh Saxena, Yutaka Leon Suematsu, Quoc Le, Alex Kurakin.
International Conference on Machine Learning, 2017.

Neural Architecture Search with Reinforcement Learning, Barret Zoph, Quoc V.
Le. International Conference on Learning Representations, 2017.

Inception-v4, Inception-ResNet and the Impact of Residual Connections on
Learning, Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alex Alemi.
AAAI, 2017.

Bayesian Optimization for a Better Dessert, Benjamin Solnik, Daniel Golovin,
Greg Kochanski, John Elliot Karro, Subhodeep Moitra, D. Sculley. NIPS,
Workshop on Bayesian Optimization, 2017.

------
Torai
Oh. What an inaccurate headline. MIT Technology Review: you did it again.

------
tanilama
As ridiculous this headline is and holding my suspicion that it might be a PR
piece, I wish wholeheartedly the following claim from the article is true.

> Automating the training of machine-learning systems could make AI much more
> accessible.

------
kmax12
This from Google’s January launch of Clould AutoML. Related discussion here:
[https://news.ycombinator.com/item?id=16168098](https://news.ycombinator.com/item?id=16168098)

------
castle-bravo
"Google's self-fitting curves turn bootcamp graduates into spreadsheet users"

------
vonnik
The head line should not be in the present tense. This is not something Google
does now. It should use a modal verb like “could.”

------
tluyben2
I like the AutoML and it has its uses but there is a long way to go before
they are qualifying for that headline.

------
lostmsu
So Google catches up with Azure ML?

