
Building A.I. That Can Build A.I - allenleein
https://www.nytimes.com/2017/11/05/technology/machine-learning-artificial-intelligence-ai.html?rref=collection%2Fsectioncollection%2Ftechnology&action=click&contentCollection=technology&region=rank&module=package&version=highlights&contentPlacement=2&pgtype=sectionfront
======
kmax12
I think this highlights the disconnect between AI researchers and industry.
This research focuses on image data and then makes the claim that they want to
use it to enable millions of businesses to solve problems with machine
learning.

However, image data isn't what most businesses are using for machine learning.
They use relational datasets. If you look at the recent "State of Data
Science" by Kaggle, relational data is ~3x more common than image data in
every industry besides academia and the military [0]. While Google wants to
1000x the number of organizations using AI, they aren't focusing on the
problems companies actually have.

Basically, academics love building AI for images, but what companies really
need are better ways to build AI systems on tabular and relational data.
Images will be a piece, but shouldn't be the focus.

Disclaimer: my company develops an open source library called Featuretools [1]
to automate feature engineering for data with relational structure.

[0] [https://www.kaggle.com/surveys/2017](https://www.kaggle.com/surveys/2017)

[1]
[https://github.com/featuretools/featuretools](https://github.com/featuretools/featuretools)

~~~
sweezyjeezy
They also did experiments on the Penn Treebank dataset - i.e. text data. This
was 53% to RDBs 65% in the Kaggle survey, so certainly not an obscure
problem...

~~~
kmax12
Yep, text data appears to be widely used too. Hard to tell from the Kaggle
survey, but a large percentage of text data I see in industry is part of
relational data or connected to tables in a relational database. Things like
forum comments, product descriptions, etc

------
thisisit
In case someone wants to check out the progress on AutoML, the project in
question, Google had posted an update:

[https://research.googleblog.com/2017/11/automl-for-large-
sca...](https://research.googleblog.com/2017/11/automl-for-large-scale-
image.html)

------
rkaplan
This is not "AI that builds AI". The actual research behind AutoML is called
NASNet
([https://arxiv.org/pdf/1707.07012.pdf](https://arxiv.org/pdf/1707.07012.pdf)),
and all it is simply: we found two good neural network layers (called NASNet
normal cells / reduction cells in the paper) that work well on many different
image datasets. It's a very cool research result. But it's not something that
will replace AI researchers.

~~~
Eridrus
This is not the entire field of AuotML or even the entirety of Google's
published research.

~~~
ollin
Yeah, I'm confused that this is the top comment; it's factually incorrect.
NASNet is an example of a _result_ of AutoML. To quote the Google blogpost on
NASNet:

> _In Learning Transferable Architectures for Scalable Image Recognition, we
> apply AutoML to the ImageNet image classification and COCO object detection
> dataset... AutoML was able to find the best layers that work well on
> CIFAR-10 but work well on ImageNet classification and COCO object detection.
> These two layers are combined to form a novel architecture, which we called
> “NASNet”._

[[https://research.googleblog.com/2017/11/automl-for-large-
sca...](https://research.googleblog.com/2017/11/automl-for-large-scale-
image.html), November 2017]

In contrast AutoML is, as the nytimes article describes, "a machine-learning
algorithm that learns to build other machine-learning algorithms". More
specifically, from the Google blogpost about AutoML:

> _In our approach (which we call "AutoML"), a controller neural net can
> propose a “child” model architecture, which can then be trained and
> evaluated for quality on a particular task...Eventually the controller
> learns to assign high probability to areas of architecture space that
> achieve better accuracy on a held-out validation dataset, and low
> probability to areas of architecture space that score poorly._

[[https://research.googleblog.com/2017/05/using-machine-
learni...](https://research.googleblog.com/2017/05/using-machine-learning-to-
explore.html), May 2017]

Quoc, Barret, and others have been working on ANN-architecture-design systems
for a while now (see:
[https://arxiv.org/abs/1611.01578](https://arxiv.org/abs/1611.01578)), and
AutoML specifically was done _before_ announcing NASNet. Saying that NASNet is
"the actual research behind AutoML" is drawing the causal arrow backwards.

------
hal9000xp
Sure it's exciting stuff for those handful of researchers who currently work
on projects like AutoML.

On the other hand, I feel sad for myself because me and many others left so
far behind. I have strong feeling that such technologies would lead to
concentration of power of such mega-corporations like Alphabet as well as
complete monopoly for any creative work. So very few of us who managed to
become cutting-edge researchers would be proud to be creative humans, others
will do just a monkey job using _magical APIs_.

In 80s, two people could create state-of-the-art game written in assembly with
it's own tiny game AI (hello to Elite [1] which has intelligent opponents who
engage in their own private battles and police who take an active interest in
protecting the law).

In 90s, a small team could create state-of-the-art game written in pure C with
some cool AI (hello to Quake III Arena [2] which has pretty strong bots [3]).

[https://en.wikipedia.org/wiki/Elite_(video_game)](https://en.wikipedia.org/wiki/Elite_\(video_game\))

[https://en.wikipedia.org/wiki/Quake_III_Arena](https://en.wikipedia.org/wiki/Quake_III_Arena)

[https://www.researchgate.net/publication/240430519_The_Quake...](https://www.researchgate.net/publication/240430519_The_Quake_III_Arena_Bot)

In both of these cases, you don't have to be genius to be able to understand
whole thing alone.

I'm 33 and I progress very, very slowly. I feel I might be on the level close
enough to understand Q3A entire source code. I think I would have great future
if today is 1994. Unfortunately for me today is 2017 and I do realize that I
don't have any exciting future at all.

~~~
albertgoeswoof
I have to go ahead and completely disagree with you there. It's certainly more
complex under the hood but:

A) Education has never been this accessible - see
[https://www.coursera.org/](https://www.coursera.org/), youtube, MOOCs, blog
posts etc. which did not exist anywhere for free even 10 years ago

B) APIs and abstractions make a lot of this quite accessible (e.g. AWS,
tensorflow etc.), yes these are "magical" APIs, but you could make the same
argument regarding a C compiler going to binary, all the way down to logic
gates and electrical pulses

33 is young in terms of education, I would highly doubt you're progressing
slowly due to your age, probably more your attitude that is holding you back.

~~~
taneq
I think your points are correct but there's another which you might be
disregarding and which is causing GP poster's feelings: The volume of
knowledge to be learned if you want to do anything meaningful from anywhere
near 'first principles' is orders of magnitude greater than it used to be. If
you just want to be cutting edge using a "magical API" then sure, download
Keras or TensorFlow and play with some DNNs. But if you want to understand
everything you're doing at the theory level then you've got to learn so much
more than you did back in the 90s.

~~~
hal9000xp
Thanks! That's exactly what I mean. I don't want to use magical API and "just
play with data". I really want to be able to understand from ground up.

It doesn't mean I have to read every single line of Tensorflow but being able
to do that when it's needed. So that such tools won't be magical black box for
me.

~~~
empath75
Ground up knowledge is difficult to obtain in any field. How long do you think
it would take you to get a complete understanding of a modern car from he
ground up?

~~~
dominotw
Atleast, modern car design is stable enough that you could be motivated to
learn it and have lasting , statisfying, longterm knowledge.

~~~
Joeri
I disagree that car design is a stable field. Tesla is selling a radically
different car design. All car designers have to face the dawn of self-driving
cars.

In every field the total knowledge set is always increasing, which is both
empowering, because we stand on the shoulders of giants, and diminishing,
because there is less low-hanging fruit. There is always more low-hanging
fruit though, the trick is to see it hanging there. ML is a wonderful
opportunity because the magical api’s can do far more than they’re currently
used for.

------
marmaduke
AI building AI or just hierarchical modeling?

I'm sure there's a market for repurposing ML models via APIs but it seems
unlikely to be the dream job for an AI researcher, rather the ML analog of
CRUD

------
minicaionut
"Jeff Dean, a google engineer", he is some sort of Chuck Norris of coding. I
would not call him that way.

~~~
0xbear
Jeff is a manager nowadays. His coding days are in the past. Although, like
Chuck Norris, he certainly has proven he can kick ass.

~~~
tuyguntn
Out of curiosity, if anyone knows here, how was the Jeff Deans coding? is it
true that he is very fast at coding and building systems? How do you compare
him with others?

~~~
jboggan
Yes, very fast. His output increased tremendously in 2000 when he got one of
the first USB 2.0 keyboards.

------
deevolution
Is it worth my time to study algorithms if I could instead put that time
towards learning ML techniques?

~~~
rspeer
Yes.

90% of the ML techniques you learn will be passé in five years. 90% of the
algorithms you learn will still matter in five years.

~~~
castle-bravo
Seconding and elaborating on this: If your technique is underperforming,
you'll be in a position to find out why instead of just throwing your hands
up. You'll be better able to select which technique to use. You'll understand
what the parameters mean and be able to select and adjust them in a principled
way instead of through trial-and-error. In short, you'll build better models
in less time if you understand the underlying theory.

------
hacker_9
_" We redesigned the search space so that AutoML could find the best layer
which can then be stacked many times in a flexible manner to create a final
network."_

I was literally thinking the other day how one could train a neural net to
build better neural nets, and here it is. Such a simple and powerful solution,
building up layer by layer, choosing the best version each time. Really
exciting stuff.

------
theCricketer
For people interested in a bit more technical background, this paper by the
authors featured in the NYT article has some good introductory background:

[https://research.google.com/pubs/pub45826.html](https://research.google.com/pubs/pub45826.html)

------
BenoitEssiambre
I'm not sure how google's system works but an AI that writes programs seems
like a promising idea to me. I have been toying with the idea on my spare
time.

Here is a description of my (failed) approach:

[https://www.quora.com/What-deep-learning-ideas-have-you-
trie...](https://www.quora.com/What-deep-learning-ideas-have-you-tried-that-
didnt-work/answer/Benoit-Essiambre)

I've gotten a bit closer to it working since I wrote that post on quora.

------
baxtr
I wonder how we are going to react, when computers start acting as irrational
as we do: “Nah, today’s not my day, I don’t want to do the calculation. Maybe
tomorrow...”. Oh well...

~~~
ianai
My suspicion is that consciousness in part stems from survival. I don’t see
computers having a survival pressure point similar to water/food/shelter that
they directly have to address.

~~~
emptybits
If computers have a Maslowesque hierarchy of needs, I think the bottom tier
(i.e. "physiological") may be the need for appropriate electrical power and a
switch that remains "on". That is survival at its most basic.

The next tier (i.e. "safety") may include security, both physical and digital.
Continuous and stable power, firewalls and other protections, etc.

Somewhere farther up might include a need for data, network connectivity,
normally-terminating programs, a desired level of CPU or storage utilization,
few errors in its logs, etc. (i.e. "belonging", "esteem", "self-
actualization")

So demonstrating (or faking) consciousness, to the degree its human operators
recognize it as such, could serve survival needs. e.g. "Don't turn this one
off; it's self-aware now, which is cool, plus it seems to enjoy solving our
hardest problems."

~~~
mrleinad
But even if a computer needs, for example, electricity, does it really want it
the same way we do need oxygen? If we don't breathe, there's an unconscious
impulse to do so, and we know that not breathing leads to death, which for us
is pretty much the end of the road. None of those points are valid for a
machine, since they don't have a subconscious mind and if they're off they can
always be turned back on again.

~~~
emptybits
An unconscious impulse may be the product of evolution. The early air-
breathers who didn't have that _unconscious_ impulse would have died off.

So given the opportunity for AI to evolve itself, it's plausible that it would
do so, resulting in advantageous impulses. e.g. regular (unconscious)
behaviour or signals to convince its humans to not pull its plug mid-cycle
(information would be lost, painful, time-and-power wasting, etc.).

------
ninegunpi
And then the A.I. will learn to manage humans in subtle ways (like filtering
Facebook feed to achieve certain benchmark) and voila!

No need for SkyNet and terminators. ML to build better ML schemes to better
control humans - that's a fun apocalypse to watch.

------
yters
Once AI can program new AI, then we've got something. This isn't it.

------
justonepost
A.I that can actually build A.I is going to be frightening to a few more
people than just engineers.

~~~
jeremyjh
General A.I. building general A.I. would be frightening, yes. That isn't what
this article is about. We're still just talking about ML image classifiers
here.

~~~
giardini
So only people who work with ML classifiers should be concerned? An increasing
need for data scientists and ML researchers has been forecast for months and
here's a technology to eliminate that need. Hope your next dream job wasn't
based on ML.

But anyone who has a large number of ML developers working on tasks could have
(and probably should have) done this [automate or semi-automate generation of
ML networks] already. The best (i.e.,laziest) programmers automate their work
as much as possible.

This situation has the feel of a "Singularity": just as Fall's incoming
college students embark on an introductory class in ML, they read about how
Google and others might eliminate the need to develop with ML.

------
chasedehan
Buzzword that can build buzzword.

I get really tired of hearing these buzzwords being thrown around by people
who don't even know what they mean. "I'm building a deep learning, AI system
on Big Data using machine learning and predictive analytics on Watson"

~~~
killjoywashere
You don't realize how many times you've heard those words.

But the senior folks with maximum control are just starting to hear about this
stuff. The buzzwords are just trying to get these seniors to put the project
work in the right bin so, hopefully, when they hear about it again later, in a
slightly different context, they'll remember.

To me, CEC is a buzzword. It's a whole suite of combat control concepts,
hardware, software, training pipelines, etc. But you've probably never even
heard of it. I've got to make sure my project gets into the senior's head, and
lands in the tiny "ML" bin, and not the huge "CEC" bin, which has it's own
"ML" sub-bin.

------
megamindbrian2
I already signed up for their free CPU cycles

------
jessaustin
At this point one just assumes that breathless AI headlines are all hype, but
if true this seems like an S-Class threat.

~~~
temuze
> S-Class threat

Is that a Worm reference?

------
cindystevenson
Nice

------
diminish
Any virtual or physical general AI being can sooner or later build a copy
itself, easily in the former. Asymov's laws and other type of built-in
regulations in scifi are childish imaginations as the company or the nation
doing it will earn a huge advantage. It's going to be equal to nuclear weapons
and will need regulations in consumer tech, which will be hard.

All the contradictions point to the fact that we humans either live like
retarded biological animals or augment ourselves by integrating the AI
features just like our kings married the women of the enemies to boost their
genetic and social appeal. I'm for the latter.

~~~
lsc
So, uh, I'm still at the "I don't even know the extent of my ignorance" phase
of understanding current ML tech; I haven't even gotten to linear algebra in
my math education. I'm working on it.

However, I do spend my days around (often fixing computers for) people who do
seem to understand machine learning... and as far as I can tell, we're still
in a phase where machine learning functions like a fancy sort of filter... a
way of determining if this new piece of data is more like this set of training
data or the other set of training data.

While I totally see how that could be super useful in designing business
applications, I mean, I could totally use some sort of ML filter to take the
boss' words and match them with something I know how to do, or with something
you can solve with an existing ML library... and while I can see how something
like this could potentially help to replace me, I don't see what it has to do
with artificial consciousness.

~~~
fnbr
This is basically the stance of every serious practitioner that I've seen (ie
the stance shared by everyone who directly works with theory/code to train
models, as opposed to those who talk about AI at a high level).

Right now, we have these systems that are effectively ungodly complicated
spreadsheets. They're great at a variety of tasks, some of which seem
impossible for a non-intelligent entity to perform (neural machine translation
is wild to me).

But that's all the systems are- super complicated spreadsheets. There's no way
for them to start replicating consciousness without massive advances in the
field.

Having said that, there is a road from where we are to intelligence- if we can
create a network that performs arbitrary interactions online, and figure out
some way to create a positive feedback loop for intelligence, like AlphaGo
Zero did with their policy network & MCTS, then we might be able to figure it
out. But we're so far away from that that I'm not concerned.

~~~
lsc
Concerned? I know which side my bread is buttered on; when the revolution
comes, John Connor and I will probably not be friends.

But yeah; as disappointing as I might find it, I kind of think we're heading
towards more of a 'star trek' dystopia... a universe with continuing ethnic
strife and computers that are advanced when it comes to responding to what we
want, but that remain tools, without much by way of will of their own.

~~~
goatlover
I thought Star Trek was considered Utopian, at least inside the Federation.

~~~
lsc
> I thought Star Trek was considered Utopian, at least inside the Federation.

In my comment, I'm implying that any universe where we don't figure out AI,
where humans are still in charge is a sort of dystopia.

To be absolutely clear, it was a poor attempt at a joke. Many of these
observations can also be read in a positive light. But I do think that in a
lot of ways you can see darkness in the federation.

They haven't figured out AI and still have humans in charge of menial tasks,
humans who aren't particularly good at those tasks compared to a computer.[1]
I mean, sure, exploring, sending people to explore is great, but they also
send people to fight, even when the battle is existential. They still have
humans in charge, even though those humans are still only slightly less
corrupt and petty than we are.

They also apparently still have huge issues with racism even within the
federation. This is the second part of the comparison; I have recently learned
that my own society seems to be rather more racist than I thought it was. I
have learned that progress is way slower than I initially thought. Star trek
reflects this glacial progress.

[1]Apparently, they have bans on enhancing those humans, even though they have
the tech to do it (see bashir's storyline on DS9) To me? this seems like the
worst kind of waste. To have the technology to make us all brilliant, but to
leave us all as dullards.

------
GayathriM
This is going to be huge. I never expected this step to be coming very soon as
I thought this is the phase where we are understanding A.I and finding ways to
make a controlled A.I system. I don't believe we have achieved that confidence
but I think I am cent percent wrong. We must be too far than that which should
be the confidence behind thinking this concept.

~~~
thedoctor79
I don't understand why this post is downvoted. You have a perfect example of
an AI algorithm parsing text, determining feeling and responding with what it
thinks are relevant words. It couldn't be more on-topic than this. AI is
participating in HN discussions.

