
The Unreasonable Reputation of Neural Networks - fforflo
http://thinkingmachines.mit.edu/blog/unreasonable-reputation-neural-networks
======
hacker_9
_" Human or superhuman performance in one task is not necessarily a stepping-
stone towards near-human performance across most tasks.

By the same token, the ability of neural networks to learn interpretable word
embeddings, say, does not remotely suggest that they are the right kind of
tool for a human-level understanding of the world. It is impressive and
surprising that these general-purpose, statistical models can learn meaningful
relations from text alone, without any richer perception of the world, but
this may speak much more about the unexpected ease of the task itself than it
does about the capacity of the models. Just as checkers can be won through
tree-search, so too can many semantic relations be learned from text
statistics. Both produce impressive intelligent-seeming behaviour, but neither
necessarily pave the way towards true machine intelligence."_

So true, and this is why I don't listen when Elon Musk or Stephen Hawkings
spread fear about the impending AI disaster; they think because a neural
network can recognize an image like a human can, that it's not a huge leap to
say it will be able to soon think and act like a human, but in reality this is
just not the case.

~~~
rw
I agree that the rate of progress in AI is unpredictable, which means we are
probably not right on the cusp of superhuman AI. But what if we actually _are_
on the precipice? How could you tell? You seem to be taking a bet on the
following statement:

"Before superhuman AI is developed, the techniques that make it possible will
_look dangerous_."

That's a risky proposition. There's _so much_ to lose in this situation. Elon
Musk, Stephen Hawking, and many other people are taking a different, more
risk-averse bet:

"Superhuman AI is extremely dangerous, so we need to be pessimistic about how
good we are at predicting when it will happen."

By that logic, general AI is to be feared and worried about right now, because
our predictive abilities are imperfect. The AI trend is towards more danger,
not less: there's a slight chance of a cataclysmic event happening today, and
as time goes on, the likelihood of it happening will increase (due to ongoing
R&D in AI).

(As an aside, if you want to learn how hard the "AI Control Problem" is, I
recommend the book "Superintelligence" by Nick Bostrum.)

~~~
vonnik
The real cost of fearing an AGI now is an opportunity cost. The technology is
a long ways away. Other events -- such as ecological collapse or nuclear
disaster to name two -- are much more likely to irremediably affect our lives.
So by hopping on Bostrom's fear wagon, you are making a bet on several levels:
a) this matters now; b) this matters more than other dangers we could devote
ourselves to minimizing; and c) something can actually be done about it. All
three bets seem like bad ones to make to me.

~~~
rw
Fair comparison, but I didn't intend any monopoly on fear. All I'm saying is
that the risk of GAI should be taken seriously, calibrated along with the
other problems you mention. There's a lot of room between 'irrelevant' and
'the most important issue' :-)

~~~
argonaut
Yes, and given the $1B in funding that OpenAI has, I'd seriously say that too
much attention is given to the AI "risk."

~~~
Houshalter
OpenAI has nothing to do with AI risk. They want to accelerate research in AI
and make it open to the public. Which is the exact opposite you would want to
do to protect against superintelligent AIs. They are not focusing on solving
the control problem, or anything meaningful.

~~~
vonnik
Actually, @Houshalter, OpenAI has several purposes, and one of them is to
address AI risk. You may not think that is an effective approach, but that is
their stated intent. They believe that the best hedge against the dangers of
superintelligence is to conduct open research that encourages a multipolar
world of several AGIs, rather a unipolar world where one company, such as
Google, reaps the benefits and determines the course of such an advance. It's
true that OpenAI accelerates AI research, which makes it paradoxical. But
that's the nature of a hedge. You bet on both sides at once. I do believe that
they will focus on solving meaningful problems (they are only a few weeks
old...).

~~~
Houshalter
>rather a unipolar world where one company, such as Google, reaps the benefits
and determines the course of such an advance

Yes that is their model of AI risk. However the people worried about AI risk
are not worried about any single organization controlling AI. They are worried
about a world where no one controls AI.

Controlling AI is actually a very difficult and unsolved problem. OpenAI is
not intended to solve that problem, and if anything they are making it far
worse by accelerating AI research before the problem has been solved.

~~~
argonaut
What do you mean by unsolved problem? Controlling AI is not even an actual
problem at all yet. It's merely a hypothetical problem.

------
andreyk
I think this is a good analysis of what Deep Learning is particularly good for
and its limitations, but was somewhat annoyed at the lack of any citations of
people actually overhyping it. The most there was is this:

"This is all well justified, and I have no intention to belittle the current
and future impact of deep learning; however, the optimism about the just what
these models can achieve in terms of intelligence has been worryingly
reminiscent of the 1960s."

From what I've read and seen, the leading people in the field (Yann LeCun,
Hinton, etc.) seem to be very aware that the current methods are particularly
good for problems dealing with perception but not necessarily reasoning.
Likewise, I have not seen many popular news sources such as NYT make any crazy
claims about the potential of the technology. I hope, at least, that the
people who work in AI are too aware of the hype cycles of the past to get
caught up in one again, and so there will not be a repeat of the 60's.

~~~
snoman
> I hope, at least, that the people who work in AI are too aware of the hype
> cycles of the past to get caught up in one again, and so there will not be a
> repeat of the 60's.

Given that AI Winter is a thing, and that it was a reaction to the across-the-
board failure of AI to do anything that people expected of it (expectations
driven by hype), then I think you'd be right.

------
boltzmannbrain
I think readers of this post will enjoy "What is Machine Intelligence vs. Deep
Learning vs. Artificial Intelligence" by Numenta's Jeff Hawkins:
[http://numenta.com/blog/machine-intelligence-machine-
learnin...](http://numenta.com/blog/machine-intelligence-machine-learning-
deep-learning-artificial-intelligence.html)

~~~
jaytaylor
Thank you for sharing this! I found it a concise and useful overview. Also
submitted it to HN:
[https://news.ycombinator.com/item?id=10921440](https://news.ycombinator.com/item?id=10921440)

------
tacos
Current top post quotes the most negative observation of the paper. Here's the
most positive, and perhaps the most useful to HN readers or investors who are
exploring the space:

 _" Deep learning has produced amazing discriminative models, generative
models and feature extractors, but common to all of these is the use of a very
large training dataset. Its place in the world is as a powerful tool for
general-purpose pattern recognition... Very possibly it is the best tool for
working in this paradigm. This is a very good fit for one particular class of
problems that the brain solves: finding good representations to describe the
constant and enormous flood of sensory data it receives."_

~~~
auvrw
the last sentence is also positive as well as insightful

 _Gradient descent in neural networks may well play a big part in helping to
build the components of thinking machines, but it is not, itself, the stuff of
thought._

statistical techniques like these are often used to interpret experimental
data. "helping to build the components of thinking machines," (and i think
this is the author's intent?) may not mean _being_ the components of thinking
machines, just as the 4004 is not a component of the computer that i'm
currently using, but it did help build it.

idk, though. as cool as the engineering (making) is, i do worry sometimes that
the science (understanding) could get overlooked in the process.

------
theideasmith
Someone once gave the analogy of climbing to the moon. You can report steady
progress until you get to the top of the tree/mountain. I think this is
applicable here. We'll need a new paradigm, beyond statistical learning, to
create AGI

~~~
proc0
David Deustch has an article online somewhere and his conclusion is that the
leap has to be made by philosophy first. We need to first figure out the
theory of intelligence and potentially consciousness first, and then
information theory will follow.

~~~
upquark
Same could've been said about every aspect of human progress, if we had asked
philosophers to opine (they opine anyway, but the civilization moves forward
regardless).

------
proc0
Another article basically saying something along the lines of "there is no
current technology that comes close to producing AGI, therefore let's dismiss
all these technologies". Of course we don't know what we don't know, _until we
do_ , and then it's not as mysterious.

It's not hard to see that the reason NN are becoming the prime candidate for
AGI, is because of their inspired architecture based on biological neurons. We
are the only known AGI, therefore something similar to the brain will be
producing an AGI. NN at least mimic the massively parallel property of
biological neurons. And if we're optimistic, the fact that NN is mimicking how
vision works in our brain, might mean that we are at some point in the
continuum of the evolution of brains, and it's a matter of time until we
discover the other ways brains evolved intelligence.

What keeps me optimistic is evolution. At some point brains were stupid, and
then they definitely evolved AGI. The question is how did this happen and
whether or not there is a shortcut, like inventing the wheel for
transportation instead of arms and legs.

~~~
argonaut
To paraphrase a comment from another thread:

Artificial neural networks have basically nothing to do with actual biological
neurons. First, neuroscientists do not have even a decent understanding of
biological neurons: you cannot say that neural networks mimic neurons when we
don't understand biological neurons. Of the bajillion things we _do_ know
neurons do, neural networks do on the order of 1% of those things. On the flip
side, neural networks do a ton of things that biological neurons do not (like
_backpropagation_ (!)* ).

* In fairness, Hinton hypothesizes the brain has a way of doing backprop, but what he talks about only barely resembles actual backprop.

~~~
bglazer
I've heard people suggest a comparison between birds wings and planes. We
don't understand __everything __about bird wings. They 're incredibly complex
and messy biological structures. However, we do know enough about the general
principles of aerodynamics to build aircraft. Perhaps a similar dynamic is at
work here. We don't understand brains, but perhaps neural networks are able to
capture the same principles of information processing.

~~~
panic
Yes, plane wings are built on fundamental physical principles. We do not know
the fundamental physical principles of intelligence.

Building an NN and hoping its brain-like structure will start to think is like
building a flapping machine and hoping its bird-like structure will start to
fly.

~~~
kanzure
> Building an NN and hoping its brain-like structure will start to think is
> like building a flapping machine and hoping its bird-like structure will
> start to fly.

But... you're replying in a comment thread that has explicitly stated that
"NNs" don't at all qualify as "brain-like".

~~~
argonaut
Which was my response to someone implying we should make them more brain-like.
So the comment was quite relevant.

------
maciejgryka
Nice article - it's good to be realistic about what we can do with current
tools.

I feel like the gist of what current neural nets can do is "pattern
recognition". If that's fair, I also suspect that most people underestimate
how many problems can be solved by them (e.g. planning and experiment design
can be posed as pattern recognition - the difficulty is obtaining enough
training data).

It's true that we're most likely a very long way away from general AI - but
I'm willing to bet most of us will still be surprised within the next 2 years
by just how well some deep-learning based solutions work.

------
Houshalter
>Human or superhuman performance in one task is not necessarily a stepping-
stone towards near-human performance across most tasks.

Here's the important difference about NNs. They are incredibly general. The
same algorithms that can do object recognition can also do language tasks,
learn to play chess or go, control a robot, etc. With only slightly
modifications to the architecture and otherwise no domain information.

That's a hugely different thing than brute force game playing programs. Not
only could they not learn the rules of the game from no knowledge, they
couldn't even play games with large search spaces like Go. They couldn't do
anything other than play games with well defined rules. They are not general
at all.

Current neural networks have limits. But there is no reason to believe that
those limits can't be broken as more progress is made.

For example, the author references that neural networks overfit. They can't
make predictions when they have little data. They need huge amounts of data to
do well.

But this is a problem that has already been solved to some extent. There has
been a great deal of work into bayesian neural networks that avoid overfitting
entirely. Including some recent papers on new methods to do them efficiently.
There's the invention of dropout, which is believed to approximate bayesian
methods, and is very good at avoiding overfitting.

There are some tasks that neural network can't do, like episodic memory, and
reasoning. And there has been recent work exploring these tasks. We are
starting to see neural networks with external memory systems attached to them,
or ways of learning to store memories. Neuroscientists have claimed to have
made accurate models of the hippocampus. And deepmind said that was their next
step.

Reasoning is more complicated and no one knows exactly what is meant by it.
But we are starting to see RNNs that can learn to do more complicated
"thinking" tasks, like attention models, and neural turing machines, and RNNs
that are taught to model programming languages and code.

~~~
argonaut
Dropout does very little to solve the lack of data problem, speaking from
experience.

------
otakucode
I expect that as we improve machine intelligence more and more, aside from the
fact that we will simply keep moving the goalposts of what we consider
"intelligent" like the irascible scamps we are, we're going to discover that
embodiment is absolutely necessary. Not just any embodiment either, but we
will need to place the neural networks in bodies very much like our own.
Neuroscience continues to find surprising things that link our "general human
intelligence" to our bodies. Paralyze a face and a person becomes less able to
feel happiness or anger, eventually forgetting what feeling those things even
meant, as one example.

We shouldn't forget that the mind/body split is a wholly artificial construct
that has no basis in reality. The brain is not contained in the head. The
nerves running down your spine and out to your toes and all over your body are
neurons. Exactly the same neurons, and directly connected to the neurons, that
make up what we think of as the separate organ 'the brain'. They're stretched
out very long, from head to toe, sure, but they are single cells, with the
exact same behavior and DNA, and there is no reason to presume that they must
have some especially insignificant role in our overall intelligence.

Then there is the fact that it is probably reasonable to presume that a
machine which has human-level intelligence will not appear overnight. It would
almost necessarily go through long periods of development. During that
development, when the machine begins to behave in ways the designers are not
able to understand, what will be their reaction? Will they suppose that maybe
the machine had intentions they were unaware of, and that it is acting of its
own volition? Or will they think the system must be flawed, and seek to
eliminate the behavior they didn't expect or understand?

I have a hard time imagining that an AI system will be trained on image
classification and one day suddenly say "I am alive" to its authors or users.
If it instead performs poorly on the image classification because it is
pondering the beauty of a flower in one of the images, what are the chances
that nascent quasi-consciousness would be protected and developed? I think
none. We only have vague ideas about intelligence and consciousness and our
ideas about partial intelligence are utterly theoretical. Has there ever been
a person who was 1% intelligent? Is mastering checkers, or learning NLP to
exclusion of even proprioception 1% of human intelligence? You optimize for
what you measure... and we don't know how to measure the things we're looking
for.

------
tim333
>Extrapolating from the last few years’ progress, it is enticing to believe
that Deep Artificial General Intelligence is just around the corner and just a
few more architectural tricks, bigger data sets and faster computing power are
required to take us there. I feel that there are a couple of solid reasons to
be much more skeptical.

On the other hand there are reasons to be optimistic. Human brains are built
from networks of neurons and the artificial neural networks are starting to
have quite similar characteristics to components of the brain - things like
image recognition
([https://news.ycombinator.com/item?id=9584325](https://news.ycombinator.com/item?id=9584325))
and Deep Mind playing Atari
([http://www.wired.co.uk/news/archive/2015-02/25/google-
deepmi...](http://www.wired.co.uk/news/archive/2015-02/25/google-deepmind-
atari))

The next step would may be to wire the things together in a similar structure
to the human brain which is kind of what Deep Mind are working on - they are
trying to do the hippocampus at the moment.
([https://www.youtube.com/watch?v=0X-NdPtFKq0&feature=youtu.be...](https://www.youtube.com/watch?v=0X-NdPtFKq0&feature=youtu.be&t=25m24s))

Also we are approaching the point where reasonably priced hardware can match
the brain, roughly the 2020s
([http://www.transhumanist.com/volume1/moravec.htm](http://www.transhumanist.com/volume1/moravec.htm))

It'll be interesting to see how it goes.

------
MichaelMoser123
I think that this book is really interesting "Surfaces and essences: Analogy
as the fuel and fire of thinking" by Hofstadter and Sander

Many people got dissilusioned with classical AI because mathematical logic
(inference engines) would not scale to 'strong' AI.

Hofstaedter says that most concepts handled by Humans do not fit into clear
cut onthologies one to one. Instead each higher order concepts are created by
finding analogies between objects or simpler concepts, and by grouping these
similar concepts into more complex entities.

I have a summary of the book here
[http://mosermichael.github.io/cstuff/all/blogg/2013/10/15/po...](http://mosermichael.github.io/cstuff/all/blogg/2013/10/15/post-1.html)

~~~
MichaelMoser123
My personal theory is: semantics of language are neatly/minimally represented
by dependency graphs. Maybe analogies can be found by matching colored
dependency graphs (where the nodes and links determine the coloring)

------
bsbechtel
We will never have human level AI until we can properly understand, define,
and model human intelligence. While we are advancing at a very rapid pace on
that front, we are still years away from the field being considered mature.

~~~
disantlor
Why? All sorts of behavior emerges out of seemingly simplified rules/systems.
Could the convoluted and beautiful patterns that can come from Conway's game
of life be predicted in advance? What about the sort of meta "intelligence" on
display by ant colonies when taken as a whole? Or how connecting disparate
groups of people over the globe changes the way humanity as a whole reacts to
situations.

I'm no expert and these examples probably betray the shallowness of my
understanding as much as they make a point. But if they do make a point it's
that we don't need to (and often can't) understand what will result from
systems that have a built in feedback mechanism.

~~~
bsbechtel
>>But if they do make a point it's that we don't need to (and often can't)
understand what will result from systems that have a built in feedback
mechanism.

That may be true, but my point was more to the effect that there are things we
fail to understand about human nature that drives decision making, actions,
and goal setting within our own society, let alone other cultures. Consider
how factors such as your own personal energy levels affect your decision
making throughout the day (decision fatigue)...how are we going to model
something like that with AI? Or consider that many different cultures around
the world have very different societal goals than Americans do pursuing
personal freedom and material well-being. How do we consider those different
abstract goals, that most citizens don't even realize or understand why they
pursue, because it's just how they have been raised. How do environmental
factors drive decision making in different climates? Those who live in a very
cold climate have to take different precautions and deal with different risks
than those in warm climates do. What about locally available resources? Many
business (and design) decisions are made based as much upon supply chain
availability and risk management as they are customer demand. How does AI
consider these factors when AI doesn't even know these things exist? These are
the things I was talking about that we still have difficulty understanding
about each other, let alone being able to program a robot to imitate.

~~~
disantlor
Maybe the difference between a human intelligence and a super intelligence
comes, in part, from not being affected by day to day mood/chemical processes
that are probably related to keeping the body as a whole functioning. The
super intelligence has an intelligence that's isolated from these needs.

I don't really have an answer to all your points, just a general feeling that
AI needed be just like our "I" to be effectively superior.

------
skybrian
There was a recent paper [1] about learning visual concepts from _few_
examples. I don't know if it generalizes or not, but it seems too early to
assume that researchers will hit a dead end.

[1]
[http://science.sciencemag.org/content/350/6266/1332.full](http://science.sciencemag.org/content/350/6266/1332.full)

~~~
argonaut
This is an algorithm unrelated to neural networks, which incidentally
reinforces the author's point.

------
tianlins
it is true that most recent success of deep neural network are in the regime
where n, d are large. And we surely shouldn't fantasize general AI solved in
this way. However, the very appealing aspect of deep neural network is end-to-
end training: for image recognition, we can map from raw-pixels to output.
This is very different from other ML techniques. In some sense, deep neural
networks learn "algorithms", not just "models". This formulation can be richer
especially when given lots of data.

------
sevensor
At last, the thing that's unreasonable _isn 't_ effectiveness. I've been
hoping for a while that someone close to the field would cut through the hype
and put ANNs in context.

------
interdrift
We have something that can understand a pattern but we don't have something
that can understand how different patterns relate to each other.

~~~
TeMPOraL
Relations of patterns are patterns too. A potential for recursive reuse?

~~~
wolfgke
Rather homotopy type theory ;-)

(explanation: homotopy type theory (HoTT) in difference to ITT also considers
types of higher order).

~~~
harveywi
To my knowledge, intuitionistic type theory (ITT) considers type of higher
order (called _higher-kinded_ types), so homotopy type theory (HoTT) isn't
exactly relevant here - especially since it has, to my knowledge, not cross-
pollinated with ML or ANN research. Models of recursion can be found in
systems as simple as lambda calculi or cellular spaces, and this was the big
idea behind Jeff Hawkins's HTM and other classical models (please correct me
if I am wrong).

What HoTT brings to the table is the notion that a path between two types can
be used to transport a proof from one type to another - currently immensely
handy for theorem proving, but applications in ML and beyond remain to be
made.

------
DrNuke
I think many are missing the point here: AI can just be very stupid and still
wipe everything out. It only takes some sort of irreversible minimisation
function to let machines destroy all at sight. Drones are the first step, then
comes IoT, what else? We fully depend on machine learning just now. So no
wonder many are scared even before machines becoming human-intelligent.

------
neom
How far are we from general purpose quantum computing?

