
Artificial intelligence: Riders on a swarm  - iamelgringo
http://www.economist.com/node/16789226?story_id=16789226&fsrc=rss&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+economist%2Ffull_print_edition+%28The+Economist%3A+Full+print+edition%29
======
tansey
> The search for artificial intelligence modelled on human brains has been a
> dismal failure.

No, it hasn't. There have been huge strides made in artificial neural networks
in the last decade. One example is the HyperNEAT algorithm [1], which uses an
indirect encoding enabling it to evolve networks with millions of connections.
There's an entire conference on Neural Information Processing Systems (NIPS),
which is considered one of the most prestigious publication venues in AI.

This article is complete garbage. Ant colony optimization has been around for
decades. It's great for routing and similar tasks where you need to find the
best path and be able to handle breakdowns in that path. However, there is no
basis for making the leap that human brains function like ant colonies.

[1] Stanley et. al. A Hypercube-Based Encoding for Evolving Large-Scale Neural
Networks. In: Artificial Life journal. Cambridge, MA: MIT Press, 2009.
[http://eplex.cs.ucf.edu/publications/2009/stanley.alife09.ht...](http://eplex.cs.ucf.edu/publications/2009/stanley.alife09.html)

~~~
another
To disagree on a side issue, it's been a long time since NIPS dealt much with
classic ANNs (which never had much to do with human brains, in any case). Most
of the action there, as in AI and ML at large these days---eg, the focus also
at ICML and at other venues---is in statistical methods.

(On the other hand, neuroscience and explicitly biological neural modeling are
exciting areas, reasonably well-represented at NIPS. Those topics, however,
are almost entirely different from neural networks of the multilayer
perceptron / [Hyper]NEAT varieties.)

But your criticism of the article seems accurate. ACO isn't new, and there's
little evidence that it will solve any of the major outstanding problems in
AI.

Less generously, however, I'd suggest that much of the research related to the
family of population-based stochastic search methods, ACO, PSO, and HyperNEAT
included, is prone to the same risk: the lack of a field-wide theoretical
foundation, coupled with the absence of a field-wide standard methodology and
benchmark set for empirical comparison (as opposed to, say, the situation in
supervised learning), makes it temptingly easy for a particular researcher to
believe too strongly in the capabilities of that researcher's pet algorithm.
This situation seems to have balkanized the field (page through a recent GECCO
proceedings, for example), and holds back wider progress.

That's not to say that HyperNEAT can't do great things. It's a fun approach,
and Stanley et al are running far with it. But your boosterism of it, and the
boosterism of ACO that you're objecting to, seem closely related.

(For contrast, I'd suggest, eg, the natural gradient work at IDSIA. It's
unlikely to be the ultimate method, but may be a good model for solid research
in this area.)

~~~
SeanLuke
> This situation seems to have balkanized the field (page through a recent
> GECCO proceedings, for example), and holds back wider progress.

Stochastic optimization has always been balkanized, but not because of a lack
of methodology or benchmarks, but rather because the field was simultaneously
invented in several different locations (Evolutionary Programming in San Diego
and at NSF, Evolution Strategies in Germany, the Genetic Algorithm in
Michigan). That's what happens in such situations: you get competition and
differentiation when there really isn't much. Due to various political
reasons, GECCO and CEC broke apart and GECCO itself decided to divide itself
up by topic. In the meantime, alternative methods (ACO, PSO, various single-
state "metaheuristics", etc.) have danced about at the periphery.

I think your criticism of benchmarks is much too harsh. First off, it's not
really true. A number of areas in evolutionary computation have very well
established benchmarks: certainly this is the case for genetic programming;
and for vector representations (Rastrigin? Rosenbrock? Schwefel? The De Jong
test suite? Griewangk? Etc.). Also multiobjective optimization has established
a fairly common set of benchmarks. Second, much of the remainder of the field
consists of different kinds of solution representations (ACO; various graph
representations such as NEAT/HyperNEAT; list representations; etc.) and in
such situations unifying benchmarks make no sense. It's like insisting that
the Iris data set be used for problems in text mining.

I think there's quite a large degree of theoretical work among these
techniques. Indeed there are entire theoretical conferences. But if you're
looking for field-wide theoretical foundation, you're barking up the wrong
tree. The problem is that most "interesting" solution representations result
in dynamics which are essentially impossible to base a theoretical foundation
on. In a real sense, various other fields have theoretical foundations because
their problems are well formed. Stochastic optimization by its nature is
tackling nastier problems for which there is no well-formed solution concept.
The problems are ugly and hairy. This shouldn't reflect on the researchers
brave enough to tackle them.

~~~
another
Thanks, enjoyed your response.

Your "tackling nastier problems" point is well-taken, and one reason I have a
great deal of sympathy for work in these research areas.

I think that you're painting an overly-rosy picture of the benchmark
situation, but you're right in that I was likely too critical. The attention
paid to synthetic fitness landscapes, for example, is also a positive sign
that that many researchers do care about issues of comparison and analysis.

(That said, I obviously believe that the field can and must do much better---
and, since it relies so heavily on empirical evidence, its need for consistent
methodology is somewhat higher.)

------
retube
"The purposeful collective activity of ants and other social insects does,
indeed, look intelligent on the surface. An illusion, presumably. "

No more an illusion than the individual intelligence/consciousness we humans
experience.

------
endtime
>In particular, Dr Dorigo was interested to learn that ants are good at
choosing the shortest possible route between a food source and their nest.
This is reminiscent of a classic computational conundrum, the travelling-
salesman problem.

Ugh.

~~~
lylejohnson
I'm not sure if you truncated that excerpt on purpose or not, but the sentence
that follows it indicates that the author _does_ actually know what the TSP
is.

> In particular, Dr Dorigo was interested to learn that ants are good at
> choosing the shortest possible route between a food source and their nest.
> This is reminiscent of a classic computational conundrum, the travelling-
> salesman problem. Given a list of cities and their distances apart, the
> salesman must find the shortest route needed to visit each city once.

~~~
endtime
No, it shows that he looked it up and failed to understand it.

------
grg
Very interesting article. The Hard AI problem is one of the new frontiers of
science.

If you're interested in this topic, I highly suggest you check out the
Radiolab episode on emergence.

The episode doesn't focus on AI, per se, but it does talk a lot about how many
individual things (ants, fireflies, bees) are not intelligent on their own,
but do appear intelligent as a collective whole.

Here's the link: <http://www.wnyc.org/shows/radiolab/episodes/2005/02/18>

------
kgosser
I posted this article almost a week ago :-(

Good article though.

------
mirkules
A really good book on this subject is Swarm Intelligence by Russell Eberhart

[http://www.amazon.com/Intelligence-Morgan-Kaufmann-
Evolution...](http://www.amazon.com/Intelligence-Morgan-Kaufmann-Evolutionary-
Computation/dp/1558605959/ref=sr_1_1?ie=UTF8&s=books&qid=1282352355&sr=8-1)

(I'm not affiliated with the book in any way :)

------
tswicegood
Good companion resource to the article:
<http://en.wikipedia.org/wiki/Ant_colony_optimization>

------
masterponomo
Ray Kurzweil does not understand the ants and the bees.

------
presidentender
Goedel, Escher, Bach applies yet again....

