
A Deep Attention-Based Reinforcement Learning Algorithm for Model-Based Control - ctoth
https://arxiv.org/abs/1812.09968
======
xiphias2
If you look at the old Singularity Conference presentation of Demis Hassabis
(founder of Deep Mind), his main argument for using attention, reinforcement
learning and deep convolutional networks (and other known systems that were on
his slides) for reaching AGI is to use only techniques that the brain uses to
be able to decrease the search space for reaching AGI.

I believe his thinking was already proven, and going for emulating and
integrating the known algorithms that the brain uses is the fastest way to
reach AGI.

[https://youtu.be/Qgd3OK5DZWI](https://youtu.be/Qgd3OK5DZWI)

~~~
jimfleming
> I believe his thinking was already proven, and going for emulating and
> integrating the known algorithms that the brain uses is the fastest way to
> reach AGI.

That's a stretch considering AGI has not yet been created by DeepMind or
anyone else. Notable is that DeepMind's most prominent successes have relied
heavily on MCTS, a classical planning method that doesn't have much relation
to neuroscience without a lot of caveats. Their accomplishments on Atari lean
a lot more on efficient computing than biological plausibility.

I think the strategy they're actually following (and I believe they've said
this more recently) is to use what works and to look to neuroscience when
other methods fail. This feels more solid than looking to the brain first to
narrow the search space, which is the approach Numenta has taken, and does not
scale as easily.

~~~
xiphias2
You're right with using MCTS, but ,,Their accomplishments on Atari lean a lot
more on efficient computing than biological plausibility'' is a strange thing
to write.

Efficient computing is of course needed for AGI, I think that was never a
question. The question is what algorithms to use on the computers, and also
what computing architectures should be created for those algorithms.

Those Atari simulations were the first ones putting together reinforcement
learning and deep convolutional networks AFAIK, and yes, tree search was
needed (which human brains are consciously doing, but extremely bad at
compared to computers).

Just looking at what works is not enough. There was a strong reason why
DeepMind didn't start with modelling language or logical reasoning, like many
other people, and the background was based in biology (animal behaviour).

~~~
jimfleming
My point is that DQN is pretty far removed from the biological equivalent.
It's impressive and useful but the main reason it succeeded was not because of
some deep insight from neuroscience but because it scaled well (or at least
better than alternatives at the time).

EDIT: Richard Sutton (largely credited as the grandfather of RL) has written
about this recently:
[http://incompleteideas.net/IncIdeas/BitterLesson.html](http://incompleteideas.net/IncIdeas/BitterLesson.html)

