
Reinforcement Learning with Unsupervised Auxiliary Tasks - tonybeltramelli
https://arxiv.org/abs/1611.05397
======
TFortunato
Interesting. I'm not a deep learning guy, but from what I can gather, the new
auxiliary tasks are to be rewarded for "pixel changes" and "network features".

I haven't nearly finished reading the paper, but is it safe to say this is
similar (at a very high level), to a type of "novelty search", where the agent
is searching not only for a policy that is directly accomplishing the task at
hand, but also for novel stimulus (in the case of pixel changes), and novel
internal states (features, or maximally activated hidden nodes in the language
of the paper), and that the benefit of this would be to more easily find
relevant features that could be useful in the "big picture" task, and maybe
not get as stuck in a non-optimal policy?

(I may be understanding this completely wrong...just an embedded guy looking
to get more into this world, haha)

~~~
TFortunato
Quick reply to my own comment. But another reason it seems this is helpful is
that in many reinforcement learning tasks, like games, rewards are few and far
between, so these goals also give the agent oncentive to keep trying new
things / learning about the environment / task in the presence of sparse
rewards?

~~~
mmusson
It seems to me like an attempt to model curiosity.

~~~
TFortunato
Exactly what I was thinking, and put much more succinctly!

------
gallerdude
I remember seeing a lecture on the importance of novelty in these kinds of
things - good to see it applied.

------
deepnotderp
Is this an attempt at unsupervised action decomposition or artificial
curiosity?

