
Reinforcement Learning Resources, Models and Code - kohjingyu
https://modelzoo.co/blog/reinforcement-learning-resources-models-and-code
======
minimaxir
Unlike normal models with clear inputs and outputs with numerous methods of
validating robustness of results, reinforcement learning requires a _lot_ more
dedication for a _chance_ at the model converging and actually learning, with
_substantially_ more training time/cost needed.

Even with a higher-level library, it's not plug-and-play on new games
(although tools like OpenAI Gym make it substantially easier), and you'll
still need to do a _lot_ of tuning.

------
throwawaymath
Machine learning seems especially prone to attracting "aggregators" \-
resources whose sole purpose is to enthusiastically collect, promote and/or
promulgate _other_ resources for learning about the subject (or one of its
sub-disciplines). I think we're starting to get at least one of these a week.

That observation aside, this isn't the first time this has been submitted by
the author. That's not a problem in of itself, but I'd like to repeat part of
my original critique [1] of this website when it was submitted a few months
ago.

The author is pretty transparent about the way in which these resources are
collected - they're automatically scraped from GitHub. I don't think that's
intrinsically _bad_ , but it does mean that due diligence is somewhat lacking.
For example, consider the "model" page for Chess Reinforcement Learning:
[https://modelzoo.co/model/chess-reinforcement-
learning](https://modelzoo.co/model/chess-reinforcement-learning). That lists
the README for the corresponding GitHub repository [2] verbatim. But the
README specifically states that work on the model has been discontinued in
favor of a new, better model's repository [3]. Additionally, the authors
mention that they had difficulty with some of the implementation related to
Self-Play.

Now I feel a philosophical point is in order. Ostensibly, an aggregator for
resources should helpfully do all or most of the heavy lifting in 1) finding
the resources, 2) vetting their completeness as advertised, and 3) maintaining
up to date references for the resources it lists. There was no way to
distinguish the limitations or obsolescence of the aforementioned model in
first looking at it. Rather I had to open the page, click "Get Model" and read
through the GitHub README on my own. I see substantially little difference
between this activity and simply searching GitHub, considering that every
single model I can find is listed on GitHub.

This is not to say that this project is _bad_ , I just don't think it's
particularly useful in its current form. I understand that's harsh feedback.
But in my opinion the author would need to perform far more due diligence on
the included models, the implementation artifacts of included models, the
people implementing the models themselves and the methodology for model
inclusion. As it stands this is presently a collection of models which are in
varying states of completeness, developed by people who may or may not be
authors of the papers they're implementing (or even have much implementation
experience at all), and with varying levels of "out of the box" utility.

___________________

1\.
[https://news.ycombinator.com/item?id=17311346](https://news.ycombinator.com/item?id=17311346)

2\. [https://github.com/Zeta36/chess-alpha-
zero](https://github.com/Zeta36/chess-alpha-zero)

3\. [https://github.com/glinscott/leela-
chess](https://github.com/glinscott/leela-chess)

