Granted, a lot of RL thought pieces/examples on places like Medium.com take an existing RL implementation without many tweaks, run it on a new task, and see what happens. A better RL library might make this workflow more prevalent; hence why it's very important for researchers to make their pipelines transparent.
In my opinion PyTorch code is easier to understand and debug for newcomers. Code is definitely lacking in documentation, but whenever there was a tradeoff between clarity and modularity in the end I've chosen modularity. Ideally I would like others to be able to take bits and pieces and incorporate into their projects to speed up time to delivery of their ideas.
PyTorch with its explicit state that can be easily examined by hand in PyCharm debugger will be way easier for people coming into the field.
It helps me to understand something new if I can controllably break it. In other words, I progress by predicting the edge-conditions when something shouldn't work - and then testing if algorithm indeed experienced expected type of failure. Transparent algorithm implementation is key for this.
One thing, which I immediately checked in the spinningup-repo is if it uses TF Eager. And it doesn't. @OpenAI what's your reasoning for that?
At the six month review in 2019, we'll evaluate whether it makes sense to rewrite the implementation examples for TF 2.0. I'll speculate that the answer will be "yes, it does." Since Eager execution will be a central feature of TF 2.0, the (probable) revamp for Spinning Up will include it.
Good luck with your experiments! And please let us know about your experience with Spinning Up---we want to make sure it fulfills the mission of helping anyone learn about deep RL, and user feedback is vital for that.
this time...I promise myself its different
I don't have access to a computer with GPU and I am currently using google colab to do my DL projects. I tried installing Mujoco on colab but unfortunately, the computer id generated seems invalid. Any help is highly appreciated.
A few people had this question on Twitter also. Our response: "Several of us at OpenAI are thinking seriously about how to make something like this happen! I can't promise anything, but we definitely want to remove barriers to entry." (https://twitter.com/jachiam0/status/1060595172285632512)
In the meanwhile, you can still use Spinning Up with the Classic Control and Box2d envs in Gym (which don't require any licenses at all). And what's more: for most of these environments you don't need a GPU! CPU is fine.