
CleanRL: RL library focusing on experimental research (beginner-friendly) - vwxyzjn
https://github.com/vwxyzjn/cleanrl
======
vwxyzjn
Hi everyone,

Author here. I have created a yet another reinforcement learning library.
[[https://github.com/vwxyzjn/cleanrl](https://github.com/vwxyzjn/cleanrl)]

This repository focuses on a clean and minimal implementation of reinforcement
learning algorithms. The highlights features of this repo are:

* Most algorithms are self-contained in single files with a common dependency file common.py that handles different gym spaces. * Easy logging of training processes using Tensorboard and Integration with wandb.com to log experiments on the cloud. Check out [[https://app.wandb.ai/costa-huang/cleanrltest](https://app.wandb.ai/costa-huang/cleanrltest)] * Easily customizable and being able to debug directly in Python’s interactive shell. * Convenient use of commandline arguments for hyper-parameters tuning.

Currently I support A2C, PPO, and DQN. If you are interested, please consider
giving it a try :)

Motivation:

There are two types of RL library on the two ends of the spectrum. The first
one is the demo kind that really just demos what the algorithm is doing, only
deals with one gym environment and hard to record experiments and tune
parameters.

On the other end of the spectrum, we have OpenAI/baselines, ray-project/ray,
and couple google repos. My personal experience with them is that I could only
run benchmark with them. They try to write modular code and employ good
software engineering practices, but the problem is _python is a dynamic
language without IDE support_. As a result, I had no idea what variable types
in different files are and it was very difficult to do any kind of
customization. I had to see through dozens of files before even able to try
some experiments.

That’s why I created this repo that leans towards the first kind, but has more
actual experimental support. I support multiple gym spaces (still working on
it), command line arguments to tune parameters, and very seamless experiment
logging, all of which are essential characteristics for building a pipeline
for research I believe.

