
Show HN: A batched, concurrent wrapper for OpenAI gym library - MutatedFlood
Hello HN, new user here, so please let me know if I break some rules.<p>Currently I&#x27;ve been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it.<p>I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested.<p>For more details, please visit: https:&#x2F;&#x2F;github.com&#x2F;Chimpan-Z&#x2F;agymc<p>Feedback really appreciated! Have a good day everyone!
======
mooneater
Is this different than VecEnv in baselines?

[https://github.com/openai/baselines/blob/master/baselines/co...](https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py)

~~~
MutatedFlood
Thanks for the feedback.

OpenAI said they used multiple processes (it seems that they used
multiprocessing module from your link). Processes is handled by OS and, it
depends, sometimes will only switch to another process on long CPU stalls. I
used an event loop to manage that, which supposedly should be fixed those long
stalls.

Also, this version runs on a single CPU while OpenAI's probably won't, given a
multicore CPU. My initial thought was that extending using multiprocessing is
much easier than wrapping it using an event loop, hence I only implemented the
event loop part.

~~~
mooneater
Thanks!

------
wookayin
When simulation is CPU-heavy, it will never scale. Though I admit that event-
based concurrency might work better in many cases (like where communication
overhead is greater than computation), but in some environments you would need
real parallelism using many CPU cores. This library also lacks benchmark.

~~~
MutatedFlood
You're right. I plan to add multiprocessing support such that there would be
_numCPU_ processes, each running one event loop. Which should scale properly.

What kinds of benchmark do you suggest? I'll do my best to provide one.

Thanks for the feedback.

