
Beating Atari Games with OpenAI’s Evolutionary Strategies - hippidyhoffider
https://blog.filestack.com/thoughts-and-knowledge/atari-openai-evolutionary-strategies/
======
techfanatic
I see the mention of memory consumption being lower, any idea on compute load
for ES models vs reinforcement learning?

~~~
staturecrane
Reinforcement learning using gradient descent, for instance, requires
backpropgation and thus much higher memory loads, as each parameter gets
updated via the chain rule. ES does not require gradient descent and so each
individual agent has much lower memory consumption during training than a deep
Q-learning agent would.

You may end up using more memory in ES, but only because you can parallelize
with as many processes as your system can handle.

------
tree_of_item
What's the difference between "evolutionary strategies" and "genetic
algorithms"?

~~~
staturecrane
From Quora ([https://www.quora.com/What-is-the-difference-between-
genetic...](https://www.quora.com/What-is-the-difference-between-genetic-and-
evolutionary-algorithms)):

"Genetic Algorithms (GA), characterized by a binary string representation of
the candidate solutions.

Evolution Strategies (ES), which use vectors of real-valued numbers as
representation. One of the most famous ES is the CMA-ES , (Covariance Matrix
Adaptation ES).

Genetic Programming (GP), one of the most recent techniques (1990s), that uses
binary trees (or other complex structures, such as directed graphs) as
internal representation of candidate solutions. This type of representation is
incredibly powerful, as it allows for evolving almost everything, from
Bayesian Networks to equations, from Assembly programs to gene regulatory
networks."

~~~
tree_of_item
Yeah, I saw this answer and it seems wrong. I really doubt the difference is
"binary string vs. vector of real valued numbers". I've seen things
characterized as GAs that use vectors of real numbers.

Anyone familiar with the subject willing to answer this themselves?

~~~
qbrass
I wouldn't call myself familiar, but from just looking at how they both work,
evolutionary strategies don't appear to have a crossover step.

