
From Scratch: AI Balancing Act in 50 Lines of Python - amasad
https://towardsdatascience.com/from-scratch-ai-balancing-act-in-50-lines-of-python-7ea67ef717
======
X6S1x6Okd1st
Initializing the policy to random and performing thousands of trials is just
about the simplest implementation of "AI" you could possibly ask for.

It's nice to see someone actually going for the MVP.

~~~
mikeshi42
The mind blowing thing is that it actually works, and with rand(0,1), it'll
get you results with only 10 guesses (though it's a bit cheating haha).

Fun fact: a policy of [1,2,3,4] will solve the game.

When I started off the article, I was thinking of doing some kind of
evolutionary-based policy search but turns out none of that is even necessary.

I hope to have a more technical follow up w/ deeper analysis and hopefully
another one that goes into harder games where random guessing is okay, but
there are more "sample efficient" methods.

~~~
X6S1x6Okd1st
Yeah I've been meaning to work my way up the complexity ladder on openai's
gym.

Seeing that it can be reasonably done by a random walk and the fact that it's
only 4 dimensions kind of makes me want to do a uniform convering of the space
and visualize the results.

~~~
mikeshi42
You should! I did some analysis myself a bit, have to dust it off to make it
bit more presentable. There are definitely clear boundaries of "good"
policies.

------
esaym
Not sure why this has to be AI specific.. Terry Davis was doing this 15+ years
ago: [https://youtu.be/25hLohVZdME](https://youtu.be/25hLohVZdME)

Seems to be more of an issue of knowing calculus and control system patterns.

~~~
X6S1x6Okd1st
It's not. It's a trivial problem to have something easy to do when starting
out with reinforcement learning.

------
skykooler
Discussing "state":

> In the game of pong, it might be the vertical position of each paddle and
> the x, y coordinate of the ball.

The velocity vector of the ball is also a (very important!) part of the state.

~~~
mikeshi42
Good point! Edited the post :)

(Admittedly I've never built anything to play pong )

