
Using the Monte Carlo Tree Search Algorithm in an AI to Beat 2048 - xtrp
https://xtrp.io/blog/2020/09/12/using-the-monte-carlo-tree-search-algorithm-in-an-ai-to-beat-2048-and-other-games/
======
instance
I think the title is a bit misleading, this is a classic Monte Carlo method,
but not MCTS per se. There is no tree structure built, but simply in the
current state simulations are run for each action and the best one is picked.

~~~
plancien
I agree, this article is well explained but it's not MCTS. To be more
accurate, this is Monte Carlo, and here it is applied to search the tree of
game turn possibilities and doing stats on it. But Monte-Carlo Tree Search is
a "reserved" name of a specific algorithm, involving more than what is done in
the article : some kind of caching, and some optimisations. In fact, it would
be a great addition to this article to do a part 2, and explaining how to
extend the code to do MCTS. I imagine this two way trip would be a better MCTS
presentation than the classic ones describing the algorithm step by step in
one shot.

~~~
Mageek
It would be very interesting to see proper MCTS applied to 2048. Monte Carlo
Tree Search performs well on problems with linear reward. 2048 has exponential
reward, with subsequent states having much higher tile scores. This
nonlinearity tends to cause MCTS to fall into traps - one rollout will find a
state evaluation with a much higher score, and will thereafter bias all
subsequent rollouts to that same (potentially suboptimal) path. If there are
clever ways around this, such as board evaluation heuristics that scale
linearly, then I'd love to see them. In practice, a minimax approach that
enumerates all successor states up to a particular depth has worked best for
me.

------
mrkramer
I was surprised how article is easy to understand and then I figured out the
author is high school student. Impressive knowledge and skills for your age.

~~~
xtrp
Thank you, I'm glad you enjoyed the article!

— Gabriel

------
xiaodai
purely MCTS with no reinforcement learning techinques invovled.

I was making something with a actor-critic with MCTS that can assess the board
position as well.

