This is better than the vast majority of players on lichess.org, and also much better than GPT-4.
I checked a random sample of games manually, and all of them quickly diverged from each other and from the lichess database (of ~5B games), so it is inconceivable that the training data contained a significant portion of these games.
Hopefully OpenAI releases an instruct version of GPT-4 so we can see if scaling continues to bear fruit.
EDIT: I played against it once. It got a winning position and then blundered mate in 1. Considering the chat model can barely find legal moves this is still very impressive IMO.
I checked a random sample of games manually, and all of them quickly diverged from each other and from the lichess database (of ~5B games), so it is inconceivable that the training data contained a significant portion of these games.
Hopefully OpenAI releases an instruct version of GPT-4 so we can see if scaling continues to bear fruit.
EDIT: I played against it once. It got a winning position and then blundered mate in 1. Considering the chat model can barely find legal moves this is still very impressive IMO.