Hacker News new | more | comments | ask | show | jobs | submit login
Data Mining Reveals Factors That Lead People To Make Blunders in Chess (2016) (technologyreview.com)
92 points by tosh 41 days ago | hide | past | web | favorite | 37 comments

> The bottom line is that the difficulty of the decision is the most important factor in determining whether a player makes a mistake. In other words, examining the complexity of the board position is a much better predictor of whether a player is likely to blunder than his or her skill level or the amount of time left in the game.

I wonder if this also works the other way around with good moves instead of just avoiding blunders.

A parallel in business: we spend so much time thinking about hiring "A players" and "10x-ers". But could refocusing on better processes, environment, and goal-setting (reducing the complexity of the position) be just as effective for increasing performance?

On point. I think that also is applicable to your personal life btw. I had a short-lived existensial crisis a few days back, realizing that my life is getting too complex for me to properly manage it. Things were falling out of hand, I wasn’t making enough progress, etc. I even had a suicidal thought, my first in a very long while. But then reflecting on it I realized that this complexity is driven by my mounting desires. To improve my balance and regain the sense of control over my life, I should examine these desires and get rid of secondary ones, which would make my life simpler and I would be less prone to error (blunder in the article’s terminology). Instantly made me feel better :)

Our lives are frittered away by detail. Simplify, simplify, simplify. -hdt

As also suggested by lots of ancient religions and philosophers!

Seems like a “d’oh” finding

It's poorly written press coverage of https://arxiv.org/abs/1606.04956

tl;dr We find strong evidence that in our domain, features describing the inherent difficulty of an instance are significantly more powerful than features based on skill or time.

Does it mean the more complex a problem the more likely an error? (while skill and the time available to make a decision matter less)

More like if the problem doesn't fit our mental models we're likely to fall into a cognitive tunnel where we depend on a familiar process for an unfamiliar problem.

'Anderson and co have found evidence of an entirely counterintuitive phenomenon in which skill levels play the opposite role, so that skillful players are more likely to make an error than their lower-ranked counterparts.'

I would to see more how the study was done because i highly doubt this could be possible. It can be the case that in high ranked games the oponent will put you in harder spots, and errors will comes. But by definition, better players take better decisions Ev average.

An interesting study will be a list of leaks the population usually have.

For exemple in pkker before solvers, people with a very strong hand on flop, lets say straight draw+ flush draw, when the board turn bad for them, they are way more likely to call a bet on river, because they hand was so strong on flop. Solver will show is a bad call because you block possible bluffs opponent could have (flush and straight draws) so his value range is bigger.

I don't think poker is the best comparison. It has hidden information so psychology is a much bigger factor, which machines are not as good at understanding.

Regardless, this doesn't seem that surprising to me. Highly skilled players have developed a set of tools, techniques, and analysis that they are very familiar at applying. If a game enters a particularly strange but still complex position, one that would never really arise under usual high-level play, it is plausible to imagine unexpected errors may occur from time to time, as the standard high-level heuristics may not work exactly the same. Especially if the higher-level player is not being very careful.

The lower-level player, not having those heuristics, must look at the situation, like they do all positions, with a fresher perspective and far less expectations. They won't make the mistake that can come from misapplying the heuristics and tools of higher-level play, because they don't even have those heuristics.

Of course, these are specific, uncommon situations. In general, the advanced player will almost always make much better moves than the less-skilled player.

>But by definition, better players take better decisions Ev average.

Note that this is only true when expected value lines up exactly with "increases probability of winning". Lots of value calculations don't, and so better play can look strange in some situations.

Back when I was a middling chess player in High School, I'd pretty much always play King's Gambit as white when it was offered, especially against stronger players. I knew that I didn't study openings as much as other players, so I wound up in a place where I at least had as good opening book knowledge. Plus the opening is high variance, so it gave better opponents more chances to screw up and let me in the game. That said, the consensus high-level opinion on King's Gambit is that white does not get sufficient compensation for the pawn.

Or perhaps and even better example is AlphaGo Zero. When this engine is winning a Go game as it gets into endgame, it will play lines that confuse the professionals as locally sub-optimal. The pros later figured out that these moves would give up points in exchange for removing lines of play that are threatening and uncertain: instead of winning by 2.5 points with a small opportunity for the opponent to come back, AlphaGo Zero would win by 1.5 or 0.5 points without yielding any counter-play.

Make a lot of sense, specially vs humans, if ev of two lines are similar, better take the one opponent is likely to have less studied.

In the case of alphago, all depends how ev is calculed, % winning or points.

They were looking at 3 0 games (3 minutes for the entire game with 0 seconds added per move) with a total of 6 or fewer pieces, including kings. So stuff like King + Rook + Bishop vs King + Rook + Knight. There are a countless number of confounding variables in this study that do not seem have to been given sufficient consideration and this phenomena is likely a product of that. These sort of 3 0 endgames are generally time scrambles. The vast majority of 6 man endings reached without one player having already resigned are going to be technical draws, so the players are generally playing to win on time - not on the board.

A critical factor that the study did not consider at all was your opponent's remaining time. The stronger player is generally going to be the player that has more time left, often much more time. If you have 20 seconds and your opponent has 2 seconds left the easiest way to wrap up the game is to just make a move, any move. This means the quality of move is going to go down in a vacuum. By contrast you won't find this situation as frequently from the weaker player since they'll rarely be meaningfully up on time.

Another related pattern is that in a dead drawn time scramble a weaker player is going to be more likely to make a blunder, such as hanging a piece. And not exploiting that move (such as by taking the piece) would itself be a blunder. But in a time scramble with just 2 pieces vs 2 pieces you're often just making semi-random moves as quickly as possible - meaning you will miss the hanging piece. Again for similar reasons this is not an issue you'll find as commonly from the perspective of the weaker player both because they'll less often be substantially up on time and because the stronger player is less likely to randomly blunder, meaning just making a random move is itself less likely to be a blunder.

I'm not sure if I'm reading you correctly, but there are definitely situations in which highly skilled players are badly matched against lower ranked players because the less skilled players don't play according to the expectations of a master. This game where a grandmaster lost 1347 Elo points came up on r/chess recently: https://www.youtube.com/watch?v=5aI3e8gccro&feature=youtu.be... and anecdotally I've seen this play out in fencing, as well.

The finding that better players make worse decisions is only under very specific conditions where the board complexity is high.

One theory could be that stronger players are going for a win which requires more risk while weaker players are happy to play a non-aggressive move looking for a draw.

Unfortunately the article didn't state if most blunders come from being ahead or behind in material.

Sometimes you need to go for the win because of external conditions (last round of a tournament, or your chess team is losing and the captain told you to press on). Last world championship qualifiers for example saw Kramnik tilting because he needed to go all in in order to still make a chance to qualify.

I think I've noticed watching blunders in pro-chess tournies, that as a scrub chess player I see moves like: Develop, get rook to open file, don't advance pawns protecting the king, knight on rim is dim.

The pro may calculate a huge line that requires breaking a simple rule, and in the end that may come back to bite them. Where me, all I see is "hey don't leave the knight on the edge like that."

Maybe unrelated, but actually trying to think yourself out of a problem, is often worse then simply reacting on instinct. If you are really good at something you sometimes think and analyses the problem, explore new strategies, etc, instead of doing something you know works.

It is the main idea from the book 'the inner game of tennis'.

Just a guess, but it reminds me of a comment I heard a semi-pro poker player make about playing with amateurs (waaay casual amateurs, mostly playing to pass the time while traveling for sports competitions).

He said that he was commonly surprised when he'd make a move that would be immediately seen by his peers as aggressive, or strong, and cause them to back off or fold, while with us, it just went right over our heads and we'd miss it entirely...

Seems like lower-skilled players could simply not see some of the complexity that confounds a higher skill player, and an even higher skill player would handle. A sort of local minima anomaly for skill.

Or, maybe like tho old saying: "Fools rush in where angels fear to tread".

This is why playing poker against unskilled players is easy but not so much fun — you just consistently have to show down value because bluffs and bluff-inducing plays just aren’t understood by the lower-skilled opponents.

The biggest challenge for me when I play less-skilled players is to maintain a solid showdown game — it just gets boring compared to high-level play.

I don't think they correctly realized the meaning of these blunders, most games are skilled-vs-skilled or amateur-vs-amateur, and for every blunder a skilled person makes there was a trap another skilled player placed. The way I see it, skilled players are more experienced at leading their opponents into exploitable complex conditions, whereas amateurs tend towards simpler board positions that are easier to reason about, and it is not uncommon for blunders to be missed by both sides.

Anyway I'm not sure they could control for the skill of the players in both sets, because their dataset is probably missing amateur-vs-skilled data, which probably renders most of their skill-based correlations incomplete, as we don't know if they correlate to the skill of the player making the mistake or their opponent.

> by definition, better players take better decisions Ev average.

My experience has been that, in everything with a competitive aspect, there is a phenomenon where suboptimal choices become optimal because they are unexpected.

Chess, war, video games, sports, everything. As you point out, in the long run, the better player comes out ahead. That's what "being better" means. But the long run isn't always what counts. Sometimes two competitors are going to meet exactly once or the situation is going to be "winner takes all".

> pkker before solvers

Can u please explain what are "solvers"?! I used to be involved in online poker software but I am out of date by about 10 years.

You create a model with inputs (ranges, sizings), then you computer play against himself and the more time the more it is near of nash equilibrium,

Equilibrium it means both of the players cant deviate from the strategy without loosing.

It is 33% rock, 33%paper 33% scissors.

10 seconds for what kind of chess? For classical chess it has to be much higher; probably minutes. For bullet chess on LiChess we have less than 2 seconds per move including the network lag

Also important point to note is that most chess masters avoid reaching this complex position and that itself is an important skill. The best way to solve a problem is to avoid it from happening.

GMs like Garry Kasparov would deliberately go into complex positions to confuse their opponents.

re : intentionally creating complications : Yes indeed -- Petrosian was also [in]famous for doing this. It's also a great way to win on time in blitz / bullet.

Silman has spoken / written about the little-discussed but non-negligible role that chance / stochastic processes have in the outcome of chess games. In one of his books [I think the recent edition of 'Reassess You Chess'], he ballparks chance at something like 10% of the total factors influencing outcome.

Offtopic: is to me or the Technology Review writing quality has plunged for the worse? It is difficult to get through this article. What is the plot doing in the article without any labels?

I don't think I've ever seen a good article from there.

This seems to have been written by a third party?

One thing I haven't seen mentioned yet on this page is the huge role of overconfidence in chess. It serves as a devilish kind of levelling factor:

If I play someone much lower-rated than me, it's hard to care much about the game. I will play careless aggressive moves, make reckless sacrifices, thinking anything will be good enough.

They will be playing a much higher-rated person. Winning will mean a lot to them, they'll try hard, focus intently, make the most of resources etc.

Because of this, I've often had the impression that higher-rated people are easier to beat than lower!

Then when you get in a very easily won position, it's really hard to keep focus, you just think it's already won, are already celebrating in your mind. But in chess, one slip is often fatal, no matter how winning you are. Or there are stalemate tricks, and the win is thrown away. And the more you're winning, the more paths to victory to select from, and the less you care to think about it. You impatiently wonder why the other guy hasn't resigned already. Thus the saying "There's nothing harder to win than a won game".

When you are losing, you focus, use every resource... Survival mode kicks in.

I enjoyed the paper but one thing they didn't seem to consider was the human-difficulty of the move e.g. sometimes long horizontal moves are harder to see (sorry, no reference, just anecdotal based on experience), so one could see later works that fold that in.

Also, they used endgames with IIRC < 7 pieces, so again, some derivative work might use middlegame positions and use engine analysis as an oracle, similar to Regan's work (http://www.buffalo.edu/news/experts/ken-regan-faculty-expert...).

Ivanchuck (I think) said the hardest move to find is a knight retreat.


"According to my experience in chess, the most difficult in chess? To see moves with knight back." @5:34 --Ivanchuk

Setting aside the poor writing of this article, the reasoning seems to miss a key element.

The higher the skill, the higher are chances of blundering probably because: 1. Higher complexity 2. Very high likelihood of being punished for it. At lower elo, your opponent probably didn’t realize the blunder either.


> Quick decisions are more likely to lead to a blunder, but after about 10 seconds or so the likelihood of a blunder flattens out

> More difficult positions are more likely to lead to a blunder. And skill levels have a big impact in reducing the likelihood of a blunder. In general, better players make better decisions.

> The bottom line is that the difficulty of the decision is the most important factor in determining whether a player makes a mistake. In other words, examining the complexity of the board position is a much better predictor of whether a player is likely to blunder than his or her skill level or the amount of time left in the game.

I looked at the paper.. They used only positions analyzable by 6 piece tablebases, which was a surprise, i.e. only positions with a maximum of 6 pieces, e.g. each side has king and 2 pawns. The game starts with 32 pieces, so that means eliminating all the complex board positions I imagined a study of chess complexity would involve! So they're not so much studying chess proper, as endgames, and very minimal ones at that.

They have an new definition of blunder - "a player has committed an error if their move worsens the mini-max value from their perspective. That is, the player had a forced win before making their move but now they don’t; or the player had a forced draw before making their move but now they don’t. ...we will refer to such an instance as a blunder." .. "Since we are interested in studying errors, we exclude all instances in which the player to move is in a theoretically losing position — where the opponent has a direct path to checkmate — because there are no blunders in losing positions (the minimax value of the position is already as bad as possible for the player to move)."

"Our data comes from two large databases of recorded chess games. The first is a corpus of approximately 200 million games from the Free Internet Chess Server (FICS), where amateurs play each other on-line. The second is a corpus of approximately 1 million games played in international tournaments by the strongest players in the world. ... we focus on this large subset of the FICS data consisting exclusively of games with 3 minutes allocated to each side."

I've played 10,000+ games on FICS, so I have some idea how it works. Players have a standard, blitz and lightning rating, all separate. 15 0 and slower counts as 'standard', anything faster down to 3 0 is blitz, anything faster than 3 0 is lightning. [0] 3 0 is a different world to 14 0, so treating those ratings as the same is already misleading - I often played 5 14 (i.e. 5 minutes on your clock to start, plus 14 seconds added on after each move), a bit faster than 5 15, which is counted for rating purposes as the same as 15 0. (It assumes a game goes for 40 moves, so 5 minutes + 15 seconds x 40 moves = 15 minutes.) So 5 14 is the slowest blitz speed. If I played someone rated the same as me whose rating mainly came from 3 0 games, they would be much stronger than me at 5 14. (Usually, on FICS, the faster the time control, the higher the rating. And obviously the more time you have, the better moves you make.) The point being - someone's blitz rating didnt mean they got it playing at 3 0. It's not their 3 0 rating.

Also, FICS doesn't use Elo. (It uses the Glicko rating system[1]) The paper's authors seem unaware of that, or maybe I missed the part where they explained how they converted between FICS and Elo. As far as I know, it's not possible. There's not a linear relationship, or any simple conversion formula. From memory, lower rated FICS players have higher Elo, higher rated FICS players (over 2200) have lower Elo etc. But chess websites have a variety of rating systems, none of them compatible, commensurable or equivalent.

Sadly, I kind of lost interest there. I meant to study the paper more thoroughly. But already that rating confusion seems a huge 'blunder'.

[0] https://www.freechess.org/Help/HelpFiles/blitz.html https://www.freechess.org/Help/HelpFiles/standard.html


Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact