

How to Catch a Chess Cheater - dfan
http://www.uschess.org/content/view/12677/763/

======
darkmighty
This is one of those asymptotically unsolvable problems (like captcha) -- he
just created a new game. It's called beat the cheating detector. If you have
access to the detector, it becomes trivial even: no matter how advanced the
detector, using a chess engine to generate a set of moves and picking the best
ones which go undetected works.

It's one thing that bother me about many online games (like counter strike).
Unless they make a completely locked up computer, fighting cheats (server-
side) is a losing battle. The bright side is that when they become really
indistinguishable from good players, we can rest easy that the play isn't
being degraded at all (expect for the cheaters themselves) :)

~~~
danielbarla
> ... It's called beat the cheating detector. If you have access to the
> detector, it becomes trivial even: no matter how advanced the detector,
> using a chess engine to generate a set of moves and picking the best ones
> which go undetected works.

Except that you have to beat all future detectors as well, which is a lot
harder to get access to. E.g. this is how athletes sometimes get caught out
doping; they use methods which are undetectable today, but fail in a year or
two.

I don't really want to press this comparison too far though, not only are the
prevalent levels completely different, but doping is also a lot less clear cut
than cheating in chess / counter strike. It's pretty obvious when you're
cheating in the one, whereas you could argue that using as-yet not banned
substances is fair game. Mind you, we have had players type in recoil-
resetting scripts at competitions, and argue that it's just a configuration,
so perhaps it's not as clear cut as I make it sound.

~~~
A1kmm
As long as the detectors don't directly detect cheating, but instead
improvement at a rate that is too high (like in the article), then if you
limit your rate of improvement to a rate and distribution similar to human
improvement, it is unlikely that any future program will be able to detect
this.

Detecting performance enhancement solely by looking at performance is a losing
battle for the detecting side.

~~~
danielbarla
Sure, but forcing cheaters to rate-limit their improvements to within
reasonable limits would already be a huge win for the detecting side, so I'm
not sure I'd call it a losing battle. It'd severely limit the utility of
cheating, while at the same time forcing a long history of slow improvement
which can be scrutinised.

Taken by itself, I'd tend to agree with you. In practice though, I think this
can easily swing the balance in favour of the detecting side. Another angle to
analyse chess games would be to see the degree to which a player's moves
correlate with known chess engines. Over a long term (say, a few dozen
tournaments worth of games), this type of evidence would be particularly
damning; the types of chess engines which play relatively human-like, yet
still are top-tier simply don't exist. Given the difficulty in creating them,
and the relative rewards, I wonder if it will ever be practical to cheat.

------
wamatt
After that teaser intro, was hoping the article would've shed a statistical
beam on the 2006 cheating allegations against Kramnik.

------
dj-wonk
> Because the proper construction of statistical evidence against alleged
> cheaters requires such technical expertise, Regan believes that it’s
> necessary to establish a centralized authority responsible for the
> administration of anti-cheating protocol.

> Regan also believes that a centralized authority can best fix public
> confusion about what constitutes scientific versus unscientific procedure.
> It’s too easy for people with a poor methodology to spread rumors online.

Ok, this seems reasonable.

> Eventually he would like to oversee the conversion of his 35,000 lines of
> C++ code into a Windows-driven program or portable app. “I see other people
> using my methods but not necessarily using my program,” he says.

35,000 lines! After skimming the article, I'd hope it could be done much more
compactly in a high-level language, say, Julia, Clojure, or Python.

~~~
rmrfrmrf
Maybe it was a typo. I also noticed that the article says "...the number of
possible unique chess positions [is] roughly 1043," which is a typo. 1043
should be 10^43.

~~~
CountHackulus
Not to mention their error in claiming that NP problems were unsolvable.

------
A1kmm
To paraphrase the article, Ken Regan thinks that someone's chess skill
improving too fast is conclusive proof that they are cheating.

I'm not sure this is really a true analogy to the examples of catching users
of performance enhancing drugs; it would be more like saying 'your time on the
track has improved by too much for us to believe, so you must have cheated,
and therefore you are disqualified'.

Perhaps it would be better to use his software to target people for higher
scrutiny, rather than as categorical proof of cheating. Cheaters, however,
could just make sure that they always cheat, and ratchet up the search depth
at a rate below the maximum permissible improvement rate.

------
annnnd
Just curious: why don't they play in shielded rooms instead? I'm quite sure
they are not cheap, but they would be far more reliable.

This method will work only until cheaters up their game (pun intended).

~~~
a3_nm
Yeah, the article doesn't mention it but it is likely that you could run
Stockfish on a smartphone or so to get good moves without any form of
connectivity. (The main problem, I suppose, would be that you would have to
feed it the other player's moves somehow, whereas with connected devices
someone else can do it for you.)

------
vertis
The clearest give away was insisting on wearing Google Glass ^_^

