Edit: what I mean is that in this case the problem becomes answering "does this place on the board have a piece on it" not "which piece is on this place on the board." Once you know which places on the board have any pieces whatsoever you can simply infer between moves which pieces moved where.
I think the punchline is likely to be that you end up taking dot products of feature vectors generated by library code and writing fairly little chess-related code. Sorry to burst your bubble if you thought this was going to be very sexy. On the plus side, it will work, and if you're as good at programming as you appear to be you're almost certain to succeed on this.
P.S. I hope you like taking photos because odds are you're going to get the opportunity to take lots.
It may be easier to just track the game state from the start instead of trying to identify the position of the board at a random snapshot in time.
Cool project though, seems like a good intro to image processing.
"It may be easier to just track the game state from the start instead of trying to identify the position of the board at a random snapshot in time."
This is certainly true! But the original idea was to take a picture (probably from a mobile phone) and continue the game online. Taking a picture after each most is a lot of work :D
EDIT: Another feature ... in a properly set up board, the bottom right square is always white.
I did find a very recent paper on a chess playing robot which has some good starting points:
I bet you could get away with determining the color of a corner square and assuming the rest.
Some of the square edges actually belongs to the surroundings squares because the cut is not 100% accurate and the chess piece are not always position at the center of the square but I could get around that.
The way this algorithm works is simple. I compare the RGB values for each pixel of a square to the black color and white color. Black is Red=0, Green=0, Blue=0 and white is Red=255, Green=255 and Blue=255. I add up the 3 RGB values for a pixel, if it is closer to 0 and 255*3, I consider the pixel is mostly black. I keep a counter of black pixels and another for white pixels.
Currently, the algo is just comparing the two counters. If I found most of the pixel were black, I consider it is a black square.
However, I could improve it as following:
- > 90% of black pixels: empty black square or black square with black piece
- > 90% of white pixels: empty white square or white square with white piece
- other white: square with a piece of the opposite color.
For the 3rd case, I might need to implement your idea.
If you're ok with helping the computer out a bit, you could put a very small matrix bar code (think QR Code) on top of each piece. Since there are 6 chess pieces and 2 colors, you could have a code for each piece in just a 2x2 matrix (4 dots / bits).
Otherwise, it seems like it may be very difficult, as I'm not even sure of what some of the black on black pieces are myself (bottom right).
If you want only to modify the chessmen and not the board then I'm sure some transponder could be fitted and the table could have 3 receivers removably fixed to it that could triangulate their positions.
If it has to be camera based but you're going to modify them men could you just give them all different coloured hats.
It make me think it would be easier to recognize a domino game than a chessboard!
The other option, although a magnitude harder, would be to take two (or maybe three) images from a lower angle on two different sides and use that to determine the silhouette of the pieces.
Take two photos. One at the starting position and one at the current position you'd like to save. You don't need to take them in order (take the current position, reset the board, and take the other photo). This way you know what the pieces are from the starting position, then use an algo to match based on that.
Also some good hints for most standard sets while checking your current position...both kings are always on the board and they are usually the tallest. there is never more than one queen per side, and its usually a close height to the king, but has a circular crown. Rooks are circular on top, bishops come to a point, if there are more than two pawns per side that gives them away. Knights could be singled out by eliminating the rest.
It's easy to find SIFT implementations, but you'll have to do some work to construct a training set of images for each type of chess piece. An elegant solution will probably detect the type of piece independent of its color, or the color of the square.
This might be ridiculously easy, or not work at all.