Edit: also in our case we used a region of interest that moved with the ball position to cut down on the computation.
Edit: also something to consider is using a kalman filter to help predict and smooth out noisy predictions. We did that and it helped considerably.
More fun pool table shenanigans: http://www.obscuradigital.com/products/detail/cuelight/
IIRC Obscura were selling that for $100k+. If the current business model doesn't pan out then I guess GoCardless could always pivot into that market instead :-)
Great application - looking forward to the future posts :)
As for the speed issue, your can often get massive gains by restricting the colour classification and blob detection to areas of the image which have changed since the previous frame.
So it learns your strengths/weaknesses then when you're playing uses this information to evaluate each possible shot based on the predicted final position of the ball and how useful this would be to the opponent and your chances of success...
One possible difficulty would be the learning phase - it would have to be able to know your intentions when you take a shot, which would be difficult to guess automatically. I believe it might be billiards where players have to say which ball they're going for - you might have to build that in...
Actually, figuring out how to do this without a webcam sounds like a pretty snazzy hurricane project.
There are very stable linux drivers out there for it and you can get up to 120fps. For a past project, my team was able to get up past 60FPS using this camera.
You could "cheat" and track the game's progress, but that wouldn't work for pawn promotions.
For some reason I'm thinking of those guys who did this with a roulette wheel.. but I'm not sure how that will be handy for pool.