Show HN: Visual Sudoku solver in the browser

[taiters_]

https://sudoku.dotslashdan.com/

Hello!

I recently wanted to learn a bit about computer vision. Initially, I wanted to build something which could solve a jigsaw puzzle, but figured I should start with something (much) simpler, so I've built this instead.

This is a visual Sudoku solver which runs in the browser. It works by using OpenCV to identify and process the Sudoku grid, passing this to a simple ML model to identify the digits, and then solving the puzzle with a backtracking algorithm. The ML model was trained on the TMNIST data set using a model built with Keras, also a completely new area to me.

It's far from perfect, and doesn't like non perfectly lit or overly warped puzzles, but the main goal here was learning, which I did, a lot.

As this was primarily a learning project, I've tried to document my approach as much as possible, which can be found in this Python notebook:

https://github.com/Taiters/sudoku-solver/blob/main/notebooks...

I used Python while exploring OpenCV and training the models etc, and eventually ported this over to web (OpenCV.js + Tensorflow.js) to get something I could actually share with people.

Feel free to have a dig around the source or play around with the solver!

Github: https://github.com/Taiters/sudoku-solver

[giza182]

I think it would be super helpful to have an example puzzle there so one can quickly try it out without having to find a sudoku puzzle elsewhere

[taiters_ → giza182]

Yeah that's good idea, thanks for the suggestion. I even went out and bought a puzzle book while building this, so it's definitely not something most people have around.

[devilbunny → taiters_]

Just FWIW, if you need a supply of sudoku puzzles, the wonderful Simon G. Tatham's (PuTTY author, for Windows users) Puzzles page has a lot of algorithmically-generated, guaranteed-solvable puzzles, including sudoku (called "Solo" on his page).

I'm partial to "Patterns" myself, though it definitely needs a mouse or trackpad (preferably the former).

[rlexsu → giza182]

You can also just grab 1 sudoku from nytimes for testing (not paywalled) https://www.nytimes.com/puzzles/sudoku

[iamflimflam1]

I made a similar thing a while back - works really well.

https://www.atomic14.com/2020/07/25/browser-ar-sudoku.html

You can see it in action here: https://sudoku.cmgresearch.com/

(assuming it still works - haven't tried it for a while).

[taiters_ → iamflimflam1]

I tried the blur threshold approach you mention in your post, and it's giving much better results!

Great post btw, very detailed.

[taiters_ → iamflimflam1]

Nice! Yeah the puzzle detection works really well in your one. I'll need to have a read through your write up to get some pointers.

[hermannj314]

I undertook this exact project last year but stopped after writing a solver and playing with OpenCV to do edge detection. I also bought a Sudoku puzzle book for making a data set to work with.

So of course I am glad to see this and thank you for writing this up. Now I'm reinvigorated to throw some time at this project again.

As an aside, a great link I encountered during my project regarding building the fastest Sudoku Solver (conversation): https://codegolf.stackexchange.com/questions/190727/the-fast...

[Sanctor]

Here are a couple attempts trying to solve an NYT hard sudoku puzzle: https://ibb.co/gtJghBP https://ibb.co/9nvHT1P

1) The half transparent green font color is really hard to read over a white background.

2) It fails to recongize already filled in squares and tries to change them. The source is computer generated and the image is pretty clear. It seems like a best case scenario for digit recongnition, nevertheless it fails.

3) It constantly re-evaluates while I'm trying to hold the puzzle up to the camera with varying degrees of success. Too fast for me to get a screen grab when I see a successful attempt among all the failures.

Cool project though! I was excited to try it :)

[taiters_ → Sanctor]

Thanks for the feedback!

I've recently made a change to try to improve results (Specifically the thresholding step when it's processing the image).

For me it's now finding the grid a bit more consistently, though your mileage may vary.

I've noticed the digit recognition still struggles a bit when looking at a screen though. Perhaps I need to get a bit more data for training the model.

[NKosmatos]

Hey, thanks for sharing this! I was thinking of creating a visual minesweeper solver as a web app and for sure I’m going to check your notebook ;-)

ps1. As commented, I don’t have a sudoku available to try this. ps2. Yes, it’s cheating but we’re doing it for learning.

[httpsterio → NKosmatos]

off-topic but I thought you might find this interesting or helpful :)

generally, when adding post scriptums, p.s. notes, it's common to add a p for each following note. So, p.s., p.p.s. etc :)

[zeograd]

I love the idea but couldn't get it to work.

When I click the snapshot button from the /viewer URI, I end up on /image and can only download the photo I took or go back.

What is the intended workflow?

[taiters_ → zeograd]

I've uploaded a quick recording here (Seems Youtube forced it to be a "Short"): https://www.youtube.com/shorts/rdZ25pyqqmA

So ideally it should "solve" a sudoku puzzle when there's one in the frame. Then you can use the snapshot button (taking you to /image) to save and download the solution.

Though if it's not solving it (Which is definitely not uncommon in less than perfect lighting etc), I can see the workflow being unclear.

[zeograd → taiters_]

Great, thanks!

I was probably not stable enough showing it to the webcam and had more luck with the sudoku lying on the table and moving the webcam.

The lightning was also quite important in getting the grid recognized.

Nice job :)

[piyushtechsavy]

I would be good if you can also allow upload of an image with camera.