Or better yet: figure out the average dot, replicate it all over the image, and run its Fourier transform. I think that subtracting it from the original image Fourier transform and then inverse transforming the result should also work.
Subtracting the Fourier transform of a dot pattern from the Fourier transform of a photo is equivalent to (but slower than) subtracting a dot pattern from the photo. Subtracting an average-sized dot from a random dot would leave behind a ring, and subtracting a cone from a random dot would instead leave behind odd sloped shapes. It might be a good first step though.
Also, halftone dots aren’t necessarily simple shapes. They are often designed to change shape as they go from 0 to 100% in order to have a shape that holds ink better (for some reason I don’t understand, a circle isn’t necessarily the best shape for holding its edge when made out of wet ink), and also for style (think large halftones where the dots are visibly long streaks all oriented in a particular direction which isn’t necessarily the same angle as the halftone grid). In fact, postscript allows you to prepend code that defines your own custom halftone function f(u,v) -> 0/1 (where u,v is the coordinate within the halftone cell) before printing your job, kinda like a weird little pixel shader from the 90’s.
