

Fourier Image Filtering - daviddotli
http://david.li/filtering

======
calhoun137
This is very cool and impressive, really good job!

Image filtering is a popular subject these days thanks partly to Instagram,
and this subject is on the boundary between art and science, which is nice for
a change of pace sometimes.

Anyway, to master image filtering in photoshop/GIMP for example requires
learning a (very) large number of words and concepts which seemingly have no
organizational structure. These words/concepts exist for historical reasons,
and not because they represent some sort of cohesive or unified approach to
filtering in general. The exact same problem exists for simulating effects
pedals for guitars in programs like Pro Tools, interestingly.

Only Fourier methods can possibly unify this type of filtering, for a number
of reasons which are actually obvious to anyone who is familiar with the
fundamental role played by these methods in analysis.

The problem seems to be that the people who actually use this stuff all the
time do not know Fourier methods and are not in a position to learn it; and
this has created a massive inertia against re-formulating the entire UI/UX for
image filtering based on the only possible unified approach.

It might be thought that Fourier methods are too hard for a graphic designer
to understand, but I believe that it is up to people like us who do understand
these things to redo the entire approach to filtering from square one, based
on Fourier methods. In fact, I believe such an approach will make this subject
much more accessible to everyone, and will be significantly easier for power
users as well.

~~~
sharpneli
"It might be thought that Fourier methods are too hard for a graphic designer
to understand"

I find this argument funny in general. Just look at sound engineers working
with the mixboard and their equalizers. Equalizer basically allows you to do
the same as what is done in that webpage. And yet the persons using it won't
necessarily even know what the term "Fourier transform" means.

An useful UI so artists can play with it is probably the only thing needed for
this.

~~~
calhoun137
Since you used the example of sound engineer's, I would like to expand on the
point I made in my previous comment about pro tools; and also clarify that I
believe sound engineer's and graphic designers are a smart bunch, and are more
than capable of handling it.

I actually recently went to a studio and worked with a professional sound
engineer. Most of the time, he would right click on a track, and then select
from a list of filters to apply to some segment. These filters have a GUI that
looks like an effects pedal with knobs etc... Having used many of these
programs myself, I can assure you virtually nothing is based on fourier
methods, even though this is all that these effects pedals are doing, often
with large amounts of redundancy.

A curious fact is that a large majority of the popular types of effects pedals
actually correspond to the most well studied _convolutions_ , which is
something I hope and plan to research more carefully and write up in a blog
post one day soon.

Effects pedals for guitars were originally circuits discovered basically by
chance; and it was only much later that these things were made digital. This
clearly demonstrates the problem I was talking about when I said there is a
"massive inertia against re-formulating the entire UI/UX for ... filtering
based on the only possible unified approach"

~~~
tfinniga
A convolution in image space is a component-wise multiplication in fourier
space. As the size of your convolution kernel increases, sometimes it is more
efficient to convert the kernel to fourier space, multiply pixel by pixel, and
then convert back.

This is what's going on with the presets at the bottom of the article.. for
example, if you took the gaussian blur filter and converted it to image space,
you'd have a gaussian blur convolution kernel.

~~~
smorrow
> [...] convolution [...] component-wise multiplication in fourier space [...]
> convolution kernel [...] gaussian blur filter [...]

Can I ask where people learn this stuff? Programming computers is pretty easy
to learn by yourself, but the more I look at technical subjects other than
programming computers, the less I can understand how it's possible to learn
that stuff other than someone showing you.

~~~
tripzilch
Many many years ago, a fellow demoscener pointed me to this:

Yehar's DSP Tutorial for the Braindead
[http://yehar.com/blog/?p=121](http://yehar.com/blog/?p=121)

It skips over some of the more fundamental math, but it serves as a great
introduction to general ideas of digital signal processing. You can then
proceed to look up the fiddly bits and deeper theory in more detail as you see
fit (start from Wikipedia, hit the external links--that's how I'd do it).

This particular tutorial speaks mainly about digital _audio_ signal
processing, which is the 1D case, but it generalizes to 2D image processing
just as easily, if you've got the imagination (although you need some tricks
if you don't want an IIR filter to look weird on an image).

------
angry_octet
This is really cool.

A nice explanation of using image FFTs is also given in the Image Magick docs:

[http://www.imagemagick.org/Usage/fourier/](http://www.imagemagick.org/Usage/fourier/)

------
claudius
As it wasn’t immediately obvious to me: You can click on parts of the
gain/frequency line to fix points and individually move points, causing the
gain/frequency function to be not just constant but an interpolation through
these points.

~~~
daviddotli
Thanks for pointing this out. I added some brief instructions to the curve
editor.

~~~
claudius
Excellent, thank you! :)

------
nullc
Has ugly cyclic artifacts, would probably look better with mirrored extension
(where the image is padded with flipped mirrored copies of itself). E.g.
lowpass and watch the sky bleed into the bottom.

Generally block transform techniques are not that interesting over a whole
image because the signal isn't stationary.

~~~
fensterbrett
I wonder whether our agreement upon the ugliness of Fourier artifacts (such as
ringing) stems from a learned distaste for low quality image and video
compression.

~~~
nullc
Forget preferences about the ringing, it's actually spilling light from one
side of the image to the other, as a result of failure to pad.

But yes, the ringing is ugly— and while there might be some learned elements
here, the images are not periodic, they're mixtures of regions with different
statistics, and it's inappropriate to spill one to the other.

------
thearn4
Partially related: Here is a Python implementation of Conway's Game Of Life,
using numpy.fft

[https://github.com/thearn/game-of-life](https://github.com/thearn/game-of-
life)

This advances the game state image through re-expression of the game rules as
a convolution filter.

------
gus_massa
It's interesting, but a little slow in my slow machine. Does it use FFT or the
"normal" FT?

~~~
daviddotli
It uses an FFT implemented on the GPU via WebGL.

~~~
ygra
Interesting result on IE:

[http://hypftier.de/temp/2014-09-01_182732.png](http://hypftier.de/temp/2014-09-01_182732.png)

The FT part seems to work, though.

------
thewarrior
I notice that the low frequency components represent the large scale features
of the image while the high frequency components seem to fill in the details.

Maybe we could use this to build a human powered custom compression algorithm
since they work by discarding components which don't make much of a
difference.

~~~
darkmighty
[http://en.wikipedia.org/wiki/Quantization_%28image_processin...](http://en.wikipedia.org/wiki/Quantization_%28image_processing%29#Quantization_matrices)

------
theoh
If you have 3D data instead of 2D rasters, you can not only apply the same
filtering techniques but also use the Fourier transform to do volume
rendering!
[https://graphics.stanford.edu/papers/fourier/Levoy_GI92_Volu...](https://graphics.stanford.edu/papers/fourier/Levoy_GI92_VolumeRendering.pdf)

------
quarterwave
Is 'frequency' 1d or 2d? Also, gain transfer function looks like a spline, why
not try an FIR?

It would cool to add a few images with periodicity. I expected the lake bed
image to show some quasi-periodic lines, but couldn't make that out in the
spectrum.

~~~
sharpneli
In images frequency is 2d. However in that page (and many other image
processing applications) we only care about effectively what is 1d frequency
(all the frequencies with the same euclidian distance from the center are
binned as one).

------
Fice
Darktable has similar wavelet-based filter:
[http://www.darktable.org/2011/11/darktable-and-
research/](http://www.darktable.org/2011/11/darktable-and-research/)

------
starmole
Somewhat small bug: The edges wrap around, which you certainly don't want for
image filters. Especially noticeable with the blur. You want to use either
clamping or mirroring at the edges.

~~~
jjoonathan
DCT to the rescue!

------
TophWells
The curve editor works very nicely. Did you write it yourself? I'm working on
something similar, and it's really interesting to see all the things you did
differently.

------
codehero
Is there a similar way to play with the phase component?

~~~
tboerstad
Came here to ask for the same thing. Considering phase is very important when
designing filters, and just looking at the magnitude can overly simplify what
happens.

Just look at what happens when you use only the magnitude or only the phase
information to reconstruct an image:
[http://www.imagemagick.org/Usage/fourier/](http://www.imagemagick.org/Usage/fourier/)

------
gagzilla
Strangely for me- neither Chrome nor Firefox seems to load this. Would be
helpful if we knew what's missing to make this work.

------
mrcactu5
why use Fourier analysis when it is not a repreating signal? I guess any
function can be approximated using Fourier series, but it spreads information
that is local and localalizes information that is spread out. I think a
wavelet analysis would be more appropriate -- or at least separate the
features and then use FFT.

Great job tho!

~~~
retroencabulato
How often are frequency-domain tools actually used on perfectly periodic
signals?

The mathematics assumes the sample is of a (bandlimited) periodic signal, but
so what? What is of interest here is the power spectrum of the sample.

------
bcheung
Wow, so much more accessible than trying to do it with Photoshop. Is there
code for this as a library anywhere?

------
Elizer0x0309
This is awesome! I'm expecting an audio version, soon-ish :)

