
History and Derivation of the Fast Fourier Transform - mjfl
http://michaeljflynn.net/2017/03/20/history-and-derivation-of-the-fast-fourier-transform/
======
hughw
In computational seismology we've long had our own creation story for the FFT:

"The first machine computation with this algorithm known to the author was
done by Vern Herbert, who used it extensively in the interpretation of
reflection seismic data. He programmed it on an IBM 1401 computer at Chevron
Standard Ltd., Calgary, Canada in 1962. Herbert never published the method. It
was rediscovered and widely publicized by Cooley and Tukey in 1965." [1]

[1] Claerbout, J., 1985, Fundamentals of Geophysical Data Processing, p. 12.

~~~
hughw
The Claerbout book also gives a 27-line Fortran program to perform an FFT.

------
btilly
I remember a math professor retiring in the late 1980s, who was giving away
all of his books. Looking through them I saw one from the early 1960s that was
on numerical methods for certain classes of problems.

Several chapters started off by describing a problem, explaining that ideally
you'd do a Fourier transform, and then coming up with a special case hack that
worked well enough and performed well enough to actually be done.

The FFT made all of this knowledge obsolete.

~~~
jjoonathan
Especially in math, a distinction is often drawn between

    
    
        FT   f(real) -> F(real)
        DFS  f(real) -> F(int)
        DTFT f(int)  -> F(real)
        DFT  f(int)  -> F(int)
    

Only one of these, the DFT, is easily handled by a computer, but they all have
an important role in both theory and practice. For example, why does JPEG use
a Discrete Cosine Transform rather than a Discrete Fourier Transform? What are
the pitfalls of approximating a continuous domain with discrete samples? Why
are engineers obsessed with window functions? A continuous domain will lead
you to the right answer. You can't easily understand the DFT (or FFT) without
the FT and DFT. DFS has its own uses.

To many (applied) mathematicians, the fact that a computer uses the FFT
algorithm to compute a DFT is largely immaterial and not particularly worth
mentioning, because they are more interested in the questions in the preceding
paragraph than questions like "how many points can I take on X hardware while
maintaining Y performance?"

------
thomasahle
Also notice the awesome "Sparse FFT"
[https://groups.csail.mit.edu/netmit/sFFT/](https://groups.csail.mit.edu/netmit/sFFT/)
which takes time just about `k log (n/k)` to recover the `k` largest
coefficients. So often a complete Fourier transform is taken only to
immediately discard nearly all of the computed values. (Compression for
example.)

------
rrherr
Additional resources I'm finding helpful:

[https://betterexplained.com/articles/an-interactive-guide-
to...](https://betterexplained.com/articles/an-interactive-guide-to-the-
fourier-transform/)

[http://jackschaedler.github.io/circles-sines-
signals/](http://jackschaedler.github.io/circles-sines-signals/)

[https://books.google.com/books/about/Who_is_Fourier.html?id=...](https://books.google.com/books/about/Who_is_Fourier.html?id=89UpAQAAMAAJ)

------
hughw
I recommend Fast Algorithms for Digital Signal Processing, by Blahut, 1985 for
a mind-expanding survey of all the work in this area up to that time. Winograd
FFT is even lower complexity than C-T. And the book spends a lot of time on
fast DFTs over integer fields of size a Mersenne or Fermat prime. It's tragic
how many transistors we've wasted doing floating point math on what is
essentially an integer problem.

~~~
dbaupp
My impression is Winograd FFT is still O(N log N), i.e. exactly the same
complexity as Cooley-Tukey. That said, they work best on different N, which is
possibly what you're referring to?

~~~
hughw
A nice table from the book at [1]. For block length 1024, CT radix 2 requires
~20k multiplies and ~30k adds. Block length 1008 Winograd requires ~4k
multiplies and ~35k adds.

[1]
[https://books.google.com/books?id=8sbS2Bi4BmIC&pg=PA396&lpg=...](https://books.google.com/books?id=8sbS2Bi4BmIC&pg=PA396&lpg=PA396&dq=blahut+winograd+fft&source=bl&ots=fpxNovGZgd&sig=lg7WxOLB2rfVMBAuagSKzfQPxVM&hl=en&sa=X&ved=0ahUKEwjFpKT5rObSAhWBKCYKHTRSAXgQ6AEIJDAC#v=onepage&q=blahut%20winograd%20fft&f=false)

~~~
dbaupp
I see. That sounds like a smaller constant factor, rather than a lower
complexity.

------
medymed
Is is just me being a n00b or are there a few errors in the equations? Like
the exponential definitions of sin and cos, a missing negative sign in proof
of m≠n, maybe more later not sure. Somewhat perplexing for someone learning
the stuff BUT then again, I'm in medicine; so it might actually just be me.

Relevant for other learners is the digital signal processing lectures on
Coursera, highly recommended with python notebooks and quizzes.
[https://www.coursera.org/learn/dsp](https://www.coursera.org/learn/dsp)

~~~
mjfl
You are correct on the sine and cosine thing, the minus is switched. Will fix,
thanks.

------
jgalt212
dumbish question: if cosine is simply sine but offset by pi/2 radians, why are
we approximation these functions with an infinite sum of sines and cosines and
not doing it just with sines?

~~~
wayn3
Well, you may not like this answer, but:

If cosine is just sine rotated by 90°, then we really just use sines. Except
we write cosine instead of a sine with a 90° rotation because it looks less
confusing :)

The real goal is to arrive at the exponential representation, where the sines
and cosines are absorbed into the exponential functions, which is later needed
to write down the fourier integrals.

Your question boils down to preference. The two things you name are, in fact,
equal. Math people just choose to write cos(x) instead of sin(x+pi/2)

------
peter303
A Chevron geophysicist coded it before Tukey, but did not publish it

~~~
mjfl
I am genuinely interested in the source for that if you have it.

~~~
my_first_acct
Here [1] is a mathforum.org thread discussing the unpublished predecessors of
the FFT. Vern Herbert of Chevron is mentioned, but so are several others. The
mathforum thread has a link to a pdf that supposedly contains more
information, but that link is dead.

[1]
[http://mathforum.org/kb/message.jspa?messageID=1640228](http://mathforum.org/kb/message.jspa?messageID=1640228)

~~~
peter303
Correct. Since I am a geophysicist I met him a few times. He was pround of his
achievements despite limited recognition. A defect of the US R&D system which
promotes competitive secrecy over scientific gain.

------
derstander
Here's another history, of sorts, of the FFT that traces similar mathematical
techniques back to Gauss. Here's a snippet:

"In a recently published history of numerical analysis, H. H. Goldstine
attributes to Carl Friedrich Gauss, the eminent German mathematician, an
algorithm similar to the FFT for the computation of the coefficients of a
finite Fourier series. Gauss' treatise describing the algorithm was not
published in his lifetime; it appeared only in his collected works as an
unpublished manuscript. "

[1]
[http://www.cis.rit.edu/class/simg716/Gauss_History_FFT.pdf](http://www.cis.rit.edu/class/simg716/Gauss_History_FFT.pdf)

------
thearn4
Great article. The FFT and it's many applications (especially convolution-
based ones such as matched filtering) is one of my favorite algorithms, and
possibly one of the more important ones developed in the 20th century.

I'd put the Simplex method as a close 2nd.

------
pitaj
There are a couple misspellings of "Fourier." One example is the table about
half way through the article which spells it "Fouirer."

