
Real-Time Music Visualization on the iPhone GPU - birslip
https://deezer.io/real-time-music-visualization-on-the-iphone-gpu-579d631272d3
======
rocky1138
At first I was thinking "this isn't a big deal, Atari Jaguar CD did this back
in 1995 on way less hardware," but then​ I realised it was a hackathon project
and now I'm really impressed at how far they got in that short frame of time!

------
mpalmer
This looks interesting but it's pretty hard to judge the quality of real time
audio visualization with no audio.

~~~
bottled_poe
I wrote a very similar prototype a while ago. Here's a video -
[https://www.youtube.com/watch?v=oYZEDSzX7MU&feature=youtu.be](https://www.youtube.com/watch?v=oYZEDSzX7MU&feature=youtu.be)

I didn't realise at the time, but the X axis is not in logarithmic scale, so
most of the audible frequencies are bunched on the left hand side.

~~~
stiGGG
Nice, do you share the code?

------
thedjinn
Pretty boring if it's just FFT based. I'd love to see music visualizations
make use of more sophisticated musical feature detection, like polyphonic note
recognition or tempo/rhythm analysis.

~~~
acoye
Hey, I'm Adrian. I did the 'More colors' shader.

Given the nature of the project, a hackathon, sticking to the last computed
FFTs and a notion of time was all we need to play with nice shader effects.

I'm currently playing with tensorflow on my spare time. I'd like to use a
neural net to find interesting features of a track and inputting this to a
shader. That's what I would do if I had another go at it.

~~~
stillkicking
Cool work, but note that a 1024 point FFT isn't really sufficient to visualize
music. At 44.1kHz, the lowest frequency you can resolve is 43Hz. Anything
below that will be folded into your DC component. Your bass response is mostly
going to be overall volume, rather than beat.

~~~
thedjinn
Most music is mixed not to have any useful spectral information below that
frequency anyway. And if you want beat then a better thing you can do is look
at the half-wave rectified difference between the sums of two FFT frames (with
optional lowpass). This results in an onset graph that is the basis for a lot
of real-time beat detection algorithms.

------
taylodl
I'd be interested in seeing the power load of these visualizations - are they
a battery drainer?

~~~
birslip
There is definitely an impact on the battery. The FFT part is well optimised
(thanks to the Accelerate framework). What is costly is mainly the shader...
and it really depends on the shader. A very simple shader is usually less than
5% of the GPU usage (according the Xcode Instruments, and on a recent iPhone).
However if you do a lot in your shader it's going to be much more costly.

------
pmlnr
I miss Winamp effects as well.

~~~
parkersweb
Yeah - we need a Geiss port :)
[http://www.geisswerks.com/geiss/](http://www.geisswerks.com/geiss/)

~~~
acoye
Is the code available somewhere? It doesn’t look like to be open-sourced.

~~~
FigBug
Milkdrop, by the same author has been open sourced.

