I've looked a tiny bit into this sort of thing (using Metal compute shaders), and the problem is that, even with all of Metal's optimizations around CPU/GPU synchronization, the overhead seems to be too high for real-time audio DSP in practice.
The thing I tried experimentally implementing is basically a 128x128 matrix mixer with a little bit of extra processing. On a three-year-old MacBook Pro, the GPU barely had to lift a finger to crunch the data, but the round-trip latency was still high enough that it would struggle to keep up with anything less than a buffer size of 512 or so at 48kHz (which is on the high end for live mic processing). It would be fantastic for offline processing with larger buffers, though.
I haven't tried CUDA or OpenCL, so I don't know if the situation is the same there—but of course they have the problem of vendor support as well.
Our original thoughts on that was "nah, that'd be silly because GPUs are the wrong shape for audio workloads"
..however, having a few conversations with more knowledgable people has changed our mind and we're definitely going to give it a try on Metal and Vulkan to see what happens. Should be interesting.
I can understand the aversion to GPU (with "Graphics" being a primary point). Most ways of interfacing with the GPU use x,y,z, color, lighting wrappers around what is essentially a super powerful vpu. I also wonder how the traditional pipeline (e.g. vertex shading, fragment shading) can be repurposed for audio.
I don't have any experience with Metal or Vulcan but my intuition is that audio DSP is going to include a healthy dose of linear algebra and multi-variate calculus. That points towards some kind of fit with the GPU. Given that basic GPUs are available on practically every device (including phones) it seems like a fantastic fit. Even audio hardware developers would benefit since it would open up access to commodity priced chips (rather than custom asic/fpga/whatever).
And even for very simple audio loads, there's often unused capacity in compute cores (even when running games) which would could do the job "for free" without bothering the CPU.
Yeah, we made both of those assumptions. But it's apparently less true of the latest generation of compute engines. We probably won't bother with CUDA, but Metal is certainly a viable platform to try. It's one of those things where nobody really knows how it'll do until you try it.
Also, we do know a few people who want to use SOUL to write audio code which does need high parallelism. It's not a super-common use-case for audio, but it does exist.
I hope you'll consider writing up your results in a blog post or something! As I just mentioned elsewhere in this thread, I have experimentally found latency with Metal compute to be too high to be feasible, but I would dearly love to be proven wrong, if there's a trick I'm missing.
