Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Audiocube – A 3D DAW for Spatial Audio (audiocube.app)
253 points by noahfk 9 months ago | hide | past | favorite | 107 comments
I’ve recently released my solo project Audiocube

I wanted to make a 3D DAW, where spatial audio, physics, and virtual acoustics are all directly integrated into the engine.

This makes it easy to create music in 3D, and experiment with new techniques which aren’t possible in traditional DAWs and plugins.

I’d love to get any feedback on this software (Mac/Windows) to make it better.

You can download it for free through the website.

Thanks, Noah



I am so glad you built this, I just bought a license.

I sketched out an similar app, but never had super pressing need. I can think of many many uses for this from modeling performance spaces, minimizing resonances in industrial settings, crime scene reconstruction, art installations, speaker placement for large concerts and many more.

What research or similar tools did you look to for inspiration?

Some things that come to mind

Efficient Interactive Sound Propagation in Dynamic Environments https://cdr.lib.unc.edu/concern/dissertations/5425kb409

Precomputed Wave Simulation for Real-Time Sound Propagation of Dynamic Sources in Complex Scenes http://gamma.cs.unc.edu/PrecompWaveSim/

Immersed boundary methods in wave-based virtual acoustics https://www.pure.ed.ac.uk/ws/portalfiles/portal/257303782/Bi...


Thanks so much for the support!

All of these uses are possible - my dream is for it to become much more advanced and powerful, almost like Blender but for audio and acoustics. It's still in early days and I'm a solo dev for now...

I'll look at those links. For me it was blender, and spatial audio plugins like the DearVR stuff, and game engines!


I assume this can do sound rendering, like simulating a conversation on a subway platform while a subway passes by?

Or singing while walking through a tunnel?

Since it has capabilities that would be hard to replicate, rather than show the tool on the landing page, I would show the output. Remove the clutter and force people to listen to what the tool can produce.

The tool as it is now, is being marketed towards yourself, people that wanted to build that tool and know what it is. But everyone will know what it can do after listening to sample output.


Yes it can do all of those things.

You are right about the audio demos, I'll update the frontpage with this soon. Thanks!


This is great (in theory). During lockdown I got an ambisonic mic (Rode NF-SFW1) and used it to create Dolby Atmos experiences. The workflow - including sending it to Dolby's tool every time - was such a pain. Adding additional 3d elements was especially annoying and limiting.

Unfortunately that's no longer my hobby so can't test this for you but definitely scratches an itch for past me. Nice


That mic looks fun to play with. On their site there’s a link to a bunch of recordings that can be downloaded:

https://library.soundfield.com/


Thanks!

Yeah, frustration with the existing solutions was one of the main reasons I started this project.

I was working in creating immersive audio, and I just found that none of the plugins allowed the level of placement that I wanted. It was all fiddly, and you couldn't easily move the listening position around, while also moving sound sources.

I came from an audio engineering background, not programming, so it took me a while to even learn how to make software! But now I made the tool I always needed.


Very cool project! And well done on doing it without being a software eng!

How long have you been working on it out of interest?


Lots of cool software was created by domain experts who taught themselves how to program. And quality is more than often is excellent.


Thanks! Yeah it was a steep learning curve at the start but I’m very confident now.

I started it in 2020 - here’s a blog post about the history https://www.audiocube.app/blog/audiocube-history

I’ve recently got it working on an iPad which was a frustrating grind but I think the concept is a lot of fun on touchscreen!


It's funny to see this now because I've been for a couple weeks looking into audio spacialization. After a lot of research and even trying to write my own spatializer plugin, I found that Game Engines have probably the most complete toolset to do this task. (Specifically I'm using Godot with https://valvesoftware.github.io/steam-audio/).

Steam audio is pretty awesome in that regards because it supports HRTF and all the physical based goodies like occlusion/reflection and sound propagation. So you can get really really immersive spatial audio.

The only downside with this solution is that you can't do offline rendering. So my question is:

can Audiocube do offline rendering? seems like it would be one killer feature for my use case.


It sounds like you’re in a similar field to me so it would be great to connect! Totally agree, yeah Audiocube is built in unity as game engines made the most sense for what I wanted to achieve. Steam audio is super cool and was a lot of use to look at for writing my own stuff.

Offline rendering is at the top of my research list currently. It’s not in the app yet. It’s going to take a lot of work to build given that the physics engine calculations are tied to frame rate etc, so there is a lot of work to do - but I’m sure it’s possible.

Thanks!


Very cool app. If you can crack multi-channel output formats (5.1, Atmos) I can see a lot of prosumers who would happily buy the product. Even the most basic tools for Dolby 5.1 are overpriced IMHO and Atmos encoder prices are either far beyond the reach of most DAW users or require use of Pro Tools.

One downside of selling into the pro audio market is piracy unfortunately. I learnt that the hard way and ended up having to use iLok.


Thanks!

Yeah multichannel is a priority. I don't think it's too far away - maybe a month or so in the lab.

And true, although I'm not a fan of iLok.


Really? I mean I haven't seen an iLok used in anger since the bad days of Prosumer ProTools.

Ableton, Reaper, Reason, and all associated VST and VSTi's from the incumbents like Arturia through to Korg and Roland, are all hardware-dongle free. Most use some sort of 30 day license server check-in or similar cloud platform intermittent auth.

N.B. I don't use Waves or UA software. Ever. Licensing models are akin to common assault.


This was over a decade ago. We were selling into the Pro Tools market, alongside Waves, UA and other tools more likely to be sold into studios and didn't see a lot of objections to the iLok.

I still remember walking into one of the highest profile million dollar mixing stages in LA, opening their Pro Tools plug-in menu and almost every single piece of software was pirated. Unfortunately even people who could pay for software (and had a strongly developed sense of IP rights for their own work product) seemed to have no issues ripping us off.


I can't disagree, although pricing models in the industry were a bit insane comparing 1:1 today with the 'don't pirate' site that deep discount a lot of premium plug-ins.

That said, there was a non-negligible amount of studio owners who had licenses but chose to use cracked versions of software stripped of DRM or otherwise bypassing the security checks in favour of maintaining stability. Lot of BSODs and IRQ issues with competing dongles with bad serial implementation.


(iLok has a software-only client that works similarly to the cloud license check-ins you mentioned, it's commonly used by VSTs.)


YouTuber Benn Jordan would probably get a kick out of this. He's a major audiophile and did a series of ambisonic ambience.


Thanks, I think so too by the looks of his channel, I'll get in touch!


I was just thinking about this!


Will there be a Linux version anytime soon!?


I don't have a Linux machine or much experience with it, but I can easily build to it. If you want to help test a Linux build email noah@audiocube.app


Great! Look for a private message.


[flagged]


Bitwig is all right though.


Easy decision to send you a few pounds for this. This is no small task to put together and looks really impressive. I can't wait to try it out :)

I have a history experimenting with 3D audio - about 15 years ago I build myself a pair of ambisonic microphones, but until only recently I think the software support for ambisonic capture and mixing has been seriously lacking. Back when I built the mics I started working on a plugin suite for the processing, but I could never get it quite right. Nowadays, there are more 3rd party options I can use, and I will spend some time with this again :)


Thanks so much - yeah it's been a bit of a grind, especially not coming from a coding background. But it's been enjoyable for the most part...

Yeah there is defiantly a lot of stuff missing or neglected in this field!


No expert at all here. But could I use this to say, model my room and understand how to treat it acoustically to remove reflections and stuff like that?


Environmental modelling is normally a little seperate to what you'd expect from a DAW. If you're unfamiliar, it's worth taking a look at EASE [1] (the semi grand-daddy of electro-acoustic simulation). Treble [2] are also doing some really interesting things. 10log [3] too for some coarser (but free) analysis.

[1]: https://www.afmg.eu/en/ease

[2]: https://www.treble.tech/

[3]: https://10log.io/


Nice! Thanks a lot!


Eventually, but I'm not sure it's quite there yet. It has reflection surfaces, but I'd still need to add some more features like absorption, and more advanced frequency analysis. Treating a room is pretty techy - but I'm sure it could give you some ideas.

I'd like to look more into the acoustic modeling capabilities later on - and give it some more advanced features


Really cool! I've been working on a side project that utilizes spatial audio and I've been pleasantly surprised by the quality I'm getting just using the WebAudioAPI HRTF spatialization. I'm sure this is leagues ahead but it was really nice to find that I didn't really need to do anything to get decent spatial audio other than set the panner node to HRTF mode.


Thanks! Yeah this is definitely a lot more complex and deep than the WebAudio API but that’s still a great tool for some situations.


IIUC, I'd be able to take individual closely miked recording on multiple different instruments and mix them into a soundspace, such that when I listen on stereo headphones, I'd be able to "locate" the sounds on a virtual stage?

(asking because I listen to a lot of live jam music in stereo and noticed that they use a stereo mix with a virtual image)


Yes exactly - this was something I wanted to achieve.

For example, you could have an entire orchestra with each instrument close miked - and then arranged in audiocube in a vritual concert hall.

The user could even move around the space for full placement control - or you could simply render it to a binauralized stereo file for more immersive listening on headphones.

I'd like to develop a mobile "listening" app, which isn't for making projects, but listening to them and placing you in the virtual environment with control


That's how normal panning works, yes (or using a stereo pair of ambient mics, same thing but baked into the recording). But that's left-right only, I assume that these Atmos tools can do it in 3D space.


Does that work if you only have a pair of stereo headphones?

I've had quad speakers before (with a "delay" unit for the rear speakers) but I think it would be hard to do 3D in stereo (I know about https://en.wikipedia.org/wiki/3D_audio_effect and HRTFs but I have never noticed any 3d-ness to the sound I'm listening to in games).


Yes, on standard stereo headphones, a binauralized sound can give the impression of sound coming from all around your head. It mimics the filtering acoustic effects applied by your head and ears


Try https://www.youtube.com/watch?v=IUDTlvagjJA


I hear stereo, but no 3d effect beyond distance from microphone attenuating volume.


You using Blender's viewport? (Great idea if so, nearly perfect recreation if no!)


Aha, yeah blender was a huge UI inspiration, it’s all handmade from scratch though!


Awesome work, you nailed it!


This looks great! How small of an audio buffer have you been able to get down to? Any plans for an API?

I've been developing a VR spatial sound and music app for a few years with the Unity game engine, bypassing the game engine's audio and instead remote controlling Ambisonic VSTs in REAPER. I can achieve low latency with that approach but it's a bit limited because all the tracks and routing need to be setup beforehand. There's probably a way to script it on REAPER but that sounds like an uphill battle. It would be a lot more natural to interface with an audio backend that is organized in terms of audio objects in space.

What I'd like is more flexibility to create and destroy objects on the fly. The VSTs I'm working with don't have any sort of occlusion either. That would be really nice to play with. Meta has released a baked audio raytracing solution for Quest, and that's fun for some situations but the latency is a bit too much for a satisfying virtual instrument.

Here's my project for context: https://musicality.computer/vr


Hey, I’ve got down to under 20ms audio buffer, but it depends on the complexity of the scene.

What you’re working on sounds really cool, I’ll have a look at it!

It sounds like Audiocube offers the kind of features that you need, although it doesn’t have realtime audio input (yet, I’m working on it and have it partially working).


Hmm 20ms is quite a lot. I'm targeting <3ms, ideally.


I realised I meant 20 samples, not ms!


oh that's MUCH better!! I was hoping you erred.


Looking at your project it looks like it would be great to integrate the two somehow - Audiocube would be awesome in vr but I have no vr dev experience


There's a library Valve made for spatial audio for games (inc. VR). I've played around with it a bit, it's incredible. I'm surprised more games haven't adopted it.

https://valvesoftware.github.io/steam-audio/


Oh yeah, I forgot about this. I think it was a bit less developed last I looked. Not sure if there was another reason I wasn't considering it. Thanks!

edit: ah, it's because it is a plugin for Unity's audio system which I've already determined adds more latency than I want.


Thanks! I joined your Discord... And looks like I "liked" your Audiocube post on FB about 4 years ago :-)


Venus Theory and Andrew Huang are two other YT channels that would probably love this. Venus Theory does a lot of sound design and cinematic things, and Andrew Huang just loves to experiment.


For sure! I've tried emailing Andrew but I imagine his inbox is pretty hectic - and I'm not sure the best way to approach it, like asking for a paid video or seeing if they'd do one for free


Very interesting! Did you write your own acoustic simulation engine for this?


Yes - while the app itself is built in unity, it doesn't use Unity's native audio simulation. I implemented a custom spatializer and acoustic engine that can simulate reflections, occlusion, and other stuff that wasn't native in unity


> As a standalone app, Audiocube offers tools, workflows, and processes that go beyond the capabilities of VST plugins.

But does it support VST/AU in order to load instruments rather than "samples"?


There is currently no VST/plugin support, although this is something I would like to include later on.

It's just too much development work at this stage - not only the VST integration itself, but building a full MIDI sequencer timeline, editor, etc to support instruments.

Other DAWs do this perfectly well so I'm focusing on the 3D virtual environment moreso than tradtional DAW functions. Audiocube isn't a tool to compete with the likes of Ableton and Logic, but something to run on the side.

Although, if I had a big enough team I would love to give it all of these features. I'm just doing it solo at the moment, so I need to have a narrower focus.


"Why You Should Not Write a DAW" (ADC 2024)

https://www.youtube.com/watch?v=GMlnh6_9aTc


Yeah this is an interesting talk. A full DAW is maybe over ambitious for one developer, but I don’t think vst plugins would be able to achieve what I wanted. For example vsts can easily spatialise individual sounds, but there is no way to then easily move the listening position through the scene, without a disgusting amount of automation - hence building a virtual 3d environment daw with more placement flexibility


... or start with an open source DAW that already has all the rest, and add the placement flexibility you want ...

But I'm speaking out of self-interest here, and anyway for now your output is limited to binaural stereo which is not really a viable target for production workflows at this point.


That's the thing: if you have VST or ReWire support, you don't need to implement any of that. That's now something someone can use a VST/ReWire bridge for, taking all that work off your plate. "If you need MIDI sequencing, or multi-pattern input, or randomized pattern generation, just connect your DAW of choice as input".

(And of course, "and if you then want to consume the result in another DAW, you can". Chaining up multiple DAWs because they're all good at different things is one of the more fun things to do ;)


Exactly. Some kind of audio bridge is one of the main priorities for me. I think this will really open up options, and almost make it like a virtual studio space/echo chamber for reamping stuff.

It's just a matter of working out the best way to enable multi-channel input, and connect to DAWs, I'm researching it and I've had some basic success but I think I need to go deeper into the programming grind to achieve it.


Looks interesting, will check it out. I am curious what makes audiocube different and better than Logic Pro’s binaural and spatial Dolby atmos mixing?


Thanks.

I used to use those plugins, but I found that there wasn't enough control.

Sure, you can automate the position of individual objects, but you cannot then easily move the listner's position in relation to your sources - which Audiocube can.

It gives you more control, flexibility, and realtime interaction when it comes to moving the listener around a moving scene. And you can control the distance of sound propogation, you can add in walls for reflections and occlusion etc - essentially building virtual acoustic environments.


I've had this idea a few times and happy to see someone do something similar. Will try this out, thanks for sharing it here.


Thanks, any feedback and ideas are welcome too!


I can see it being useful as a VST, actually- could be an interesting part of my workflow in Live.


I'm working out how I can make a vst type bridge to stream audio in and out of other DAWs, I'd love to use it in realtime with Live. I've had a lot of requests to put the binaural processing into a vst too, which I'll consider later down the line!


Nice work! Can you export for multichannel playback or is it binaural / stereo?


I'm working on multichannel playback, I've had some experimental success, and plan on adding it into the next update. it just needs more testing on a range of audio interfaces etc.

Currently it only exports/plays back in binaural stereo


Ah. I think I misunderstood the purpose of the app. When viewing the main page, I had the impression that the output would be targeting large Atmos multi-speaker installations. Or that it would export to a format that can play spatially on AirPods.

So right now, is it possible to make something in Audiocube and export it to native spatial AirPod sound ?


I am working on multichannel output but it’s a bit of a headache with all the formats and interfaces etc - I want a simple solution, so that needs a bit of research.

Currently I believe apple spatial uses some kind of 5.1 or 7.1 format audio files mixed with a binauralizer to distribute on headphones. I’m building a method to export in surround formats but it needs a bit more research. Currently it exports as a binaural wav that sounds very 3d, but I’ll look more into apple Spatial Audio as I haven’t researched it much


Outputting 5.1 as a DTS-CD (or something I can convert to a DTS-CD) would give me a reason to buy it.


I'm researching the possibility of other export formats that can be encoded with more spatial information - it's highly requested


Is there VST/AU support? I didn't see it mentioned.


Currently not, it would be too much work to implement all the other frameworks as a solo developer - but I'd love to have that functionality when I have more resources


Neat interface the grouped ones with matching neighbor contours


thanks!


not sure if it's the ad blocker or safari itself, but the download links don't appear to work in safari. I had to switch to chrome to download.


The free download requires account registration, which will discourage most people from trying it. Even third-party login might make it easier, but I didn't find that option.


Does it work with vst plugins?


Currently no, but I'd like to impliment it when I have more resources. See my explanation in reply earlier.


This seems intriguing but I'm genuinely confused.

It seems like "bakes in" spatial audio to binaural stereo?

But who is the market for that?

I love spatial audio on my AirPods but a big feature is that it moves with my head and can even be customized for my ears.

And I certainly don't want it applied when downsizing to a mono Bluetooth speaker.

It seems like you'd need to export your final product to surround/Atmos for the way people want to, and currently do, consume spatial audio? I assume the target here is Apple Music, short films, etc.?

I mean I think the concept of the 3D DAW is great. I just want to make sure there's a product-market fit here, so you can succeed. Or is there a market I'm overlooking?


Thanks for the intrigue, and yes these are all valid questions and things I'm trying to work out.

I think I need to narrow down the market a bit. Currently I'm aiming more at audio and sound design experimenters rather than the deep technical stuff - although that is a goal later.

I'd like to create a 'listener app' so people can have full immersion, control, and exploration of audio scenes.

The binaural process is more of a high powered, more flexible and controllable version of a binarual plugin, so this would be for making binaural mixes rather than slapping on everything.

I personally think the binauralizarion is one of the best sounding out there, it sounds great on headphones and has excellent sound localisation.

But you are completely right. I'm still working out product market fit, and I think it's why I'm finding a bit of friction reaching the right audience.

Thanks for your thoughts! Any feedback and input is very appreciated


In case it helps, I think producing binaural recordings, or using a custom listener app, would be appealing to "experimenters" if it were open source, but it doesn't strike me as something with appeal to consumers or professionals that would pay for it.

Since you're going for a paid approach, it does seem like you need to locate the audio professionals mixing for spatial audio on Apple Music and TV shows, to see if you can show them why your 3D editor results in a better surround/Atmos final result.

I will say that I've often wondered what it would be like to "walk through" a band or orchestra as they were playing, with the binaural audio adjusting accordingly, which would require a full 3D approach like you've created, not just angles. I once attended an art project of a choir singing in 360° in a circle around you, where you were invited to move around to change the "mix" of voices, and it was magical. I think that's what you must mean by "exploration of audio scenes" and it sounds really cool. On the other hand, it's hard for me to imagine how you'd turn that into a business. Especially since I assume that's already relatively easy to do with video game/VR engines, even if perhaps the binaural effect isn't quite as good?

Good luck! I think the concept is very cool, and I hope there's a way to monetize it as an independent product.


Just guessing, but even setting aside VR and whatever kind of stuff they are doing for theaters like imax, there does seem to be an interest in new kinds of immersive 3d audio/video. The Las Vegas sphere aimed to do this iirc, and the money behind it aimed to develop the IP and open similar venues in different places around the world. Supposedly media that’s under development actually uses a smaller copy of the target venue, so if you can do that with a software suite instead it’s probably a good thing.


I've even heard of them using this kind of tech in theatre performances, so there is definitely a bunch of use cases!


[deleted]


Not spatial audio, but reminded me of audioGL (2012, but a newer video posted in 2024): https://m.youtube.com/@AudioGL/videos

https://news.ycombinator.com/item?id=3579543


There was also a BeOS application with a 3D mixing interface: https://en.wikipedia.org/wiki/3dmiX


I believe the HRTF binauralization of object-based audio does qualify as spatial audio, but there are many different definitions floating around. AudioGL is cool though!


Is there an audio format that stores the 3D origins of each sound, so that you could theoretically play it through Airpods (or some other spatially-aware headphones) and hear the sound change as you tilt your head?


Dolby Atmos is literally that, I think there are several physical formats that can support it.

Ambisonics, especially higher-order ambisonics, also is capable of spatial panning, though it encodes the 3D soundfield rather than individual sound sources. Again, there are a few physical formats.


This is probably a very odd and roundabout way of doing something akin to what you are asking for, but my recently released VR game DodgeALL[0] has a form of modding support for (wav/mono) audio files which can be uploaded (via SideQuest) to the headset. These are then assigned to "obstacles" which are launched at the player i.e. you could move your head around and hear spatialized audio files as they fly around the arena. You don't even need special headphones, the ones on the Quest do a good enough job, and Meta's Spatial audio SDK does a wonderful job of spatializing audio. I was originally using SteamAudio[1] but switched mainly due to moving from PC-VR to Standalone.

Sorry if this sounds like some self-promotion - I specifically built this functionality[2] for my previous audio engineer so that he could quickly design and test sounds without the hassle of having to build/install an APK with every new audio file.

[0]: https://www.DodgeALL.com (only for Quest 2/3/3s/Pro headsets)

[1]: https://valvesoftware.github.io/steam-audio/

[2]: https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...


Another way to do this would be to make an app that uses positional information from the headphones and feeds the sound of your choice into a head related transfer function (HRTF) to give the illusion of spatial audio.


You could also just emulate what the music would sound like with a stereo system and support head tracking with that. dear reality does something like that with their dearVR digital audio workstation plugins.

And then I recall there was this one old Kickstarter project that made headphones with that idea. It had an IR transmitter for head tracking.

I think it might be quite difficult to track head movement with just earbuds and not have the tracking drift, though.


I'm working on a mobile app that lets you playback projects created in Audiocube and gives you full control over where your listening position.

It's definitely possible to have head tracking, I think it will just depend on what headphones and hardware offer it. It would be cool to give you that kind of VR listening experience but just on headphones


In addition to Dolby Atmos there is MPEG-H 3D Audio.

https://en.wikipedia.org/wiki/MPEG-H_3D_Audio

https://ieeexplore.ieee.org/document/7056445


Something like https://en.wikipedia.org/wiki/Dolby_Atmos and https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X


I sort of expected this sort of thing to be a nice use for VR, since it already needs to do the tracking. But VR generally didn’t seem to catch on, so…


I have a home theater which is 5.1 which is great for movies [1] and some music [2]. There are 'virtual surround' systems based on [3] of which Airpods are the most famous. Most of those don't work for me except in the sense that I learn 'sounds like I have a cold' means 'the sound is supposed to be above me'. The exception is my Hololens where if I use the web browser to play music and turn around it really sounds like the music is coming from behind me. I have yet to be impressed by spatial audio on my Meta Quest 3.

[1] Oddly the surround sound soundtracks that impress me the most are old VHS tapes encoded with Dolby Pro Logic

[2] let's see... Close to the Edge by Yes, Supernature by Goldfrapp, Maximum-Minimum by Kraftwerk

[3] https://en.wikipedia.org/wiki/Head-related_transfer_function


For those of us unfamiliar with the term DAW, I assume it’s Digital Audio Workstation.

At first glance I thought of: DEW

https://en.m.wikipedia.org/wiki/Directed-energy_weapon


Digital Audio Workstation: https://en.wikipedia.org/wiki/Digital_audio_workstation


It’s a digital audio workstation.


[flagged]


Looks as though they offer “Perpetual” for 99.00


I see "Sale Ending Soon: Lifetime Licences 71% off - only £29!" at the top of the pages. It looks really neat, but my interest was more curiosity. You need to login to download and then login again from the app (for the trial), and I ran into more friction than I had time for. Will bookmark in case I have a need for this later.


Thanks for the feedback, I'll make it easier to get in in the next update! Also you don't need to log in inside the app. Just click log in and then trial mode, you dont actually need to enter details. But yeah it's maybe a bit confusing




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: