
Spatial Audio in VR - bharatkhatri14
https://developers.google.com/vr/concepts/spatial-audio
======
pfista
This is actually quite amazing and one of the fields I'm most interested in.

Spatial audio / 3d audio is much more immersive and less gimmicky than 3D
video, but I feel like people don't really care too much because they were so
underwhelmed with 3D video. It's one of those things with high fidelity audio
you can't tell a difference between what is real and what is fake.

Edgar Choueiri, a Princeton professor, was the first I know of that tried to
commercialize his algorithm for spatial audio. I believe he got a deal with
Jawbone to put in Jambox as a feature called "live audio mode". He did try to
sell some hardware for audiophiles to build out home systems, though I'm
pretty sure they were insanely expensive (10-100k range).
[https://www.princeton.edu/3D3A/index.html](https://www.princeton.edu/3D3A/index.html)

There's also a great podcast that tells stories using only spatial audio.
They're really immersive and sometimes I can't tell if the audio is happening
in the podcast or outside my room. It's called The Truth:
[http://www.thetruthpodcast.com/](http://www.thetruthpodcast.com/)

Makes me happy to see tools like this are open to developers now. I expect to
see a lot more content featuring this technology now.

------
gasbag
If you want to hear a great example of spatial audio, check out The Quiet
American [0] and be sure to use earbuds. He records his environment using two
in-ear microphones to get all of the effects discussed in the article: sound
shaped by the external parts of the ear, timing differences between the left
and right ear, etc.

I remember listening to one recording where he is at the edge of a lake or
ocean, and listening while standing with my eyes closed I almost expected to
feel the waves around my ankles. It's incredibly immersive, and will work
wonders for VR.

[0]
[https://www.quietamerican.org/introduction.html](https://www.quietamerican.org/introduction.html)

------
Theodores
Back in the day 'spatial audio' was excellent for what it was in VRML, as
played through the SGI viewer. I was sad VRML never happened as I quite
enjoyed worlds where you could play different songs in different virtual rooms
able to mix albeit not to the beat between different tracks by moving around.
A fantastic music experience could be had.

This new take on 3D audio just makes me want to go back to 1998 and what was
achievable then in 3D on the web if you were willing to do a 20Mb download
over a modem to get the plugin required to 'see' VRML. The actual hurdle was
use case, instead of a 3D web we ended up with a 2D flat table web. If you got
good at VRML and used all of the features and built significant sized worlds
then you soon realised that you were on your own, few others in the world had
got that far, their models were more like Duplo Lego rather than Lego
Technical.

The alleged 'bar' back then was the download for the plugin, that allegedly
prevented VRML from catching on. The authorship tools were awesome - Maya like
- and yet there were few takers. CosmoWorlds was awesome, has to be said. The
lack of content creators was the problem, 2D Flash was the way people went,
nobody was thinking in 3D.

So when can we start doing cool 3D just on the web in a web browser that you
don't need specialist hardware for? With 3D spatial audio working too?

------
cromwellian
I've never really been impressed by HRTFs compared to true Binaural recordings
(e.g. virtual barbershop) There's something happening in the binaural
recordings for positioning that just isn't captured in HRTFs it seems.

------
mdf
Nice to see audio finally getting the attention it deserves! And seems like VR
is the driving factor. I wonder how this compares to NVIDIA VRWorks Audio[1]?

When you think about it, 3D audio has many similarities to 3D video: rays of
sound(/light) traveling through open space and attenuating in the process,
bouncing off surfaces whose material filters different frequencies. From this
point of view, in hindsight, the accelerated sound cards from ’00s seem a bit
of a misstep – isn’t this something that’s better computed on a GPU?

[1] [http://www.roadtovr.com/nvidias-vrworks-audio-brings-
physica...](http://www.roadtovr.com/nvidias-vrworks-audio-brings-physically-
based-3d-gpu-accelerated-sound/)

------
AceJohnny2
So, are we going to revive A3D for mobile/VR? Have any of the patents expired?

~~~
Terr_
If Creative ever used A3D to improve their own product -- after lawsuiting
Aureal into oblivion and buying the company -- I never noticed.

For readers who aren't familiar with the story:

[http://www.roadtovr.com/sightline-chair-goes-binaural-
develo...](http://www.roadtovr.com/sightline-chair-goes-binaural-developer-
shares-thoughts-3dception-integration/2/)

[https://toni.org/a3d/](https://toni.org/a3d/)

------
JetSpiegel
What's the difference between this and OpenAL? It has 3D audio for ages, but
not many games really use it.

~~~
stevenwoo
Pretty much every third party sound library for games implements 3D audio
support (commercial ones probably for the last 10 years or more) but in order
to use more than just the spatial sound source location/user facing vector
requires inputting the material types/locations/sizes/audio properties for
everything in the game world (to calculate audio propagation loss/occlusion)
and using a certain percentage of cycles per frame to this - not everyone
chooses to dedicate this much work to audio or has the extra CPU cycles to do
this after graphics/physics. Also they have to be able to support simulating
the head transfer function via speaker (many possible configurations) or
headphone so that's doubled or tripled the amount of QA for audio sfx instead
of relying on a VR headset for a standard setup for VR, in addition to
possibly adding some sort of customization for each individuals head transfer
function. If your games don't really use it, it's because the developer didn't
have the time/money/CPU budget for it because it probably wasn't a key
feature.

Not sure what is different about Google's offering in VR space versus
tradtional game audio engines here.

~~~
gens
> Also they have to be able to support simulating the head transfer function
> via speaker ..

Do you mean head _related_ transfer function ? As in
[https://en.wikipedia.org/wiki/Head-
related_transfer_function](https://en.wikipedia.org/wiki/Head-
related_transfer_function) ? HRTF would be practically impossible to do with
speakers in a room.

~~~
stevenwoo
yes, HRTF, just typing hastily. If one enables HRTF support for instance, as
this quick googleing finds support in an eight year old query on FMOD,
[http://www.fmod.org/questions/question/forum-29423/](http://www.fmod.org/questions/question/forum-29423/)
one needs to support/not support/test all the cases, with the HRTF
enabled/disabled, user deciding to use headphones/speakers with whatever
configuration they have. Maybe one will decide it sounds better with HRTF on
with speakers or decide that's not a good idea and not allow it or not even
support HRTF. It's up to whomever made the game to determine how to support
whatever the user decides to use. So in the general case of games not just VR,
one might have a console version and a PC version and may need to support
headphones/2 speakers/multiple speaker set ups and enabling/disabling features
on demand per configuration. So back to the original query this requires
signficantly more QA/testing than supporting just HRTF with VR headphones.

