If you've ever been in a recording studio with headphone monitoring / studio talkback, it's like that. Really intimate.
This project uses RTCDataChannel for audio, which is very neat, but it's a shame that once again audio on the web has to be hacked to perform well.
I get the problem with JackTrip (though JamTrip looks a good start).
Out of interest, what were the problems with Jamulus? I've found it easier than expected (once I'd got Jack set up and happy / understood). On Windows / MacOS it's actually easier as there are are clear, well-trodden install guides (and no Jack / low-latency kernels to explain).
Testing in Chrome on Mac OS 11.1, the lowest latency I can get is 23ms while in Firefox it's 14ms: https://www.jefftk.com/test/latency-demo-5
(As far as I know, there is no way to get the browser to tell you this latency, and you have to measure it.)
Obviously, even if I'm right, this has a target audience -- musicians, interviewers and other people who talk for a living -- for which it might be fantastic.
Are there not any good nonproprietary lossless low latency audio codecs? A brief look at Wikipedia shows a couple that are 5ms or lower, but either they are proprietary or they are low bitrate.
In my experiments with transmitting realtime audio signals using Opus, 10ms frame sizes are acceptable for 1-way synchronicity (e.g. if you want the user to perform an action and hear the result as simultaneous with the action) but it's definitely the upper bound. From what I remember my signal processing professor say, the threshold to try and hit is 7ms total latency for truly undetectable processing, but I can't find a reference for that, so take it with a grain of salt.
Robert H. Jack, Tony Stockman, and Andrew McPherson. 2016. Effect of latency on performer interaction and subjective quality assessment of a digital musical instrument. In Proceedings of the Audio Mostly 2016 (AM '16). Association for Computing Machinery, New York, NY, USA, 116–123. DOI: https://doi.org/10.1145/2986416.2986428
AOO is a C/C++ library for flexible on-the-fly multi-channel audio streaming and OSC messaging over local networks and the internet. The repo also includes Pure Data externals and Supercollider extensions.
WARNING: AOO is still alpha and there are breaking changes between the pre-releases, but I hope to publish a final release soon!
AFAIU the benefit in general is that they run in the audio thread so the data does not need to be handed over to a different thread for processing. In this case however, the data is handed over to the main thread anyway to be send over the data channel.
Has anyone measured the impact of using AudioWorklets this way?
If you're on mobile, or corporate net, you might not be able to get "punched through" by the other party for actual p2p connection and will need a TURN server.
I'm guessing that an uncompressed data channel with audio in it might send a lot of traffic through it.
I really don't get why everything needs to be in a web browser nowadays. WebRTC is simply the wrong choice for low-delay audio streaming. Audinate's DANTE protocol has been around for 14 years, is affordable, hardware-accelerated, and has practically no avoidable latency when sending audio over the network.
Why add 20+ms of latency just to use WebRTC?
If you could send UDP directly then WebRTC could presumably be avoided.
But that's exactly the problem. Usually, nobody knows what's actually happening in that 20000+ lines of code WebRTC stack.
The README says “Multi-machine network music performance over the Internet is achieved” but then later describes how to create multi-person video chat sessions. Comments mention music.
What does this have to do with music? Is it just that remote live music collaboration and multi-machine audio playback are use cases for low-latency codec implementations?
Because it seems like this is an audio codec implemented in the browser that comes with a video chat demo. Am I missing something?
The way to have rehearsals over the internet is usually to use JackTrip for the audio and (Zoom etc.) for the video (which can be out of sync; obviously conductors have to tap in an audio metronome channel since video won't be in sync.)