Here is what we came up with:
Whilst professionals can deal with the delay (playing notes in the future) that comes from Jamulus with a suboptimal connection, Students can not. We misuse Jamulus' concept to create a workaround. We walk students through port-forwarding and students start the jamulus server via a script, that connects to our private jamulus directory server. Then the teacher connects to the students machine.
As a result, the student gets 0 delay and the teacher, with his experience has to carry deal with the delay and has to compensate for it. Huge pain to setup, but the quality of the lesson on the student's side does not suffer, even if the connection is bad.
On a funny side note, IRL I witnessed the exact opposite problem. A piano soloist professional was playing the Weichnachts Oratorium with the Kreuzchor choir I sang in. The soloist had to play the Harpsichord, which responds instantly (strings plucked under tension), as opposed to the Piano's hammmers, which have a "slight delay". The professional was so used to this "small delay", that he had to relearn and adjust a lot, because his professional experience with the piano produced too early timings with the conductor's actions. So the opposite of what teachers face with Jamulus.
I think when learning to e.g. play drums like I once did, another realization is that you need to start the motions well before the impact. Actually, this is what a lot of people that start warbling among with songs badly fail at - they try and follow along, but they need to start singing (or tapping) before the song they play along with does, which means they need to know the lyrics and what comes next already.
Maybe the Jamulus folks can combine forces with Sonobus folks, implementing any Jamulus features Sonobus lacks.
The Sonobus UI is nice and responsive, coming from JUCE, but it's not native. More features, like metronome, are nice to have, but most musicians won't use them anyway as the instruments are usually connected through DAWs which provide all the necessary instruments for routing and layering sources.
The main difference is in the architecture, Sonobus is p2p, so it's great users have the option to try different approaches and find what works best for them.
Also, p2p requires O(n^2) links whereas client-server has O(n) links, so if the number of peers increases, the total travel time difference will be more significant.
I assume that it's not time-prohibitive either for each device not to be directly corrected to each other device.
Most new people who try out Jamulus and join our servers would complain about the latency (especially professional musicians), to which I give the same advice: 1) Use a fiber-optic internet service. 2) Use a LAN cable. 3) Use wired headphone (no bluetooth/wireless headphone/AirPods). 4) Use an audio interface with ASIO device or use a Mac. Finally, and most important: 5) Just play 100 songs on Jamulus, and you’ll eventually get used to it.
Most friends are jamming with latency of around 20ms. I have found that for vocals and drums, maximum latency tolerance is around 30ms. For me I play the keyboard and most of the time I’m jamming over a 4G mobile network, I have increased my latency tolerance to about 100ms now.
It's pretty wild to think about. My dad used to point out to me that, if I was watching a live concert on the TV, the sound would reach me before it reached much of the live audience.
I found that most of the latency in networked audio applications, when jamming within 800 km within domestic internet, mostly comes from (a) a large network buffer (to prevent sound stutter when the network isn’t reliable, e.g. due to wi-fi interference) and (b) extra audio processing on top (echo cancellation, noise suppression), and not the limitation of light speed.
You're vastly exaggerating. FTL communication is not here yet.
The theoretical limit for half the planet is ~67ms, in practice it's north of 100ms for Jamulus (plus the additional delay of a user's sound reproduction system).
I believe the example above is more about sound speed than light speed.
I experienced concerts in stadiums, in the back rows, where the distance from the stage makes the sounds arrive noticeably later than the visuals.
The delay is really perceptible, and disturbing, to a point.
A TV audience would not suffer from it (assuming a live broadcast, with negligible broadcast latency, for the example to make sense).
Apt-X "low latency" is 33ms - the equivalent of being about 11m away from the sound source.
Standard Apt-X is 100ms, or equivalent to 33m distance in open air.
SBC codec is 200ms.
Then there's the buffering headphone bluetooth chipsets may do to avoid customers complaining about clicking/popping. If you're on a crowded subway train there can be a shitload of bluetooth traffic bouncing around, in which case having a healthy buffer is helpful.
I have an external "headphone out and 2 microphone in" USB interface now, and it's latency is still in the 5-15ms range, so I use onboard to write and headphones to master. At one point I bought a macbook air and a FireWire interface, but the screen was no good and the single FireWire interface tied my hands. I also have a few pci cards that I've been collecting parts for, like an m-audio 8x8 interface, there's a box that connects to the back of the soundcard with some 30-40 pin d-sub. I finally got all 5 or 6 parts and don't really have a computer to put it in anymore! I wanted that card for around 10 years by the time I found all of it...
I'm assuming anything midi-ish can just use quantization and only the player will notice, the final mix should sound ok.
Every musical instrumented (voice too) has a latency. The practiced musician is trained on the latency of themselves and their instrument and will initiate a musical note at the right time. That might even be a few seconds in advance.
Part of learning an instrument is learning its latency. Jamulus adds a bit on top of that. That requires some getting used to. A good tip is to listen to your sound over the internet. So wear insulating headphones that keep out your real-time sound.
edit: replace hands with whatever is needed to produce the sound: vocal chords, mouth, feet, arm, ...
Playing from Thailand in a Singapore server, add 27 milliseconds.
Playing from Thailand in a Hong Kong server, add 65 milliseconds.
Next event scheduled for Dec 4th 17:00 GMT
...or GNU/Linux, where Jamulus on JACK works wonderfully with low latency and easy audio routing across apps.
I don't think that "latency" boils down to a single dimension, because there are multiple competing latencies when you play music. It's a complicated feedback loop with multiple input and output channels.
I think how well it works depends on being able to isolate the acoustic sound of your instrument from what's coming through your headphones. Your brain will prioritize whatever comes first, so if you can hear your own acoustic sound against the delayed sound of the band, you will instinctively slow down, and a band allowed to do this will grind to a halt. 
Electric instruments are easy, because they have little or no acoustic sound. I found it much easier to play solid body electric bass than double bass, though double bass is my main instrument. My hands adapted to the latency, as they do anyway because each instrument has inherent latency.
Winds and voice are difficult, because you hear yourself through bone conduction. Drums are hard to isolate with headphones. But people tend to (mistakenly) get the time from the drummer.
It fell to the electric instruments to "drive" the band. That was profoundly fatiguing. When I tried to take a solo, the band would get lost. And my solos can be blatantly rhythmic if I need them to be.
This is compounded by the fact that a lot of musicians are not techies, so the explanations about how they have to play differently, and work with the technology, goes over their heads.
 Imagine playing at 120 beats per minute, which is 500 ms per beat, with 30 ms latency. That's like 6% per beat. So while everybody says that 30 ms is pretty good, that much latency with the feedback loop described above will cause the band to lose tempo by 6% per beat. And 120 is a relatively stately tempo if you like "hot" jazz.
So in the end we designated some instruments to drive the song. Usually its the drums. The person who drives the song must not wait for other instruments and keep their own tempo, and everyone else try to time their instruments to be in-sync with the drums. It works quite well for us.
I guess I’m a musician that makes this mistake. Where does it come from?
Although, optimally, the tempo just seems to sustain itself, a condition that's only met by the best players. Playing bass alongside a drummer with great time is certainly a better experience, and there are drummers with better time than me. Time is one of those things that's a lifelong pursuit, not something that is ever "good enough." Likewise intonation.
Using Jamulus, it seems to work best when the tempo is driven by instruments that can be isolated from the feed coming from the headphones, so for instance electric bass works better than double bass, though they both function as "bass."
Another group was interested, but the setup was too complicated for them (wired ethernet, messing around with asio). But if you can get beyond that, it's apparently pretty great. There might be a product idea there, a plug and play solution that avoids the manual setup.
It's unknown if Starlink will routinely route packets directly between user terminals or if that will be a special service sold only to customers willing to pay a premium. We'll see when the constellation is ready.
The speed of light gives you ~186 miles per millisecond, which assuming you need to make a round trip in 20ms gives you an upper distance of ~1900 miles for live performance.
In reality however this will be much lower due to processing overhead, speed of light in fiber (~0.7e), and internet backbone topology.
Curious to know if the creators have done any tests, (or can show any demos) of the effective range this works at?
But in general, 20ms is considered the top end of fine. A good player can totally feel the difference between 10 and 20 though. Under 10 is not really an issue. After that, you have some stuff to get used to and while you might be able to jam, it's affecting you.
> There are a lot of people arguing that online jamming does not work because the overall delay is too high. They claim that latencies above 10 ms are not acceptable. In Jamulus typical overall delays are in the range of 30 to 70 ms. It is also typical that audio dropouts occur frequently (usually every 2 to 10 seconds per audible pop/interrupt sound). It has been our experience that, if the audio dropouts are not occurring too often, they are not perceived if the band members concentrate on their playing. If one focuses on the dropouts (and this is what people usually do when they try out the software) it feels annoying.
If you pull a few tricks, it wouldn't be anywhere near impossible to manage sub-20ms latency on your average connection. Now, over a web browser? I couldn't tell you. I wouldn't bet on it. But definitely possible natively.
In your case, 25ms network latency is equivalent to being 8 meters from another musician. That's basically "the other side of a medium-ish room."
It is funny seeing HNers so incredulous that something like this would work, when apparently lots of musicians have been using stuff like this for over a year, without issue. I wish more HNers realized that being smart isn't about knowing things. It's about knowing what you know, and understanding things like biases.
"Well, my knee-jerk reaction is that latency would be a problem. But I'm not a musician and I have no experience here. And clearly a bunch of people put a fair amount of effort into this. Let me see if my knee-jerk reaction is correct by either waiting to seeing what people who have actually used it say, or doing out the math to compare typical network latency to sound-in-air delay."
That's what a smart person says to themselves.
A smart person does not say, "This couldn't possibly be practical, I'm going to comment immediately to that effect."
18.104.22.168 is anycasted to many points of presence, one of which might be in the same state as you, so this data point is somewhat irrelevant. I'd be more interested in seeing what king of latency & packet loss you experience to another residential connection far from your location.
Like any modern video game, or half of the decent video chat options in the world.
Good musicians can adapt to low-latency group playback, but it's difficult, and at that point, it isn't true realtime collaboration. The band/musicians are more focused on staying in time and fighting the latency versus really listening to each other and being a band.
One great tool for doing long-distance collaboration is Endlesss. (yes, three "s"s). Loop-based work that allows for real collaboration, and getting around the latency issue.
I had the same worries when my father told me about Jamulous and that his band started using it during the pandemic. They are a cajun band and mostly jam and they were singing the softwares praises and have used it for all practice leading up to the real life perfomances they've had the last few weekends.
It seems it's not actually a big issue, at least over a distance of about 300 miles.
Instead of trying to beat network latency (which is impossible) endlesss provides a shared looper and a clock to sync all your gear to. This way you can jam with people with zero audible latency, building up ideas. And a bonus is that every version (“riff”) is stored and can be revisited or exported into a DAW. It’s an incredible tool.
For those who are too lazy -- TL;DR:
The software allows players to set up servers easily so if a band is geographically close, but remotely dialing in, latency is reduced. The software is intended to be used as the primary method of sound feedback so it's advised that amps are turned off and earplugs/headphones are used to block out "local" sound. And finally, the author of the case study found that their experience with latency was between 30-70ms and that was more than acceptable. The only problem stemming from this latency was that songs with fast drum fills needed some adjustments to slow down the drum fills.
8 months ago, before Jamulus became more popular in my area, we used Clubhouse to connect with more musicians. We started a Clubhouse room and streamed the sound from Jamulus to that room. We put the server IP address in the room title and in the bio so listeners can connect to our server and jam with us. This brought our group from 3 people to about 20 people.
No, it's not perfect but c'est la vie as they say
Rather than reduce latency, why not increase latency so that for the participant with the worst bandwidth, it can be delayed 1 sec, for the one with the best, it is delayed so that it syncs with the other person.
For a period of time when all participants have clicked that they are ready, real time is suspended, and some kind of calculation is made that will the voices in sync.