

Ask HN: Can there be a true real-time collaboration over any distance on earth? - tsenkov

It might be a stupid question due to my lack of knowledge in physics and low-level networks, but I don&#x27;t see how there can be a true feel for a real-time (with imperceptible to the human brain latency) collaboration.<p>I made the following simple calculation - the biggest distance on earth between 2 points would be 1&#x2F;2 * 40000km (the circumference of the Earth on the equator) = 20000km.<p>Lets say these 2 points are connected with an absolutely straight optical fibre. For simplicity&#x27;s sake, there are no penalties for signal amplification and repetition and the speed of the signal is 300000 km&#x2F;s.<p>From these const&#x27;s we get that the latency (in milliseconds) of this signal would be:<p>1000 &#x2F; ( 300000 &#x2F; 20000 ) ~ 67 milliseconds.<p>So, 2 people, 1 in Sydney AS and the other in London UK, will have a theoretical min latency = 1000 &#x2F; ( 300000 &#x2F; 16983 ) ~ 57 milliseconds.<p>While this might seem OK, its far away from it - when a piano player plays on a midi keyboard, the perceptible latency between hitting a key and hearing the sound starts from 10 milliseconds, grows annoying on 20 and for most becomes hard (even impossible) to play from &gt;30 milliseconds.<p>If I was to attempt to design a solution I would probably divide earth&#x27;s area on regions, but still I am not sure what the real distance would be for latency &lt;=10 milliseconds?<p>It will be really interesting to read your thoughts on this topic.
======
pedrocr
The first-person shooter crowd has probably run into these limits so that's a
good market to go see what works and what doesn't. Your music example may be a
little misleading. Although we can detect small delays (10 or 20ms) our
reaction time for input is an order of magnitude greater (100ms or more), so
we may be able to tell a note is off by 10ms but we only do it 100ms after the
fact. Depending on what you're doing that distinction can be important. For
human communication for example a 500ms roundtrip time would still allow a
conversation easily.

Your math is also slightly off because the speed of light in fiber optic is
only 200000km/s, so the minimum latency between two opposite points on the
globe using fiber is 100ms.

PS: The circumference of the earth is also 40 thousand km not 40km, so I
suppose the correct notation is 40Mm, although that's unusual.

~~~
tsenkov
Even if the reaction time is higher than what we detect as a latency, it's
still a bad news, but delivered ... later. :)

I left 300000 km/s, because I thought that maybe this is the absolute
theoretical limit (there might be some breakthrough in reflective materials,
who knows?).

Sorry for the typo of the circumference of the earth, I think I've fixed it
before your comment.
[https://news.ycombinator.com/item?id=7862387](https://news.ycombinator.com/item?id=7862387)

~~~
pedrocr
>Even if the reaction time is higher than what we detect as a latency, it's
still a bad news, but delivered ... later. :)

It would be hard to do a distributed band. It's an interesting case because
it's not a question-response situation where the roundtrip is crucial, because
for the most part each musician is playing independently and only listening to
the rest to stay in sync. I wonder if you could do this:

1) Have every member stream to every other member directly with an accurate
absolute timecode (good NTP-adjusted or even GPS clocks)

2) Have every member mix in all the streams using the absolute time code and
buffering as much as needed for the slowest stream (so you'd have 100ms of
buffer in the worst case).

This would give you a synced up band where every member listens to the same
mixed stream and yet everyone is listening to themselves on a slight delay. I
wonder how well our brains could cope with that introduced delay and if it
would then screw up live performance, or if we can adapt to that sort of thing
quickly.

------
nomailing
Maybe it would be possible if you can circumvent the human sensory organs and
directly feed the signal to higher brain areas. The delays in sensory
modalities are anyway very diverse and not at all synchronously processed in
real-time. Even within the same modality the signal is disintegrated in the
brain. For example an image from the retina is processed with completely
different delays in the ventral and dorsal stream. The ventral stream has
latencies of several 100 ms [1]. It is still an open question in neuroscience
how these different latencies are combined in higher areas in a way that they
are perceived as synchronous.

So even if you are in a conversation with people directly in front of you, you
do not actually have a real-time system. It takes roughly 400 ms until the
image of a face of a person standing in front of you is processed in the
ventral stream and is actually detected. On the other hand there is the
auditory modality which is much more sensitive to latencies. But in the end
the brain gives you the illusion that anything you perceive is actually real-
time, although it is not. So if your brain decides to give a motor signal to
actually press a piano key, then only many 100 ms later the actual sound will
arrive. The perceptible latency of hearing a sound is actually only because
the brain itself takes into account the delays in itself.

So in the end it comes down to trick the brain into believing something is
real-time. This is for example done in multiplayer ego-shooters where the
motions of other players are actually interpolated into the future and later
on corrected if the other player actually changed their last motion command.

[1] Barragan-Jason G, Besson G, Ceccaldi M and Barbeau EJ (2013) Fast and
famous: looking for the fastest speed at which a face can be recognized.
Front. Psychol. 4:100. doi: 10.3389/fpsyg.2013.00100

------
enqk
As a data point, an organist may have to deal with latencies between few
hundreds to seconds. So it's not that we can't deal with latency at all.

Ninjam[1] arranges realtime music collaboration by exploiting musical
structure and its construction in intervals. Human collaboration which can be
similarly structured could still be arranged to work with.

Counter examples: games appear to work over latencies because a lot of them
are about restricting other's agency and making an experience worse (i.e. not
true collaboration) 2d fighting game can be seen for instance as two game
designers attempts to make a game worse for the others. It would be
interesting to study why games like left4dead (which requires team play and
collaboration) still work.

[1] [http://www.cockos.com/ninjam/](http://www.cockos.com/ninjam/)

------
sferoze
Even when jamming in my personal studio anything above 5ms latency on my
electric drum set can be noticed. I have to lower the IO buffer size to about
64 samples to get above 5ms latency.

I would love to be able to jam with friends over long distances. For this to
actually be enjoyable without frustration the latency would have to be no more
than 5-10ms.

I wonder how much latency a local telephone call would use? Then maybe it is
possible to set something up where you jam over the telephone line and each
musician can hear each other just fine.

~~~
jchrisa
One way to do this is to jam based on a loop, so everyone can be off by one
bar. I think there are some services that do this.

~~~
turnip1979
Specific pointers? I'm curious how this would work in practice.

------
rz2k
An interesting point to consider is that `(speed of sound) * 67ms ~ 25yds`. A
musician could elaborate on the challenges involved in playing in a large
symphony orchestra, and yet it's accomplished.

~~~
qq66
That's one of the reasons you need a conductor. Orchestra players learn pretty
early that if you listen for your cue you'll come in late.

~~~
3JPLW
What a wonderful comparison. And it gives hope to the fact that software at
either end could play the role of this conductor — it can negotiate a "true"
sync time and pre-empt the delay, just like a conductor.

~~~
tsenkov
Indeed. Still, it might be enabling to someone who's already used to the
delay, but in a 4-10 person band, there isn't such an issue.

I have to check something, though - how do things change if all musicians
tracks go through a mixer and everyone listens to his own mix in headphones.
On stage, the delay from the main speakers is huge for the perfomers. That's
why they have either in-ear headphone or speaker monitor. Can't this work for
symphonic orchestras? And does it solve their latency problem?

~~~
qq66
Symphonies are pretty traditional in their methods, and the conductor is
already an adequate solution to the problem (and has many other roles).

------
waqf
What exactly is your application (and your budget)?

For videoconferencing, musical-instrument-playing (as in your example), or
multiplayer video games with less latency than the speed of light appears to
allow, you could take advantage of Libet's work (see
[http://en.wikipedia.org/wiki/Neuroscience_of_free_will](http://en.wikipedia.org/wiki/Neuroscience_of_free_will))
showing that with neural interfaces you can reliably predict people's physical
actions hundreds of milliseconds before they actually do it … which would give
you hundreds of milliseconds to send off the signal and get a response which
you play back to them "instantly" when they perform the action.

~~~
tsenkov
I haven't started anything, but I always wanted to be able to have a band not
restricted to the local pool of musicians and this is what made me think about
it. So I started looking for a solution, I found about the Internet2 project
and when I saw that they should be pretty close to the theoretical limit, I
decided to check what that limit really is. That's when I realized that I
probably couldn't do this for any earth-distance without any prediction of
data.

------
jb55
In fighting games people are very sensitive to latency when you get to a high
enough level. There is an interesting networking library called GGPO
[http://en.wikipedia.org/wiki/GGPO](http://en.wikipedia.org/wiki/GGPO) that
cancels lag out using a clever algorithm. I imagine you could apply a similar
technique to other problems.

~~~
FatalLogic
The wiki article isn't very clear, but it sounds like GGPO hides part of the
network latency by disguising it as the natural delay between deciding to act
and the action taking effect, therefore delaying actions in the game in a
believable way. Is that right?

------
frankus
There's an interesting demo of the effects of lag on something like VR. I
think it's intended as a statement against avoidable lag, but the conclusion
is the same :)

[http://livingwithlag.com](http://livingwithlag.com)

------
s_kilk
Short answer, no.

As you've observed, even under theoretically perfect conditions, the latency
is much more than would be tolerable for true real-time applications. And
that's not even counting the time spent in hardware, on the various cpus in
the path, buffers, etc.

~~~
tsenkov
What about a fragmentation of the area - what would be the radius from your
location where you would be able to collaborate with others at <=10
milliseconds of latency?

~~~
6d0debc071
Roughly 2997 kilometres, with C as the speed.

~~~
tsenkov
What about with C' (being the current state-of-the-art/max)?

I guess the current speed, judging by [1] is about 200000 km/s (for light in
optical fibre), which leads to 2000 km radius (or diameter, if more than 2
parties are involved).

[1]
[https://news.ycombinator.com/item?id=7862412](https://news.ycombinator.com/item?id=7862412)

------
fjabre
Yes, its' true. It turns out that the speed of light isn't really all that
fast.

[http://en.wikipedia.org/wiki/Superluminal_communication](http://en.wikipedia.org/wiki/Superluminal_communication)

------
cliveowen
The circumference of the earth is roughly 40 _thousand_ kilometers.

~~~
IvyMike
Fun fact: This is easy to remember if you know that the kilometer was
originally defined so that the distance from the equator to the poles was
10000 kilometers.
[https://en.wikipedia.org/wiki/Metre#Meridional_definition](https://en.wikipedia.org/wiki/Metre#Meridional_definition)

~~~
turnip1979
Neat. So 10000km = an "arc" from the equator to a pole. If you assume the
Earth is a sphere, the circumference at the equator = 4 * 10,000.

------
lifeformed
67ms isn't too bad. First person shooter games feel pretty much fine at 67ms.
Once you get to 100+ does it get very noticeable, and past 150 is when it gets
unplayable.

However, it's important to note that FPS games use a lot of tricks with
client-side estimations to infer the game state. But depending on the
application, you could probably use similar tricks to make things feel
responsive.

------
edo
Yes, this can in theory be achieved through quantum entanglement:
[http://www.engadget.com/2014/05/30/physicists-claim-
reliabil...](http://www.engadget.com/2014/05/30/physicists-claim-reliability-
breakthrough-in-quantum-teleportati/)

~~~
alexknvl
Nope. Quote from the article: "the process still requires the transmission of
measurement data from one side to the other via classical communication (read:
regular, non-instantaneous communication)".

------
qdcarnicelli
See Stuart Cheshire's ""Latency and the Quest for Interactivity", from way
back in 1996:
[http://www.stuartcheshire.org/papers/LatencyQuest.html](http://www.stuartcheshire.org/papers/LatencyQuest.html)

------
makmanalp
Check out ninjam, which tries to get around this by exploiting the idea that
music "loops" in intervals anyway:
[https://www.youtube.com/watch?v=Gg2UtVdwQYo](https://www.youtube.com/watch?v=Gg2UtVdwQYo)

------
willvarfar
I think the premise is a bit wrong.

When talking on the phone to someone on a satellite phone in a distant desert,
you get a noticeable delay but its not a problem.

"Real time collaboration" anywhere on earth has worked for a century.

------
frozenport
Through air you get 30ms latency after 10 meters (340 m/s * 30 ms). So, doing
a back and forth communication is much harder.

~~~
tsenkov
Latency of what? Sound? Light? :)

