
Achieving super low latency for critical real world internet applications (2018) - yoquan
https://www.netdevconf.org/0x12/session.html?achieving-super-low-latency-for-critical-real-world-internet-applications
======
lordnacho
HFT guy here.

There's problems with using the internet if you want low latency. Not just low
latency, but also low jitter, ie consistent latency. You're sharing a bunch of
infrastructure with other people, so sometimes your packet will have to wait.
Try to ping some internet server a few times and you'll see it changes. I've
done this with rented lines, and the number is the same every time.

Not inherent to the internet is the fact that REST seems to be a standard.
Probably people will want a REST interface. So that means serializing and
deserializing messages, and the messages are always going to be a lot longer
than they need to be. It's not that you have to do it this way, though.
There's also the fact that internet architectures tend to have a bunch of hops
(load balancer / gateway / etc), which isn't going to help, but definitely
something you could work around.

The figures he gives are interesting. If you really need under 20ms for AR to
work, how is remote surgery going to work? 20ms is close to the time you get
to render a frame in real time games at 60fps, seems a bit fast to me. The
surgeon can't be far away if this is true.

~~~
devereaux
Regarding hardware: I try to not share infrastructure. What I can not fully
control can and will dysfunction, even in datacenters. For example, the LAN
options are most often VLAN. So for my servers, I have extra NICs and direct
connections one-to-one by crossover cable and ARP monitoring. It costs more,
but it quickly pays for itself.

For software, everything is done in house. As you said, it doesn't have to be
done a way or another.

~~~
walrus01
I was under the impression that the most serious HFT doesn't even use
_Ethernet_ anymore. At each end, where it meets a common buy/sell platform,
yes. But not for the competing HFT data link companies' inter-city messaging.

For the shortwave links in place between NY-Tokyo, for several I'm privy to
details of, people have written their own custom dedicated purpose protocols
that implement very low bitrate, ultra low latency data messaging with less
overhead and ASIC packet processing time.

~~~
devereaux
Depends on what you do.

For what I do, Ethernet is acceptable as long as the latency is below 1 ms.
Here're some quick stats for what I consider good enough this specific
usecase:

rtt min/avg/max/std-dev = 0.069/0.095/0.121/0.016 ms

Generally it is better to keep things simple and use existing stuff whenever
possible.

If we are talking about very low latency (say below 30 us) you may need many
other things first, like a good timesource.

Then indeed, when you start rolling your own stratum1 NTP servers, rolling
your own protocols can be a sensible answer to the problem you have.

~~~
MuffinFlavored
At a high level, what do you do with HFT? You aren't doing the trading, right?
You just manage the systems behind it? So what is? Are there really bots "in
production" that are making buying and selling decisions tied to real money so
quickly that <1ms is needed?

~~~
dmurray
The advantage is being first. If you learn in Tokyo that a stock is going to
go up, you can buy all of the available stock in New York, or cancel your
order to sell it. It's going to take you 50 ms to get that information to New
York, but if your competitor takes 49 ms, you lose money; if he takes 51 ms,
you make money. So 1 ms is important, and so is any time difference big enough
to outweigh the noise in the exchange's ordering of events. I know people
measuring and caring about speed gains of single-digit nanoseconds, roughly
equivalent to replacing your two-metre cables with one-metre cables.

The end customer (whoever actually wants to buy stocks) doesn't need the price
to update every millisecond. But no one has yet figured out a better way to
run an electronic exchange than "first come, first served".

Source: also worked in HFT for a long time.

------
client4
I do low latency network architectue for HFTs; I'm not sure there's a real
world application for shaving 50us off a latency for a route outside of this
industry. That said I've been impressed with Cloudflares consistent global low
latency while traveling. It's a habit to ping 1.1.1.1 and see if it goes above
15ms.

------
acranox
It's terrifying to hear him mention self-driving cars as a use case for super
low latencies. If a few milliseconds of network latency compromises some
safety system in a car, that's bad. I'd be engineering a car for worst case
scenarios, not for reliance on a fast network. It makes me think of this.
[https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...](https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing)

~~~
howard941
It's not clear why some folks are dredging the self-driving cars, vehicle-to-
vehicle, and vehicle-to-infrastructure use cases out for 5G but when the
rubber hits the road I suspect vehicles will deploy with dedicated short range
communications for the bits that must be low latency and 5G for ordinary
cellphone use. The wiki page has some starting places for DSRC info.
[https://en.wikipedia.org/wiki/Dedicated_short-
range_communic...](https://en.wikipedia.org/wiki/Dedicated_short-
range_communications) . It _is_ terrifying.

~~~
aphextron
It turns out car-to-car local networking is not really necessary. The use of
4G/5G networking + simulation based modeling and prediction gives great
results. That’s what companies like Nexar [0] are doing.

[0] [https://www.getnexar.com](https://www.getnexar.com)

------
iabacu
Then there are mixed reality applications. It won’t need HFT levels of low
latency, but will demand a lot more bandwidth. Many of the tricks won’t work
there.

That’ll require exploiting edge distribution to its limits, with an entire new
level of internet infrastructure that doesn’t exist yet.

------
gameswithgo
Network Next is a company working on this, not just for critical applications
but also games:

[https://networknext.com/concept/](https://networknext.com/concept/)

------
altmind
If you are looking for information on low-latency computing, including some
real advice how to tune for lower latency, look for info on HFT.

5ms reaction time mentioned on these slides are far cry for what computers can
do. trading servers reaction time packet-in packet-out even without fpga can
take as little as 20 microseconds.

~~~
3chelon
Yes, but they sit right on transatlantic fibre-optic trunk cables. Presumably
leased lines? It's a totally different proposition to a public mobile
internet.

~~~
altmind
Many trading companies now utilize over-the-horizon radio. Its the fastest,
most direct connection available. But even without this:

\- cpu thermal profiles and power governors

\- cpu affinity

\- non-evented io(if you prefer latency over the number of clients served)

\- network interrupt coalescing(actually, disabling coalescing), network
msi-x, hardware offloads, 10gb fiber or DAC connections(very cheap nowadays)

\- lock-free and kinda-crdt data structures(between threads)

\- predictable memory allocations(better - no allocations on critical path)

\- specialized logging and tracing(no stdio)

can drive you quite far.

~~~
bloomer
What is over the horizon radio? I am familiar with over the horizon radar, but
not radio for communications. Do you have any references?

~~~
walrus01
Shortwave radio with big-ass yagis and dipoles, aimed at each other in point-
to-point links. Very low data rates, carrying custom messaging protocols, but
also measurably less latency on critical paths such as NY-Chicago or London-
Tokyo.

There is a big crossover in tech with shortwave band military over the horizon
communications systems, and also serious ham radio enthusiasts who use big
yagis and antenna rotators to talk to each other from opposite sides of the
planet.

Things like chained 6 and 11 GHz microwave between NY-Chicago, which was the
"hot new thing" in HFT 7-8 years ago to achieve better than fiber latency on
that route, seem quaint now.

------
jayd16
For VR and AR you really don't need these low latencies over the internet. As
long as the visual output matches the users head movement the game assets can
have noticable lag. Like most existing games, rubberbanding is an annoyance
but not disorienting.

For those use cases it would be better to come up with a full 3d (not just
stereoscopic) streaming video format that allows for some local camera
movement.

~~~
bewo001
In 5G circles there is this "vision" of VR/AR servers that reside in "edge
datacenters" close to the customer. The user's movements are streamed to such
a server which returns the calculated video stream. This doesn't make sense at
all, of course. But you can put buzzwords like VR/AR on your shiny 5G slides..

------
smartmic
Could RAMCloud be a candidate for this … at least to tackle the datacenter
level, and —isn't the rest more or less out of control?

[https://ramcloud.stanford.edu/docs/doxygen/md_README.html](https://ramcloud.stanford.edu/docs/doxygen/md_README.html)

------
bsder
A whole talk about low latency networking and not a single mention about TSN
(time sensitive networking)/AVB (audio video bridging) which _actually_ does
low-latency?

