
Apple’s Low Latency HLS differs from the community-developed solution - UkiahSmith
https://mux.com/blog/the-community-gave-us-low-latency-live-streaming-then-apple-took-it-away/?a=b
======
jhallenworld
I've been recently tasked to find a live video solution for an industrial
device. In my case, I want to display video from a camera to a local LCD and
simultaneously allow it to be live streamed over the web. By web, I mean that
the most likely location of the client is on the same LAN, but this is not
guaranteed. I figured this has to be a completely solved problem by now.

Anyway, so I've tried many of the recent protocols. I was really hoping that
HLS would work, because it's so simple. For example, I can use the gstreamer
"hlssink" to generate the files, and basically deliver video with a one-line
shell script and any webserver. But the 7 second best case latency is
unacceptable. I really want 1 second or better.

I looked at MPEG-DASH: it seems equivalent to HLS. Why would I use it when all
of the MPEG-DASH examples fall back on HLS?

I looked at WebRTC, but I'm too nervous the build a product around the few
sample client/server code bases I can find on github. They are not fully
baked, and then I'm really depending on a non-standard solution.

I looked a Flash: but of course it's not desirable to use it these days.

So the solution that works for me happens to be the oldest: Motion JPEG, where
I have to give up on using a good video compression (MPEG). I get below 1
second latency, and no coding (use ffmpeg + ffserver). Luckily Internet
Explorer is dead enough that I don't have to worry about its non-support of
it. It works everywhere else, including Microsoft-Edge. MJPEG is not great in
that the latency can be higher if the client can't keep up. I think WebRTC is
likely better here.

Conclusion: here we are in 2019 and the best low latency video delivery
protocol is from the mid-90s. It's nuts. I'm open to suggestions in case I've
missed anything.

~~~
squeaky-clean
I have no idea what the underlying tech is, but Steam Link can do extremely
low latency on the same network and very low latency over the internet. It can
also stream non-game applications, though I imagine automating steam is a
nightmare.

~~~
AstralStorm
And Steam uses RTMP for upload like anyone she would do. Just like OBS and
other tools.

Sadly browsers do not come RTMP enabled.

~~~
squeaky-clean
Thanks for the info. I did a quick search before commenting but couldn't find
out what the backend was.

------
chime
The major criticism the author has is the requirement for HTTP2 push for
ALHLS, which many CDNs don't support. While I agree it is a valid criticism, I
am glad Apple is forcing the CDNs to support push. Without the 800lb gorilla
pushing everyone to upgrade, we would still be using pens on touchscreens.

I am not a fan when Apple obsoletes features that people love and use. But I
always support when Apple forces everyone to upgrade because friction from
existing providers is what keeps things slow and old. Once Apple requires X,
everyone just sighs and updates their code, and 12mo later, we are better off
for it.

That being said, I agree with author's disappointment that Apple mostly
ignored LHLS instead of building upon it. Chunked encoding does sound better.

~~~
spenczar5
There are good reasons CDNs don't support http/2 push. It’s hard to load
balance and hard to operate, since it requires a persistent TCP connection
with the client for the entire duration of the stream, which can be hours. It
has consequences that echo almost everywhere in the architecture.

~~~
chime
I agree. But now there is motivation for the CDNs to solve these problems in a
clever way so that users can get better performance and lower latency.

~~~
spenczar5
They are likely to “solve” them by charging more. Serving low-latency HLS to
Apple devices will cost more, continuing consolidation into the few tech
giants big enough to pay what it takes to get inside the iOS walls. Hardly
progress.

~~~
zackbloom
I believe Cloudflare supports it and it's free.

------
jedberg
It's ironic that "live streaming" has gotten worse since it was invented in
the 1930's. Up until TV went digital, the delay on analog TV was just the
speed of light transmission time plus a little bit for broadcasting equipment.
It was so small it was imperceptible. If you had a portable TV at the live
event, you just heard a slight echo.

Now the best we can do is over 1 second, and closer to 3 seconds for something
like satellite TV, where everything is in control of the broadcaster from end
to end.

I suppose this is the tradeoff we make for using more generalized equipment
that has much broader worldwide access than analog TV.

~~~
mantap
Google seems to think they can implement video gaming over IP. And they
probably can, my ping to them is only 9ms, less than a frame.

There is just a broad lack of interest in reducing latency past a certain
point unless there is a business reason for it. People don't notice 1 second
of latency.

~~~
baybal2
> Google seems to think they can implement video gaming over IP.

No the didn't. Early attempts at streaming videogame were unplayable even with
a server in another room or a direct DC connection.

And they want it to work over an average internet connection in America.

No, the definitely did not solve the issue

~~~
girvo
And yet, I was able to game competitively from my apartment in Brisbane, using
a server in Sydney, using Parsec; usually coming in at less than a frame of
latency, sometimes just over a frame. This was two years ago, too. And
Australia isn't known for it's amazing internet connections (though mine was
better than most).

------
floatingatoll
This title is unnecessarily inflammatory with intent to gain our sympathy to
the position presented.

The technical writeup of this post are spot-on, though. I prefer less drama
with my bias but I’m very glad I read this.

~~~
CharlesW
Thanks, that's exactly how I felt — that there’s a _really good_ and useful
article in here, but clouded by assumptions and an attempt to create
controversy.

------
nvahalik
Looks like as far back as 2014 research has pointed to some big gains using
HTTP/2 push:
[https://dl.acm.org/citation.cfm?id=2578277](https://dl.acm.org/citation.cfm?id=2578277)

------
buraktamturk
> A Partial Segment must be completely available for download at the full
> speed of the link to the client at the time it is added to the playlist.

So with this, you can not have a manifest file that point to next future
chunks (e.g. for up to next 24 hours of live stream) and delay proccessing of
http request until the chunk became available. Like HTTP Long Polling used for
chunks.

> On the surface, LHLS maintains the traditional HLS paradigm, polling for
> playlist updates, and then grabbing segments, however, because of the
> ability to stream a segment back as it's being encoded, you actually don’t
> have to reload the playlist that often, while in ALHLS, you’ll still be
> polling the playlist many times a second looking for new parts to be
> available, even if they’re then pushed to you off the back of the manifest
> request.

Which could be avoided if Apple didn't enforced the availibilty of download
"at the full speed" once it appeared in the manifest. (long polling of chunks)

LHLS doesn't have this issue as the manifest file itself is streamed with
chunked responses hence it makes sense. (streaming manifest file)

> For the time being at least, you’ll have to get your application (and thus
> your low latency implementation) tested by Apple to get into the app store,
> signaled by using a special identifier in your application’s manifest.

And this makes me to think about the implementability of the 1st and 2nd point
on ALHLS. Maybe the current "implementation" is compatible but not with the
specs itself.

~~~
Fripplebubby
To your last point,

> Maybe the current "implementation" is compatible but not with the specs
> itself.

It's perhaps worth noting that this is a "preliminary specification" and an
extension of HLS. HLS itself is an IETF standard (well - an "Internet Draft"):
[https://tools.ietf.org/html/draft-pantos-http-live-
streaming...](https://tools.ietf.org/html/draft-pantos-http-live-streaming-23)

~~~
TD-Linux
It is not an IETF standard - those have RFC numbers. It is just a personal
draft - any IETF member can upload one of those, regardless if it's useful.
I'm happy that it has a specification - but it's just a one-man project, not
something that has gone through the IETF standards process.

~~~
lultimouomo
A couple of years ago it finally was published as an RFC:
[https://tools.ietf.org/html/rfc8216](https://tools.ietf.org/html/rfc8216)

------
kevleyski
Apple low latency test stream I set up if useful (uses CDN) [https://alhls-
switchmedia.global.ssl.fastly.net/lhls/master....](https://alhls-
switchmedia.global.ssl.fastly.net/lhls/master.m3u8)

------
sparker72678
Any chance some of this is related to patent avoidance?

------
ec109685
> measuring the performance of a blocking playlist fetch along with a segment
> load doesn’t give you an accurate measurement, and you can’t use your
> playlist download performance as a proxy.

I don’t see why this would be the case. If you measure from the time the last
bit of the playlist is returned to the last bit of the video segment is pushed
to the client, you’ll be able to estimate bandwidth accurately.

~~~
Fripplebubby
> from the time the last bit of the playlist is returned to the last bit of
> the video segment

Based on my loose understanding of HTTP/2 server push and ALHLS, the sequence
of events will be:

1\. Client requests playlist for future media segment/"Part"

2\. Server blocks (does not send response) until the segment is available

3\. Server sends the playlist ("manifest") as the response body along with a
push promise for the segment itself

The push then begins with the segment.

The push stream can presumably occur concurrently with the response body
stream. So I don't think you can wait until every bit of the playlist comes
in. Likewise, you can't use the playlist bits itself to gauge bandwidth
because the server imposes latency by blocking.

~~~
ec109685
It’s still over the same TCP connection, so there is no way for the push to
beat the playlist.

------
shmerl
As usual, Apple pushes NIH, instead of supporting DASH which is the common
standard. And they also tried to sabotage adoption of the later by refusing to
support MSE on the client side that's needed for handling DASH.

~~~
citruspi
> As usual, Apple pushes NIH, instead of supporting DASH which is the common
> standard.

I mean... HLS predates DASH. It would've been hard for them to support a
common standard which didn't even exist at the time. Initial release of HLS
was in 2009[0], work started on DASH in 2010[1].

I'd also disagree with the characterization of DASH as "the commmon standard"
\- it's certainly a legitimate standard, but I feel like support for HLS is
more ubiquitous than support for DASH (please correct me if I'm wrong).

[0]
[https://en.wikipedia.org/wiki/HTTP_Live_Streaming](https://en.wikipedia.org/wiki/HTTP_Live_Streaming)

[1]
[https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_ove...](https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP)

~~~
shmerl
Predating doesn't stop them from supporting something else once it becomes
common. They don't do it since they want to impose HLS on others. And their
refusal to support MSE[1] on iOS stinks even more clearly as an anti-
competitive method to do it.

[1].
[https://en.wikipedia.org/wiki/Media_Source_Extensions](https://en.wikipedia.org/wiki/Media_Source_Extensions)

~~~
xenadu02
Why didn’t MPEG just adopt HLS?

I think you have the NIH cause-effect the wrong way around.

~~~
shmerl
Apple isn't exactly the champion of free standards. Is HLS free for others to
adopt? DASH is. The same messed up story happened with touch events for
JavaScript. What Apple were pushing wasn't free.

~~~
jchb
> DASH is free to adopt

Citation needed :)

From
[https://www.mpegla.com/programs/dash/](https://www.mpegla.com/programs/dash/)

> The royalty for DASH Clients, which is now payable from January 1, 2017
> forward, remains US $0.05 per unit after the first 100,000 units each year.

~~~
sjwright
And that sounds like reason enough for Apple to ignore the DASH spec. Case
closed IMHO.

~~~
shmerl
Apple should be out of video business, if they are scared of patent trolls.

But the problem is the reverse here. Apple have patents on HLS itself which
are not free, so it's not suitable for general adoption.

