
HEVC and the No Good Month of January - clouddrover
https://streaminglearningcenter.com/other/hevc-terrible-horrible-no-good-bad-month-january.html
======
vardump
Companies that caused HEVC patent mess had it coming.

So I understand AV1 is great for streaming content when you have a lot of
encoder computing resources.

What about other video codec application areas?

Like real-time peer-to-peer low latency streaming (<50 ms codec latency)? Like
for remote desktops, remote gaming, video conferencing, etc. What rules the
roost in that department?

More specifically, which codec would have the best image quality for low-
latency 400-4000 kbit/s (== ~50-500 kB/s) 720p/1080p streaming for different
computing resources (like say for these cases: ARM A53, high-end ARM (like
phone flagship models), mobile GPU, modern x86 chip and discrete GPU)? h.265
with B-frames disabled? Or something else?

What can achieve the lowest latency while maintaining low bandwidth
requirements? (So not including obvious bandwidth-hogging techniques like
streaming jpegs or [codec x] I-frames.)

~~~
incognito65423
Streaming of computer screens is quite different compared to real life
footage. UIs contain very shallow gradients (thus you want 10bit+), lots of
flat areas which are easy to entropy-encode without any frequency domain
quantization at all (so a modern codec that has passthrough blocks) and font
rendering engines use subpixel anti-aliasing so you want 4:4:4 chroma. And
that annoying effect where image quality refines gradually after large-scale
changes on a screen (not enough bits allocated to an I-frame?).

Screencasting is basically quite a different beast compared to live-action or
even animation footage. At a minimum you need higher profiles of a modern
codec and tweak the encoder settings for low latency AND different bit
allocation compared to the usual.

~~~
vardump
> Streaming of computer screens is quite different compared to real life
> footage. UIs contain very shallow gradients (thus you want 10bit+), lots of
> flat areas which are easy to entropy-encode without any frequency domain
> quantization at all...

Well, you can get pretty good results with h.264 without B-frames. Modern
composited desktops have caused traditional changed block (RLE, LZ) lossless
codecs to perform badly.

> font rendering engines use subpixel anti-aliasing so you want 4:4:4 chroma

I think you can just gradually stream more (chroma) bits to stationary areas
containing high detail (frequency) information, like text. In other words,
tunable bit allocation between motion and stationary detail.

No matter what, there's no way to know what's the subpixel layout on the
receiving end. Luckily Retina/HiDPI displays are increasingly lessening
importance of subpixel rendering.

~~~
incognito65423
The thing is that 4:2:0 chroma is the default in most codecs because the main
profiles only support that. You couldn't stream full chroma video on twitch
even if you wanted to.

And subsampled chroma looks terrible on colored text, edges between flat
colored areas and similar things.[0] So 4:4:4 is a must, only once you have
that you can think about adaptive quantization for chroma.

[0]
[https://upload.wikimedia.org/wikipedia/commons/1/1c/420-prog...](https://upload.wikimedia.org/wikipedia/commons/1/1c/420-progressive-
still.png)

~~~
vardump
I agree with that, but 4:4:4 also means there's twice as much data.

I think you should be able to get away with using chroma subsampling for
moving portions while allocating bits in data stream for stationary areas,
without subsampling (by using some heuristics, perhaps.)

Chroma from luma techniques might be interesting in this context.
[https://people.xiph.org/~xiphmont/demo/daala/demo4.shtml](https://people.xiph.org/~xiphmont/demo/daala/demo4.shtml)

~~~
clouddrover
Chroma from luma happens to work particularly well for video game content.
It's one of the reasons Twitch is interested in using AV1 as soon as possible.
Here's a talk by Thomas Daede where he discusses chroma from luma:

[https://www.youtube.com/watch?v=yKEDf5-2sT4](https://www.youtube.com/watch?v=yKEDf5-2sT4)

Slides:
[https://people.xiph.org/~tdaede/demuxed_av1_2017.pdf](https://people.xiph.org/~tdaede/demuxed_av1_2017.pdf)

Twitch is also interested in AV1 for its s-frames feature which allows for
better compression in low latency live streaming:

[https://www.youtube.com/watch?v=o5sJX6VA34o](https://www.youtube.com/watch?v=o5sJX6VA34o)

[https://www.youtube.com/watch?v=AeSNW2T7QwE](https://www.youtube.com/watch?v=AeSNW2T7QwE)

------
sturmen
The referenced blog post by the founder/chairman of MPEG is worth reading.[1]
In it, Chiariglione expresses regret not for the money and control his
organization is losing, but for the loss of competition which he claims will
slow innovation. It will be interesting to see if this comes to pass, but my
hope is that the collaboration of these companies will outpace the
historically siloed research consisting of a dozen companies duplicating work.

[1] [http://blog.chiariglione.org/a-crisis-the-causes-and-a-
solut...](http://blog.chiariglione.org/a-crisis-the-causes-and-a-solution/)

~~~
Sir_Cmpwn
The last thing I want to hear is one of computing history's most harmful
patent holders complaining about a lack of competition.

------
post_break
HEVC has been such a pain for me. You need to run High Sierra to use it, it's
not just a plug in for the mac but a whole operating system change. Then GoPro
switched to it and as a moderator for the GoPro subreddit, it has caused so
many support questions as to why they can't edit the video they took. Yes
there are benefits, but man it's been such a pain.

~~~
pantalaimon
It's funny how it's so seamless on Linux when even on windows you now have to
install it separately.

------
bastawhiz
The article mentions that AV1 is slow. I seem to remember that H.264 was
painfully slow in the beginning. Is this simply a matter of waiting for
hardware to add support for AV1? Or does someone need to sit down at the
spinning wheel and write some assembly?

~~~
incognito65423
Yes, the h.264 and hevc reference implementations are also terribly slow and
alternative implementations with performance optimizations, tradeoffs and
additional encoding features were developed over long time spans.

The bitstream formats provide enough flexibility for the encoders to go beyond
what the reference implementation does.

Software encoders provide the best quality, hardware encoders are kind of meh
but do their job in a low power envelope.

~~~
ksec
Well, yes, the reference implementation are, reference implementation. But
h.264 and HEVC Ref Encoder and let say X26x encoder were only a difference of
10 - 50 times.

Av1, we are taking about 1000x.

------
NelsonMinar
HEVC is starting to get some usage in pirate distributions. Most TV and Movies
are released with H.264 video. But some releases are also coming out in HEVC /
x265. The files are typically 1/3 to 1/4 the size for the same visual quality.

------
seanalltogether
The pirating community appears to have embraced HEVC however, and they can
have a lot of influence in determining which codecs/containers are supported
on playback devices like game consoles, phones and tvs.

~~~
philjohn
And don't forget the different groups "race" to get their release out first,
and get the kudos.

So if one decided to use AV1 with its much longer encode times, they would
always lose.

Saving bandwidth isn't usually a concern.

------
markdog12
I recently noticed Chrome Canary added a flag to enable AV1, any samples out
there?

~~~
muizelaar
[https://demo.bitmovin.com/public/firefox/av1/](https://demo.bitmovin.com/public/firefox/av1/)
has a sample that works in Firefox Nightly.

