
Modern codecs like AV1 can bring better quality video to the open web - ALee
https://blog.mozilla.org/blog/2018/07/11/royalty-free-web-video-codecs/
======
NelsonMinar
I've been using a lot of H.265 (aka x265/HEVC); some pirate scene TV and movie
releases come out in it. It's fantastic. About 1/3-1/4 the size of H.264,
which makes a huge difference both with download and with archiving. The
downside is that there's not a lot of hardware support for encoding or
decoding. Doing it on the CPU isn't bad but definitely not as energy
efficient.

The problem with H.265 is the IP situation is a mess. AV1 looks to be better
in every way. Looking forward to its adoption. So far the pirate scene doesn't
seem to be using it at all.

~~~
CryoLogic
The biggest downside is not energy efficiency IMO, it's speed. The speed
difference between h.264 and h.265 encoding can be easily 10 fold. More-so on
CPU reliant servers without GPUs. There is a big upfront sacrifice to get the
small video size.

~~~
zamadatix
Pascal GPUs h265 encoding is 50% or higher than h264 encoding speeds. A single
GTX 1080 can nvenc h265 8k@30 10bpp. A 150 dollar GPU will outperform a 3,000
dollar CPU only server in encoding (regardless if it's h264 or 265).

The REAL blockers are licensing and, more importantly, legacy consumer
hardware. Whether you want to call the latter a speed or power problem isn't
the real issue, it's that not all consumers have the dedicated hardware and
the older CPU then has the speed and power problems.

~~~
rasz
why do you specifically mention GTX 1080? There is zero difference between
1050ti and 1070 when it comes to encoding, does 1080 change anything?

~~~
IntelMiner
Anecdotal evidence, and possibly an FFmpeg/Nvenc bug

I've found that I'm not able to encode video above the size of the VRAM on the
GTX 1050 in my home server. It seems to spew out errors, despite the GPU's
memory usage being roughly 60MB from ffmpeg

My GTX 1080 meanwhile seems to happily churn through anything and everything
thrown at it

------
gallerdude
The thing I'm most excited about for AV1 is film grain synthesis. I can't
articulate why, but I really love the look of film grain in video.
Unfortunately, on most codecs, film grain is a huge hassle which takes up a
lot of bandwidth. But AV1 can automatically remove the grain, encode, and add
it back in.

~~~
bscphil
Curious about this. Does the grain it algorithmically adds back look the same
as that it takes away? In other words, is it just a better way to compress
existing film grain? Different film sources have extremely different grain
profiles, and people who encode movies (both production companies and pirates)
will have no interest in the aspects of a codec that remove the real grain in
a film to replace it with a generic alternative. I suppose they might adopt
AV1 with that feature turned off, but otherwise AV1 will mostly be adopted for
web video which currently doesn't have grain due to low bitrate encoding.

~~~
tfha
Depends how good you are with an encoder. It's not compression, the grain does
change, but since it's just random anyway, the human eye can't notice so long
as it's the same type of grain.

Encoders generally have a few options to tune the grain that gets added back
in to get it as close as possible to the original source.

~~~
bscphil
I would be interested in seeing a post on these options and their results if
you know of one. Even though grain is "random", it's not random noise and is
very distinctive. No two film stocks (and probably no two movies) will have
the exact same grain. It would require incredibly precise tuning to manually
add back grain that matches the original in the film.

~~~
severine
Search for "film grain modeling".

~~~
lucb1e
When I do that, I get (1) a Wikipedia page literally titled "film grain
modeling", but it's only a redirect to H.264 and the only mention of "grain"
on that page is in the redirect notification; (2) a patent; (3) some
scientific paper supported by a few images that only show noise (is that what
we're talking about? I expected grains in the sense of particle generators);
(4-5-6) paywalled scientific papers; (7) a Facebook page...

A link would be nicer than an apparently very specific search term.

~~~
theoh
This page implies that the grain synthesis step is informed by analysis of the
grain in the uncompressed video: [https://bitmovin.com/cool-new-video-tools-
five-encoding-adva...](https://bitmovin.com/cool-new-video-tools-five-
encoding-advancements-coming-av1/)

That is confirmed by this article:
[https://jmvalin.ca/papers/AV1_tools.pdf](https://jmvalin.ca/papers/AV1_tools.pdf)

~~~
bscphil
That's interesting and matches my a priori expectations. That (effectively)
makes it a way of compressing grain, rather than just masking lower bitrates.
You have to store information about the analysis for the decoder to work, and
so there are the usual tradeoffs that come with compression.

~~~
theoh
I don't think it really counts as compression. It's like taking the statistics
of a text and generating a new one with a Markov chain.

In other words, it is complete nonsense, texture synthesis, like the way some
people go for printing a digital image on a fake "canvas" for a "textured, 3D
look"... or maybe one of those flickering flame light bulbs.

Can anybody here imagine an audio codec that added simulated hiss and crackles
to make it sound like you were listening to a vinyl record? That's what this
is.

~~~
tfha
Some artists will intentionally put scratching or crackling into their music
for atmosphere. Imagine an audio codec that eliminated all scratching and
crackling from music. Some people may not mind but many artists would hate it.

Film is the same. Directors choose the film type and grain patterns that match
the look and feel they want. A more grainy film feels darker and more serious.
Splotchiness contributes to an unrefined or maybe even psychotic feel.

You don't want your encode removing that. You want the atmosphere of the
visuals to match the original, because the director likely had that atmosphere
in mind and made other decisions around it.

~~~
theoh
Well, the problem is that the encoding process in AV1 does remove grain, only
to add fake synthetic grain back in. My objection is not to the idea of
encoding grain, it's with replacing it with fake synthetic grain.

It's kitschy, ersatz, whatever you want to call it. I used to spend a lot of
time with artist filmmakers (many of whom are very nostalgic about celluloid)
and there is no way in hell they would accept this solution. It wouldn't meet
their standards of authenticity. It also "fakes" a medium-specific property in
a way that is unprecedented in audio or visual coding. Their solution would be
to either code at high enough bitrate to capture the grain, or insist on
screening on physical celluloid.

That might sound extreme and unrealistic—it's what artist's film people are
like—and even a bit snobby. After all, a lot of the options out there for
adding film grain effects (or scratches etc.) to video are not intended for
use by professional filmmakers, depending on how you define professional.
There is definitely an element of snobbery to the statement that film grain
should never be faked, whether by compositing something onto your home video
or in a hidden way inside AV1.

I hope you can see that I respect the director's prerogative in choosing
grain, I just don't think this AV1 grain synthesis methodology is sound from
an aesthetic/authenticity point of view. It's digital faking of an
chemical/analog effect, which IMO makes it unavoidably kitsch for reasons to
do with old modernist ideas of "medium specificity".

~~~
tfha
I couldn't easily pull up any screenshots, but I've been a video encoder for
some years and I can tell you that psy-rd (the x264 option to add film grain)
does wonders in terms of grain fidelity.

The problem with matching the grain perfectly without the randomization is
that it drives the bitrate crazy. A movie with 99% fidelity grain and 0 psy-rd
will have a bitrate of 30mbps+.

If you use psy-rd though, you can get to 99% fidely grain at closer to 20mbps.
The two screenshots will be visually indistinguishable, even when you are
rapidly switching between the source and encode, even though you know that the
grain is being randomly generated you can't tell.

If I get a chance later today I'll drop some comparison screenshots for a film
encode that used a lot of psy-rd, I think the results will surprise you.

------
Veedrac
Truncated graphs are really misleading; I ended up doing a double-take. There
are more honest ways to compress the height of a graph if vertical space is at
a premium, like showing %savings.

~~~
justinjlynn
Or using log charts, if you're required to use given units.

------
nstart
I've seen a few mentions of patents in this discussion. Does anyone know how
close AV1 is to infringing on any patents or when we might find out? This is
the most recent article I found -
[http://www.streamingmedia.com/Articles/News/Online-Video-
New...](http://www.streamingmedia.com/Articles/News/Online-Video-News/AV1-Is-
Finally-Here-but-Intellectual-Property-Questions-Remain-124134.aspx) and it
seems that the market doesn't/didn't know enough to know if there are patent
infringements.

Yet this article from Mozilla seems like that from June the situation might
have changed now that a 1.0 stable spec is out. Given that inspection for
infringement has been part of the codec creation process it seems promising
that they will be safe. But patents have proven themselves to be funny things.

That said, have there been any updates since June from lawyers/other blogs/etc
about the patent situation yet?

------
ksec
I think Mozilla should advocate for patents free Software or get rid of
Software Patents completely.

The graph starts at 50%?

And they didn't mention in the Moscow State University test, AV1 was 300x to
1000x slower.

While I like competition from AV1, whatever they are doing in terms of
marketing really doesn't resonate with me.

~~~
semicolon_storm
> And they didn't mention in the Moscow State University test, AV1 was 300x to
> 1000x slower.

Reference implementations of codecs are always slower. AV1 hasn't had the
benefit of years of optimizations yet, so this slowness is to be expected.

~~~
okket
If the codec can not be implemented efficiently on existing battery powered
handheld devices, it is pretty useless. That is the only metric that counts.
Even if, it will take many years to a decade to spread, considering the
replacement rates of those devices.

~~~
aseipp
HEVC/H265 implementations were initially very poor also, but it doesn't mean
much now.

Plus, mobile devices will use embedded fixed-block ASICs for this, just like
they already do for every existing codec. All codecs are completely
impractical on mobile devices if you're basing your estimates off software
numbers, even _extremely_ well optimized software will kill your battery. So
it's basically meaningless to try and extrapolate from optimized numbers, much
less preliminary ones.

Besides, encoding AV1 is what's expensive, but mobile devices also do not
necessarily need AV1 encode support anyway. They will need decode support.
Slow encode support is going to be a problem for people creating and serving
video. But better compression rates from AV1 substantially help mobile
devices, too, by reducing bandwidth needs for providers. You're likely to see
major video content providers taking the hit of AV1 encoding, while users only
need to decode, and this will largely be how it is used.

~~~
threeseed
> but mobile devices also do not necessarily need AV1 encode

So people aren't using Snapchat, Instagram Stories, FaceTime, IGTV ?

Because pretty sure encoding is just as important as decoding.

~~~
aseipp
Whether or not they use it is a completely different matter from whether AV1
is useful for those scenarios.

Assuming a gargantuan encoding slowdown, even in the face of hardware-based
encoders on mobile devices (which, again: hardware decoding/encoding is the
fact of the matter on how it will happen), then there's no reason to support
AV1 encoders since it will hurt battery. You're better off with something
else, because users care more about their battery than compression ratios or
minor artifacts on their Insta videos.

Assuming no gargantuan slowdown, and devices are equipped with AV1 encoders
that perform admirably, then your problem is solved. Just use the hardware
encoder. (Personally, it's unclear to me how much of AV1 encode support is
fundamental slowness that can't be mitigated by hardware and implementation,
but I suspect a lot of this fear is tied up in early-stage numbers. I bet it
can go much faster, even if it's still slower overall.)

If you're a media provider like Netflix for example, though, the bandwidth
savings will utterly dwarf the cost of the more expensive encoding process.
They encode once and serve thousands of times, and they don't run Netflix (the
company) on old mobile phones. So they'll do it anyway, because they want
their users to use AV1. It saves them bandwidth on a high quality video, and
it saves users on their data plans and _their_ battery.

This isn't really very hard to think about. Either AV1 encoding is
fundamentally too inefficient for mobile, or it isn't. If it isn't, users can
still get fast decoders (that wasn't the bottleneck), since the bandwidth
savings are going to be highly desirable for everyone, basically, and they'll
be pushed for. If they can be efficient enough, then there's no problem.

~~~
threeseed
The problem is that reference implementations of AV1 are 100x slower than HEVC
at encoding. And so hardware decoding may ether be (a) impossible on current
devices or (b) likely to destroy battery life. The only option is some
breakthrough in the complexity of the encoding algorithm.

These issues unless solved relegates AV1 to being a Youtube and Netflix format
and pretty much irrelevant for everyone else.

~~~
1ris
I means that the reference implementation is slower. Nothing more.

Reference implementations are not about performance, they are documentation.
They are not supposed to be used for anything else but documentation.

Your conclusion is ridiculous. There is absolutely not reason assume that av1
is fundamentaly more power consuming that HEVC.

A tank uses more gasoline that a VW Beetle. This statement is true, and as
irrelevant as your statement.

~~~
ksec
>Reference implementations are not about performance, they are documentation.
They are not supposed to be used for anything else but documentation.

That is valid for all H.262 , H.263, H.264 and H.265, as well as the up coming
H.266 VVC. The current Av1 isn't a documentation. Nor it is a Reference
Implementation in the regards of all MEPG codec. The AV1 is built on a
working, professionally made libvpx encoder built on VP9.

>There is absolutely not reason assume that av1 is fundamentaly more power
consuming that HEVC.

It definitely will be more complex, both encoding and decoding. Its
fundamental complexity is much higher then HEVC, how far could they optimise
so they are close to 5 - 10x of HEVC is a different story.

------
orastor
I hope this takes over HEVC and x264 in the near future!

~~~
waivek
Is AV1 smaller?

~~~
matt4077
Yes, by about 50% and 30%, respectively. See the graph in the article.

------
LeoPanthera
Last time I checked (very unscientific test using Handbrake) the AV1 encoder
was something like 10x slower than the hevc encoder on the same system. Is
this likely to improve?

~~~
vortico
Yes, currently there is no hardware encoder/decoder for AV1, but there is for
h264 on many processors. This may change in the future if companies and users
adopt it.

~~~
bscphil
I might be mistaken, but I believe most high quality encodes in both h264 /
hevc are not done using dedicated hardware encoders built into processors or
graphics cards. While it's much faster there's usually some quality tradeoff.
Instead, I believe that newer processors have additional instruction sets that
enable much faster _software_ encoding on the same codecs. Someone correct me
if I'm wrong about this.

~~~
vortico
You're right, there are fewer h264 encoders than decoders in consumer
software, but still dozens.
[https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC_products_and_...](https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC_products_and_implementations#Encoding)
There are precisely 0 for AV1 so far (except some may have a subset of the
standard in FPGA by now).

------
cordite
What happened to webm, which apple doesn’t even support yet?

~~~
sparrc
You might be mixing up containers and codecs. webm is the container format
that most av1 video will be delivered in.

With Apple the problem was not so much webm, but the vp9 codec, which is the
codec most webm containers use (see youtube).

ie, codecs: av1, vp9 <> h.264, h.265 containers: webm <> mp4

------
shmerl
How is the progress of the encoder? To be able to use it in real time it must
be very efficient.

~~~
snvzz
Mozilla is working on a performance-optimized encoder, and has demonstrated
realtime encoding recently on a lots-of-cores cpu.

------
kuon
How does AV1 performs when encoding fast (like realtime for twitch)? Is it
possible? If yes, what is the gain over h264?

I really hope an open format will win "the war", but I can't see it work at
scale if it doesn't support streaming.

~~~
clouddrover
Last year Bitmovin demonstrated a live 1080p AV1 stream using 32 cores:

[https://bitmovin.com/constantly-evolving-video-landscape-
dis...](https://bitmovin.com/constantly-evolving-video-landscape-display-
ibc-2017/)

Twitch wants to use AV1 for their streaming. They want to use the switching
frame feature of AV1:

[https://www.youtube.com/watch?v=o5sJX6VA34o](https://www.youtube.com/watch?v=o5sJX6VA34o)

And the chroma from luma feature of AV1 works well on video game content:

[https://people.xiph.org/~xiphmont/demo/av1/demo1.shtml](https://people.xiph.org/~xiphmont/demo/av1/demo1.shtml)

[https://www.youtube.com/watch?v=yKEDf5-2sT4](https://www.youtube.com/watch?v=yKEDf5-2sT4)

~~~
kuon
That's really promising. I really hope we will see an open video format as
standard.

