
Introducing Dav1d: a new AV1 decoder - jbk
http://www.jbkempf.com/blog/post/2018/Introducing-dav1d
======
dzdt
I understand AV1 has attempted to avoid existing patents. Is there a consensus
that they succeeded? Or might the HEVC patent pool come after them still with
lawsuits?

~~~
bcheung
"Under patent rules adopted from the World Wide Web Consortium (W3C),
technology contributors license their AV1-connected patents to anyone,
anywhere, anytime based on reciprocity, i.e. as long as the user does not
engage in patent litigation."

[https://en.wikipedia.org/wiki/AV1#cite_note-
patent_license-2...](https://en.wikipedia.org/wiki/AV1#cite_note-
patent_license-23)

We can't know for certain, but with the number of big players joining the
alliance, I find it unlikely that a company would want to give up their rights
to use AV1.

Not to mention the PR nightmare, boycotting, and developer exodus that would
ensue for that company.

~~~
MaxBarraclough
> PR nightmare, boycotting, and developer exodus that would ensue

Perhaps, but I have less and less faith in these 'corrective forces'. Tech
companies often seem to discount such consequences, and they get away with it
just fine.

More optimistically though, I can't recall any instances of a major tech
company reneging on a promise like this. The 'Microsoft Community Promise' has
been honoured, for instance. (I accept it's quite possible I'm just being
forgetful, mind.)

------
nickik
Great, so awesome to see the AV1 ecosystem grow like that. This will make the
end of the old patent mafia.

One thing I would like to see is that this runs on RISC-V. Both are very
important new open technologies and it would be great to see it run on that
officially.

------
sp332
Is there an advantage to writing ASM without intrinsics?

~~~
rbultje
Yes. For intrinsics, the compiler chooses whether and when to spill variables
to the stack. For hand-written assembly, you are explicitly burdened with this
yourself, usually causing you to optimize the flow of the function so it
minimizes stack spilling. In addition, your function will perform the same
regardless of the compiler.

In FFmpeg, we have found in the past that functions written in pure assembly
are typically a few % faste than those written in intrinsics. Depending on
your use case, that may make it worth it.

The obvious disadvantage is that pure assembly is harder, so writing functions
takes a bit longer.

~~~
Const-me
> the compiler chooses whether and when to spill variables to the stack.

In my experience the behavior is stable i.e. doesn't often change between
rebuilds (at least in VC++). I can manage this by looking at compiler's
output: when I find there something I don't want, be it stack spilling or
other reasons for me not liking the assembly, I go fix my C++ code.

> so writing functions takes a bit longer

Also reading is hard, therefore refactoring/improvements is hard, too. I have
an impression the assembly is write only language.

~~~
Daemon404
> In my experience the behavior is stable i.e. doesn't often change between
> rebuilds (at least in VC++).

This is less true when you try and support many versions of many compilers,
however, in my experience.

~~~
Const-me
Yeah, that makes sense. I usually deliver results of my work in binaries so I
can choose the compiler, and inspect compiler output of performance critical
parts. Authors of that decoder didn’t have that luxury.

------
Const-me
I think if they want wider adoption of the codec, they have to port this to
GPU. They are much faster in term of throughput. They are more power efficient
per FLOP. Also for players, the result needs to be in VRAM anyway.

Unfortunately, it’s very hard to do in cross-platform way, for Windows you
have to use D3D, for iOS Metal, etc… AFAIK, cross-platform GPGPU doesn’t work
yet, at least when you want to target wide audience i.e. both Intel/Nvidia/AMD
GPUs including older models, with default drivers.

~~~
tormeh
Well, some people are working on this: [https://github.com/gfx-
rs/gfx/blob/master/README.md](https://github.com/gfx-
rs/gfx/blob/master/README.md)

I think at this point it's clear that we'll NEVER have a cross-platform
graphics API. The incentives for platform holders speak against it, and design
by committee means the attempts are so inferior that devs would rather avoid
them.

We have to make wrappers. Ideally, they should translate at compile time.

~~~
Const-me
> it's clear that we'll NEVER have a cross-platform graphics API.

I think we will, in a few years. I never used VK in production but I’ve read
about it a lot, and I like the technology.

Only I don’t think it happened just yet. On PCs Intel is problematic, VK
requires Skylake GPUs (2015), i.e. most PCs older than 2-3 years don’t support
it.

------
BeeOnRope
What does it mean that the ask is written in NASM/GAS syntax? Those are two
wildly different asm syntaxes with different macro languages etc.

Is some asm written in one variant and some in the other, or is it all written
both ways? Maybe for some targets GAS is used and others NASM (<\-- it's this
one as answered below)?

~~~
jbk
x86/x64 is Nasm.

ARM is gas.

~~~
BeeOnRope
Thanks, makes sense!

------
dallen33
Are the Alliance for Open Media not allowed to use Apple's logo/trademark?
Weird.

[https://aomedia.org/membership/members/](https://aomedia.org/membership/members/)

~~~
rgovostes
Even the LLVM Developers’ Meeting, which is/was organized by Apple, doesn’t
use the logo.

[https://llvm.org/devmtg/2018-10/#contact](https://llvm.org/devmtg/2018-10/#contact)

------
Retr0spectrum
> written in C99 (without VLAs)

Can someone fill me in on why VLAs are a non-feature?

~~~
ux
One of the main issue is no error checking on this kind of "allocation" (it's
the same as alloca()). This issue also somehow exists with a fixed size
buffer, but at least you get a much more predictable outcome (you have full
control on the stack usage, unless you have a user controlled recursion).

~~~
derf_
You can check for errors with alloca. It returns NULL on failure. You have the
same problem as with error checking malloc. On platforms with overcommit (any
fork-based UNIX, except maybe Solaris) it doesn't fail at allocation time, it
fails when you try to access the memory and fail to map a page. But at least
on 32-bit systems it can detect when you exhaust your address space (an actual
problem for things like web browsers that still commonly ship 32-bit binaries
and use lots and lots of threads).

But yeah, the real reason is that MSVC will never implement VLAs.

~~~
nwmcsween
How could alloca return NULL? You would have to catch bumping into a guard
page would you not?

~~~
codys
_theoretically_ a C implementation could track the maximum stack size it has
allocated, detect would be overflows, and report them.

guard pages are one mechanism for doing that.

------
kuroguro
> Therefore, the VideoLAN, VLC and FFmpeg communities have started to work on
> a new decoder

Isn't VideoLAN the same as VLC?

~~~
agumonkey
maybe VLC is just one project worked on by VideoLan people ?

~~~
kuroguro
Seems so, but they are listed as if they are different "communities". I wonder
if it's just bad wording.

~~~
adrianmonk
If there are people in the VideoLAN community who don't work on VLC, then they
are different communities. One might be a superset of the other, but they
aren't equivalent.

To me, it's not bad wording to mention communities that have a lot of overlap.
There's no rule that says you should only list things that are disjoint.

------
eridius
Why is this written in C? I would have thought Rust would be the perfect fit
for this.

~~~
devwastaken
afaik safe rust would not perform that well compared to unsafe. The main safe
part could be the command line or API but using Rust brings you into the Rust
ecosystem and it may just not be worth it to do if most of their code is going
to be unsafe and purposefully use performance hacks and SIMD.

~~~
steveklabnik
Safe Rust can outperform unsafe Rust. It really just depends.

------
ninjakeyboard
"AV1 has the potential to be up to 20% better than the HEVC codec"

Can you clarify what is meant by better here? More compressed?

~~~
rbultje
Bitrate reduction for the same quality. So a file of identical quality - in
some metric - will be 20% smaller (e.g. 8 MB instead of 10 MB for a video).

~~~
vbezhenar
How quality is measured? I never saw any quality estimations in software, e.g.
when I'm exporting JPEG image, I'd like to have some kind of result quality
estimation, it might be hard to judge it yourself.

~~~
vsl
Some function of the difference in pixels between the original and decoded
compressed version. This is a complicated subject, because a good error
function should account for human perception and weight unimportant
differences less.

------
dochtman
I'd be interested in comparisons to
[https://github.com/xiph/rav1e](https://github.com/xiph/rav1e).

~~~
jbk
Rav1e is an encoder. dav1d is a decoder. (Hence the last letter).

~~~
aerodog
Am I right to conclude that av1 takes like 100x+ the duration of the video to
encode? Like it's extremely impractical at the moment?

~~~
unlord
What you are referring to is the encoding time using the libaom reference
encoder, which was developed as a research code base to test bitstream
features. AV1 (the format) was designed to be used for a variety of use cases
including real-time and interactive streaming and it is possible to create a
non-libaom based encoder that does so (see the
[https://github.com/xiph/rav1e](https://github.com/xiph/rav1e) project).

And still, for some companies the bandwidth savings of AV1 are worth deploying
today even with the longer encode times of libaom. For example, you can try
AV1 streams on YouTube right now:

[https://www.youtube.com/testtube](https://www.youtube.com/testtube)

~~~
gajjanag
So here is a possibly naive question.

From my viewing of Youtube, Google transcodes uploaded videos (typically H.264
stuff) to VP9 only when a certain view threshold is reached, which makes sense
from their perspective.

However, I have also noticed that due to the chain of encodes source -> H.264
-> VP9 (latter two available to Google), the VP9 stream is often of noticeably
lower quality. Thus, whenever I can, I use an H.264 stream.

This problem will not go away with AV1. In fact, from an archival/local usage
standpoint, as others have noted here, AV1 is pretty much impractical due to
heavy encoding time increases that will unlikely go away with SIMD as compared
to x265 or x264.

As such, from an end user experience point of view, what does AV1 offer that
H.265/H.264 do not already?

~~~
ZeroGravitas
YouTube encodes are always done from the source. When new codecs like AV1 are
introduced they reencode from the originally uploaded source not from other
YouTube encodes.

The immediate benefit of AV1 will be felt by people with decent machines but
terrible bandwidth as the higher compression will give them higher quality.
There's talks given by YouTube employees about this process when VP9 initially
rolled out and how different parts of the globe benefitted depending on their
tech and infrastructure levels.

------
ggregoire
TIL what is a recursive acronym.

~~~
mattmar96
Have you heard of GNU?

~~~
cpeterso
I don't know what GNU is, but I know what it's _not_.

------
shakkhar
Why is this a stand-alone library and not a part of libavcodec? To avoid LGPL?

------
nagVenkat
Could someone confirm if dav1d uses hardware acceleration to decode?

~~~
mbebenita
If you're referring to SIMD, then the answer is not yet, very soon though.

------
voltagex_
Can anyone test this on a Raspberry Pi 3?

------
prablenha
>BSD is not copyleft, why?

Really? People are going to complain because a licence is _too free_? ...

~~~
laken
The complaint usually comes in due to an interpretation of "free."

The author includes RMS' opinion, which is true for this particular piece of
software, though in many cases it's a warranted complaint. It gets into a
political argument usually, but it boils down to this argument: "If I don't
require copyleft, then I can't always guarantee that users of my code will get
all the freedoms I provided to them."

~~~
class4behavior
Here in detail: Why Open Source misses the point of Free Software -
[https://www.gnu.org/philosophy/open-source-misses-the-
point....](https://www.gnu.org/philosophy/open-source-misses-the-
point.en.html)

~~~
groestl
Maybe I missed something, but this essay seems to deal with the relationship
of Open Source v Free Software, not Free Software v Copyleft Software.

~~~
class4behavior
You don't need an explicit essay on that comparison. Morever it's not vs
Copyleft, but why copyleft is generally material to the purpose of Free
Software.

In the linked entry Stallman primarily explains what is understood as Free
Software. The comparison with Open Source merely serves as a device to
illustrate the nuances in a more relatable way.

>Another misunderstanding of “open source” is the idea that it means “not
using the GNU GPL.” This tends to accompany another misunderstanding that
“free software” means “GPL-covered software.” These are both mistaken, since
the GNU GPL qualifies as an open source license and most of the open source
licenses qualify as free software licenses. There are many free software
licenses aside from the GNU GPL.

~~~
groestl
I find that distinction very much of interest, and sure enough, RMS also has
an explicit text available which deals with that matter more directly [0].

While free software protects the freedoms of the user only, copyleft software
protects the freedoms of the developer as well. Both types of software are
ethical (as mentioned in [1]), so it comes down to personal choice of the
original author, who can use it as a tool to ensure their goals.

[0]
[https://www.gnu.org/philosophy/pragmatic.html](https://www.gnu.org/philosophy/pragmatic.html)

[1] [https://www.gnu.org/philosophy/free-
sw.en.html](https://www.gnu.org/philosophy/free-sw.en.html)

