
Firefox 80 and my confusion over its hardware accelerated video on Linux - eitau_1
https://utcc.utoronto.ca/~cks/space/blog/web/Firefox80VideoAccelConfusion
======
jml7c5
What Firefox really needs is a little icon (preferably displayed prominently
in the toolbar) that appears when videos are not being decoded in hardware.
I'm sure they've lost users who just think Firefox is "super laggy with
YouTube" or that "Firefox really eats battery". An indication that something
is wrong and the browser knows about it prevents the assumption that the
browser is just buggy/slow/etc; the word-of-mouth changes from "Firefox sucks"
to "Firefox told me it couldn't hardware accelerate my videos or something?"
Still not positive, but potential users will hear that Firefox might not work
with their hardware, rather than that Firefox is buggy, slow, and unreliable.

And I know technical users would appreciate it. I _still_ don't know what is
and isn't getting hardware video decode, or whether I actually need a
particular about:config flag or system package, and I have tried hard to
figure out.

~~~
stefan_
The default assumption today and in the past on Linux is that there is zero
hardware accelerated video display in any browser.

~~~
pjmlp
Which is why macOS, Windows and ChromeOS are the platforms for anyone who care
about video hardware acceleration without messing with configurations.

~~~
diamondlovesyou
ChromeOS is Linux.

~~~
klardotsh
I mean, sure, at a kernel level. It has its own GUI rendering stack (Freon)
rather than the usual X11/Wayland of "normal" Linux, and it's Freon that has
their acceleration support.

------
ronjouch
Yes, the UX is raw for now; hopefully Mozilla will make it simpler to enable
after a few versions of testing and confirming it doesn't cause crashes for
half the linux userbase. It makes sense to do little _early_ advertising of
such a feature, and to initially hide it behind flags & env. vars only within
reach of "power users", because these fresh-out-of-the-oven bits are a complex
assembly with heavy crash potential, depending on your hardware and drivers
up-to-date-ness. As the implementer, you want your initial users to know what
they are doing, and you want to maximize the chances they're technically
proficient enough to gather the mountain of gpu/driver/environment debug info
you'll need for a bug report to be actionable.

In the meantime, to save you rummaging around bugzilla, follow
[https://wiki.archlinux.org/index.php/Firefox#Hardware_video_...](https://wiki.archlinux.org/index.php/Firefox#Hardware_video_acceleration)
, it's the most constantly {up-to-date, concise, reliable} source you'll find.
You don't have to be using Arch, the information contained in this section
applies to any Linux/Firefox distro.

Note that Firefox isn't the only browser to struggle with it; see
[https://wiki.archlinux.org/index.php/Chromium#Hardware_video...](https://wiki.archlinux.org/index.php/Chromium#Hardware_video_acceleration)
for the similarly non-obvious instructions in Chromium (and by the way,
currently it's plain impossible in Chrome).

Props to Martin Stransky from RedHat who implemented this for both Wayland and
Xorg during the last months.

~~~
test7777
do what chrome does, detect crash and fallback. what are sandboxes even for if
you can't crash once in a while without whining too much about it...

~~~
ronjouch
I doubt things are that simple.

See my before-last paragraph above: no, regarding hardware _video_
acceleration, even Chrome doesn't do what you describe. What you describe
stands for non-video content crashes, which isn't what's being discussed here.
Could the technique you mention be applied to _video_ gpu crashes? I
personally don't know and I'd lean towards "nope" (else Mozilla & Google
engineers wouldn't be so exceedingly cautious about it). If anyone is
knowledgeable enough to precise / contradict this, please chime in.

At any rate, even Chrome is built with _zero_ hardware video acceleration
support on Linux (hardware acceleration yes, hardware video acceleration no).
Chromium does offer the option, just as hidden behind flags as Firefox, and
until last month you had to build it yourself with a VAAPI patch left unmerged
for years by Chromium devs.

------
est31
IDK why this post is so negative. The post says it's the first release with
that feature, and that things are still improving. I'm happy that it's being
worked on at all and maybe a couple more releases down the road it's available
to me as well. Mozilla/Firefox contributors are adding a feature here in the
open but it turn out it's very complicated to add it.

~~~
pstrateman
Hardware accelerated video decoding is a feature that is and has been critical
for battery life on laptops for years.

ffmpeg has supported this for forever and yet firefox doesn't. It's bizarre.

~~~
heftig
Do you believe enabling hardware video decoding in Firefox was a trivial
effort?

~~~
vetinari
What Firefox 78-80 brought was hardware video decoding under Linux, not
hardware video decoding in general.

Under Windows and MacOS, it has been implemented years ago. Under Linux, it
wasn't. Linux has API for video acceleration and players like VLC, mpv or Kodi
have been supporting it for years, as well as ffmpeg, just browsers ignored
it.

I'm not going to belittle the effort, quite opposite, but it is telling when
this feature has been implemented by Redhat employee and not Mozilla.

------
boring_twenties
I've never fully understood just what exactly the problem is?

Many apps ranging from vlc to Totem seem to hardware accelerate just fine, all
without seeming to know or care which specific hardware I'm using.

What makes it so much more difficult for FF?

~~~
Const-me
I think legacy contributed most.

On modern Linux, it’s possible, and not even terribly hard, to implement
hardware-accelerated video playback without any libraries or frameworks, by
directly consuming V4L2 API exposed by the kernel. Here’s a proof, I did it
alone spending a month or so part time, and resource use is lower than VLC’s
despite my code is in .NET: [https://github.com/Const-
me/Vrmac/tree/master/VrmacVideo](https://github.com/Const-
me/Vrmac/tree/master/VrmacVideo)

However, that wasn’t always the case. Until recently software had to fallback
to software decoders, and implement other workarounds. Software developers are
reluctant to throw away old code and do something completely different.

1\. They spent much time on that old code (sunk cost fallacy).

2\. Old code usually works more often than it doesn’t (don’t fix things which
aren’t broken).

3\. Massive refactor of large software is very expensive in general. Moving
stuff from CPU to GPU is going to cause large changes in many parts. You can’t
usually afford to copy uncompressed video frames from VRAM back to system RAM
and call it a day, that would be too much bandwidth. Instead, you have to
change a lot of other code to handle GPU textures instead of system RAM
images.

~~~
Jasper_
> On modern Linux, it’s possible, and not even terribly hard, to implement
> hardware-accelerated video playback without any libraries or frameworks, by
> directly consuming V4L2 API exposed by the kernel

Do you mean V4L2 M2M decoding? That's only supported on a very small subset of
devices, mostly ARM devices. Other devices might use VA-API (Intel), VDPAU
(NVIDIA proprietary), or something from the OpenMAX standard (some AMD
drivers).

M2M is far from easy to handle in a lot of circumstances, though it depends on
the kinds of devices you use. I don't really know why you're going into a
screed about "sunk cost fallacy".

~~~
vetinari
AMD uses VA-API nowadays.

~~~
selectodude
As does Nvidia.

------
kevinoid
I ran into similar issues with driver blocklists on both Linux and Windows.
There are workarounds, such as setting MOZ_GFX_SPOOF_* environment variables,
but even the suggestions on the Mozilla Wiki at
[https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drive...](https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers#How_to_force-
enable_blocked_graphics_features) no longer work, because the spoofed driver
version (8.15.10.2302) was blocked in
[https://bugzilla.mozilla.org/843273](https://bugzilla.mozilla.org/843273)
It's a mess.

I can understand the desire to avoid using drivers which cause a poor user
experience. I don't envy the task of trying to implement and maintain a system
to achieve that. I do wish it wasn't so painful and confusing for users when
the behavior doesn't match our expectations.

------
verroq
This is just one of the times where you’d just have to read the source. No
documentation usually means I start diving into the source code.

~~~
tpmx
That attitude/capability is closely related to a job interview question I tend
to use. "Could you tell me about a time you delved into the source code of
some software product or library to fix a problem you faced?"

(My pet theory: I think often it's just the attitude that is missing. So much
can be accompished with the right attitude.)

~~~
missblit
It's impossible to get anything done otherwise, and some cursory investigation
is usually a lot faster than emailing some overworked maintainer with nothing
but "plz help".

------
bleepblorp
It's not possible go get VA-API to work in FF 88 on X11 or Wayland because VA-
API was broken in Firefox 79. The fix is in FF 81.

~~~
usr1106
Not sure I follow, did you possibly mean:

It's not possible to get VA-API to work in FF 80 on X11 or Wayland because VA-
API has been broken since FF 79. The fix will be in FF 81.

Just guessing, I have not looked into the topic except in this thread. What
would be the bugzilla id for that?

~~~
bleepblorp
Replace the '88' with '80'.

VA-API hardware video acceleration was broken in Firefox 79 and won't be fixed
until Firefox 81. Therefore, it's not possible to make VA-API work, under
either Wayland or X11, in Firefox 80.

See:
[https://bugzilla.mozilla.org/show_bug.cgi?id=1656436](https://bugzilla.mozilla.org/show_bug.cgi?id=1656436)

~~~
usr1106
So to put it cynically: In FF 80 the situation is crystal clear, not at all
confusing as the original article claims: It doesn't work.

In FF 81 it will be confusing to the non-experienced again: You need to fiddle
with config and environment variables. Well, unless a new bug comes up...

------
zanny
I did manage to get it working but it has a ton of issues with Youtube vp9
decoding despite Chromium's years-old implementation working fine.

This is complex stuff. I spent a day last week trying to figure out why hevc
vaapi patches for OBS were failing with mkv containers but not mp4 ones. Its
just really ugly C apis all the way down.

------
bengalister
For me, on Arch LTS/wayland/gnome Firefox's video hardware acceleration became
buggy since Firefoxy 79 (I followed the Arch wiki recommendations). I had to
disable the gfx renderer. And cpu consumption is only marginally better than
Chromium. When enabled, cpu consumption is very low <10% watching 720p YT
videos, between 10-20% for 1080p but the video player crashes very often.

------
usr1106
What would be a poor man's reliable test to see whether acceleration is
actually in use or not? If you could just reliably switch it on and off the
difference should be obvious in top, possibly by switching to threads view by
pressing H (instead of the processes default). But if you cannot, there is
nothing to compare.

~~~
sp332
Go to about:support and look in the Graphics section.

~~~
usr1106
not at my computer now, but if I remember right, that shows you whether HW
acceleration could in theory work for some video or not at all. The second
case is a clear one. But in the first case you still don't know whether it is
actually used for a certain video or not. I understand there can be many
parameters preventing usage.

------
boomboomsubban
From what I can gather, they're using an Intel card, which I don't think
supports VA-API on MPEG-4 videos. Further, I can't tell which distro they use
but I'm not sure when the Intel work was added. Their distro provided version
would likely iron out some of these issues.

~~~
vetinari
That Intel 8700K definitely supports VA-API with H.264, HEVC and VP9, decoding
and encoding. An extra driver is (intel-vaapi-driver) is needed, it is not
distributed and installed together with Mesa as AMD driver is.

~~~
boomboomsubban
>That Intel 8700K definitely supports VA-API with H.264, HEVC and VP9,
decoding and encoding

It does not support MPEG-4, or it's disabled by default. I don't know how
widespread that format is, but I believe it was developed with streaming in
mind.

~~~
vetinari
If you mean MPEG 4 ASP (i.e. part 2, as opposed to H.264, which is also
MPEG-4, but part 10) I don't believe any Intel hardware did ever support it.
Today, it is obsolete.

------
pjmlp
Yeah, since the anti-Flash movement has won, Linux is even less interesting
for anyone wanting an out of the box experience regarding video hardware
acceleration for Web browsers.

