
The web at maximum FPS: How WebRender gets rid of jank - bpierre
https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-fps-how-webrender-gets-rid-of-jank/
======
pacaro
Humourously enough, when I worked on a team that was writing a graphical web
browser for mobile in the late 90's [1], they used a display list for
rendering.

The reasoning was somewhat different, web pages were essentially static (we
didn't do "DHTML"), if the page rendering process could generate an efficient
display list, then the page source could be discarded, and only the display
list needed to be held in memory, this rendering could then be pipelined with
reading the page over the network, so the entire page was never in memory.

Full Disclosure: while I later wrote significant components of this browser
(EcmaScript, WmlScript, SSL, WTLS, JPEG, PNG), the work I'm describing was
entirely done by other people!

[1] - I joined in 97, the first public demo was at GSM World Congress Feb 98

~~~
AceJohnny2
Opera?

~~~
pacaro
The company was STNC Ltd. The µbrowser was called Hitchhiker. STNC was
acquired by Microsoft in July 99.

My understanding is that the demo at GSM World in 98 was the first time anyone
had demonstrated a graphical browser on a cellphone that implemented common
web standards (HTML/HTTP/TCP) rather than using a transcoding gateway (up.link
or WAP style — the approach taken by Unwired Planet)

The smallest device that we targeted (IIRC) had a screen that was 100x64
pixels, we had 64KiB of RAM at runtime (`unsigned char gHeap[0x10000];`), and
our ROM footprint was about 350KiB

~~~
pavlov
I think back in 2000 I had a phone with your browser, the Benefon Q:

[https://m.gsmarena.com/benefon_q-42.php](https://m.gsmarena.com/benefon_q-42.php)

It even supported black&white animated GIFs in HTML pages.

~~~
pacaro
Yup! That was the most limited device we ported to. I wasn’t even aware that
it was officially shipped. Benefon made nice devices, the Q was really stylish
for its time. If memory served they were mostly sold in Eastern Europe and
Russia.

------
pohl
Now that this is closer to shipping, I'm curious what impact this would have
on battery life. On the one hand, this is lighting up more silicon; on the
other hand: a faster race to sleep, perhaps?

Have there been any measurements on what the end result is on a typical modern
laptop?

~~~
lwansbrough
I know it’s not officially released, so I’m hoping it gets fixed, but FF57
rips through my Mac’s battery life and runs insanely hot on simple JS apps. I
still use it daily because it generally works, but there are a few apps I just
have to go to Chrome for.

~~~
steveklabnik
Firefox 57 doesn’t have WebRender on unless you explicitly turn it on; did
you? It would be reasonale not to, as it’s still kinda buggy.

~~~
lwansbrough
Didn’t know that! I’ll be honest I was wondering what the hype was about
because it didn’t seem that fast to me...

~~~
kibwen
The new improvements to Firefox fall under a project called "Quantum". Firefox
57 is merely the first Firefox release to have any Quantum components enabled,
in particular Quantum CSS (a.k.a. Servo's CSS engine, Stylo) and Quantum Flow
(which is a general umbrella project for making Firefox UI more responsive).
Here are the others:
[https://wiki.mozilla.org/Quantum](https://wiki.mozilla.org/Quantum)

------
anon1253
Just tried it with the Nightly by setting gfx.webrender.enabled to true in
about:config. Wow, that thing flies. It's seriously amazing. And so far no
bugs or visual inconsistencies I could detect. Firefox is really making great
progress on this front!

~~~
Antrikshy
I really want to use Firefox full time, but I miss Safari's multi-touch
gesture integration.

~~~
DontGiveTwoFlux
You can try out Better Touch Tool to add a ton of custom gestures to whatever
mapping you want.

For example, I use a four finger swipe down to hit CMD-W, which closes the
window in focus for most apps.

[https://www.boastr.net/](https://www.boastr.net/)

------
jacob019
Love the little stick figure representations of the threads/cores.

~~~
Vinnl
That's to Lin Clark's credit.

~~~
mxstbr
She is great at breaking down complex concepts into digestible and easily
understandable pieces. Her whole Code Cartoons series is just plain awesome
([https://code-cartoons.com/@linclark](https://code-cartoons.com/@linclark))
and her conference talks are some of my all-time favorites!

------
vvanders
Good stuff.

Speaking of rendering text glyphs on the GPU, there's a really clever
trick(commonly called loop-blinn, from the two authors):
[https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch25....](https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch25.html)

You can pretty much just use the existing bezier control points from TTF as-is
which is really nice.

~~~
pcwalton
If only it were as simple as just using Loop-Blinn. :) The technique described
there will produce unacceptably bad antialiasing for body text. Loop-Blinn is
fine if you want fast rendering with medium quality antialiasing, though.
(Incidentally, it's better to just use supersampling or MLAA-style
antialiasing with Loop-Blinn and not try to do the fancy shader-based AA
described in that article.)

Additionally, the original Loop-Blinn technique uses a constrained Delaunay
triangulation to produce the mesh, which is too expensive (O(n^3) IIRC) to
compute in real time. You need a faster technique, which is really tricky
because it has to preserve curves (splitting when convex hulls intersect) and
deal with self-intersection. Most of the work in Pathfinder 2 has gone into
optimizing this step. In practice people usually use the stencil buffer to
compute the fill rule, which hurts performance as it effectively computes the
winding number from scratch for each pixel.

The good news is that it's quite possible to render glyphs quickly and with
excellent antialiasing on the GPU using other techniques. There's lots of
miscellaneous engineering work to do, but I'm pretty confident in Pathfinder's
approach these days.

~~~
mrec
I hadn't seen Pathfinder before; very cool! Especially if it's easily
transferable to gpuweb, if and when that ships.

Reminds me of Mark Kilgard's old NV_path_rendering extension, which (as so
often happened for interesting NV stuff) never made the jump to portability.
One of its touted benefits as an in-driver implementation was the ability to
share things like glyph caches between multiple apps with separate GL
contexts, but with "apps" increasingly becoming "browser tabs", maybe a
browser-managed cross-tab cache is almost as good.

BTW, is the WebGL demo available online anywhere, for people who don't want to
install npm?

~~~
pcwalton
> BTW, is the WebGL demo available online anywhere, for people who don't want
> to install npm?

Not yet. There are enough known issues that I'd like to fix first.

------
kidfiji
I just love how they simplify something that's seemingly esoteric to me. I
work with the web and I still have a lot to learn.

~~~
winter_blue
A lot of credit for that goes towards Lin Clark, who's done some amazing
expository articles like this before.

------
bsimpson
Great write-up - thanks for sharing.

The name "WebRender" is unfortunate though. Things with a "Web" prefix - "Web
Animations", "WebAssembly", "WebVR" \- are typically cross-browser standards.
This is just a new approach Firefox is using for rendering. It doesn't appear
to be part of any standard.

~~~
482794793792894
I remember reading at some point that WebRender could actually be isolated
relatively easily and then applied to basically any browser. That sort of
already took place, going from Servo over into Gecko.

So, it might actually turn into somewhat of a pseudo-standard.

~~~
metajack
You send display lists to it over IPC, and it's easy enough to generate those
from C++. It has a well defined API boundary, which makes it easy to use.

This is in stark contrast to the style engine in Servo which relies on memory
representation to be fast. So integrating it requires very tight coupling of
data structures.

~~~
kibwen
Is it in its own process for improved security? Specifically, are buggy
graphics drivers the concern?

~~~
bzbarsky
Buggy graphics drivers are definitely a concern, not just for security but for
stability. Specifically, graphics drivers crash. A lot. Simply moving GPU
access to a separate process (which is common in modern browsers) means that a
driver crash doesn't bring down any webpages, much less the whole browser; if
done right you just get a flicker as the GPU process is restarted.

------
djhworld
This is fantastic (for me)

I'd largely forgotten what pixel shaders actually were, so it was nice to get
a high level understanding through this article, especially with the drawings!

------
frostwhale
I was already extremely pleased with the Firefox Quantum beta, they really are
stepping their game up. If this is truly as clean as they say it is, web
browsing on cheap computers just got much smoother.

------
stevenhubertron
I really appreciate the time they are taking to describe the changes in an
easy to understand way. The sketches and graphics really help explain a pretty
complex subject.

------
shmerl
Is it going to use Vulkan? Sounds like a good fit for proper parallelized
rendering.

UPDATE: Ah, I see it's mentioned in the future work:
[https://github.com/servo/webrender/wiki#future-
work](https://github.com/servo/webrender/wiki#future-work)

    
    
        Vulkan?
        This could possibly make some of the serial
        steps above able to be parallelized further.
    

So it will be using OpenGL then?

~~~
482794793792894
Vulkan has been a consideration from the earliest architecturing steps done in
WebRender. So, the internal pipelines are all set up to be mapped to Vulkan's
pipelines.

It's actually OpenGL which fits less into the architecture, but it's still
easier to just bundle WebRender's pipelines all together and then throw that
into OpenGL.

~~~
shmerl
Interesting. Will it eventually use vulkano library, or its own Vulkan
bindings?

------
kevindqc
>For a typical desktop PC, you want to have 100 draw calls or fewer per frame

Don't PC games use thousands of draw calls per frame?

~~~
pcwalton
They do, but we're targeting Intel HD quality graphics, not gaming-oriented
NVIDIA and AMD GPUs.

That said, even Intel GPUs can often deal with large numbers of draw calls
just fine. It's mobile where they become a real issue.

Aggressive batching is still important to take maximum advantage of
parallelism. If you're switching shaders for every rect you draw, then you
frequently lose to the CPU.

~~~
SomeCallMeTim
Speaking of batching, I look at this demo [1] on my mobile phone, and I can
get ~3500 sprites without dropping below 60FPS.

A web page may not be able to reuse as much image data, I know. But smart game
engines frequently look for ways to better batch sprites.

And technically speaking, if you're using a Z-buffer, you don't need to sort
opaque layers _at all_. You can draw the layers in back and then draw more
layers in front. Yes, you get overdraw in that case, but if you're using Z
values for layering, you could potentially get better batches by drawing in
arbitrary order (i.e., relying on Z-depth to enforce opaque object order), and
in my experience larger batches gives you a bigger advantage than reducing
overdraw.

[1]
[http://www.goodboydigital.com/pixijs/bunnymark/](http://www.goodboydigital.com/pixijs/bunnymark/)

~~~
markdog12
Interestingly, WebRender renders that demo at about 2fps. It's obviously a bug
though, if you hold in the mouse, it's 60fps.

Win 10, latest nightly, WebRender enabled

~~~
steveklabnik
Weird; I'm on Win 10 with the latest nightly and WebRender enabled, I'm still
getting 60fps with 10,000 bunnies. Maybe file a bug?

~~~
markdog12
Interestingly enough, I'm getting same result on Mac OS. Unless I hold in the
mouse, I get about 2 fps. Seems like the same result on all canvas instances.
I'll file a bug.

Edit:
[https://bugzilla.mozilla.org/show_bug.cgi?id=1407448](https://bugzilla.mozilla.org/show_bug.cgi?id=1407448)

------
amelius
> What if we stopped trying to guess what layers we need? What if we removed
> this boundary between painting and compositing and just went back to
> painting every pixel on every frame?

This feels a bit like cheating. Not all devices have a GPU. Would Firefox be
slow on those devices?

Also, pages can become arbitrarily complicated. This means that an approach
where compositing is used can still be faster in certain circumstances.

~~~
aneutron
Actually, virtually every device the average grade consumer uses, has a GPU.
For instance, even Atom processors have GPUs. Granted, they don't have as much
cores as a full-fledged nVidia GPU, nor as much dedicated memory, but they are
still GPU with several tens of cores and specialized APIs that were designed
specifically for the tasks at hand. Plus, they offload (ish) the CPU.

~~~
btrask
Something without a GPU: VirtualBox. Last time I tried a Servo nightly in
VirtualBox (to be fair, a few months ago), it immediately crashed/aborted with
a GPU-related error.

I think there's a good argument for preventing security-critical apps from raw
GPU access, because graphics cards and drivers are a huge amount of attack
surface.

I still think WebRender is the way forward, but I hope they get it working
with something like llvmpipe.

~~~
aneutron
While I understand the concern about security, I think it is beside the point.
The surface attack is effectively the GPU and its driver, not the web
renderer. The technique that are put in place by the web renderer are and were
used by game engines.

If anything it'll push the GPU makers to have better drivers support.

EDIT: As for the no-GPU case, it is an edge-case, in which we could for
example switch to the classical renderer. If you're running FF from a VM as
your daily driver, there's something not right somewhere I think. (I'm
thinking of C&C servers for satellites still running on WinXP and stuff)

~~~
btrask
> If you're running FF from a VM as your daily driver, there's something not
> right somewhere I think.

I am. QubesOS is a similar setup that runs applications in Xen VMs for
security. I think you can do GPU passthrough but it's not recommended. At this
point I don't think I would ever go back to running a browser "bare", even if
there were no bugs, being able to have complete control over it (e.g. pausing,
blocking network access, etc.) is a godsend.

I think the idea that "it'll push GPU makers to be secure" is weak. We're so
far from that point it's not even on the horizon.

------
fritzy
I imagine the render task tree also has to determine which intermediate
textures to keep in the texture cache, and which ones will likely need to be
redone in the next frame. That kind of optimization has to be tricky.

~~~
pcwalton
Yeah, it's one of the two hard problems in computer science after all :)

In practice LRU caches work pretty well.

------
azinman2
Won’t this cause a lot of work to be done for a blinking cursor? Curious about
battery drain, I/O overhead, General CPU usage, etc.

~~~
pcwalton
With a compositor you're already drawing every pixel every frame on the GPU,
whether it's just a cursor blinking or not. The WR approach basically only
adds a negligible amount of vertex shading time.

~~~
randyrand
It seems like even in the compositor approach, you could optimize that. copy
the last last drawn frame buffer directly and then modify it.

I think the best approach would be some hybrid between invalidation and
redrawing everything.

~~~
lightedman
IIRC GPUs already do a more efficient version of what you're proposing, at
least the mobile ones, by using tile-based rendering. Thus only the general
area of the screen that changes gets modified while the remainder is static.
This way you can easily control how many GPU cores are actively changing and
thus need power, while you keep the static state in the other cores and they
consume negligible power.

------
poizan42
I tried testing it out on a ThinkPad T61 to see how well it works with an
older embedded GPU (Intel 965 Express), but I can't enable it (on Windows 10)
because D3D11 compositing is disabled, it says D3D11_COMPOSITING: Blocklisted;
failure code BLOCKLIST_

So does that mean that it is known not to work with that GPU? Can you override
the blocklist to see what happens?

Edit: It also says:

> Direct2D: Blocked for your graphics driver version mismatch between registry
> and DLL.

and

> CP+[GFX1-]: Mismatched driver versions between the registry 8.15.10.2697 and
> DLL(s) 8.14.10.2697, reported.

Indeed that is correct, the driver is marked as version 8.15.10.2697 but the
fileversion of the dlls are 8.14.10.2697, this seems to be intentional by
Microsoft or Intel, note that the build numbers are still the same. Firefox is
quite naive if it thinks it can just try to match those.

~~~
MrRadar
Take it from a Thinkpad X60 owner, the Intel GPUs from that era are absolute
_trash_. They don't support OpenGL 3.0 on any platform (in fact, they didn't
gain 3.0+ support until Sandy Bridge in 2011(!)) so don't expect any of this
recent GPU-centric work (which seems to be targeting OpenGL 3.0+) to work on
these GPUs. It would probably work just fine on the contemporary Radeon and
GeForce cards since they support OpenGL 3.3.

------
JepZ
While I would consider myself more a Golang fan than a Rust fan, I am
impressed by the speed by which the Mozilla team is changing fundamental parts
of their browser and somehow I believe rust has something to do with that
speed.

~~~
pimeys
I've been working professionally with Rust for a year now. When I got over the
first wall, it has become the best tool I've had for creating backend
applications. I have history with at least nine different languages during my
professional career, but nothing comes close giving the confidence and
ergonomics than the tools Rust ecosystem provides.

Firefox, especially the new Quantum version is awesome. But Rust as a side
product might be the best thing Mozilla brought us. I'm truly thankful for
that.

~~~
fb03
nice! can you talk more about Rust's tooling? How do you debug? What IDE or
text editor do you feel comfy with? Do you have/use/need autocomplete?

I'm coming from a a Python background and I'm too spoiled by Pycharm's insane
tooling and autocomplete and stuff.. I wanna trail de Rust path too!

thanks in advance.

~~~
mrsteveman1
> nice! can you talk more about Rust's tooling? How do you debug? What IDE or
> text editor do you feel comfy with? Do you have/use/need autocomplete?

For writing code, I've been using IntelliJ with the Rust plugin for the past
year.

Autocomplete works pretty well, there is Cargo integration so you can edit
your dependency file in IntelliJ and things will automatically update so that
autocomplete and documentation (go to definition) for the newly added
dependencies start working immediately.

There is also simple build/check/run support, with build and test output
appearing in the lower panel inside the IDE. There are some quirks, but they
have been very minimal.

I've never used IntelliJ to _debug_ Rust itself, because in most of the
projects I've worked on, the Rust code has been designed as a completely
separate library with a semi-public API, intended for embedding in another
language like Swift or C#.

So the debugger has almost always been Visual Studio or Xcode, but it works
exactly as I would expect even without using any sort of Rust plugin for
either one.

The other day I was stepping through the call stack to find the source of a
crash (one that I caused by failing to keep the P/Invoke signatures up to date
with our Rust functions), and Visual Studio switched right from C# to Rust
from one frame to the next, showing the source code and the line where the
crash occurred in Rust.

Really I can't think of anything that has been even a moderately significant
problem, it's been great all around.

------
markdog12
Is WebRender working on Android Firefox Nightly yet?

Update: about:support says not ready for Android

~~~
aneutron
On nightly, the flag is available in about:config and I alreay enabled it.

~~~
markdog12
Yeah, but check about:support, it'll say not ready

------
esaym
Will there finally be a unified use of the GPU on all platforms (win, mac,
linux, etc) or will WebRender just be a Windows only feature for quite some
time?

~~~
hexane360
I have WebRender working on Linux with Intel 5500 integrated graphics.
Hardware acceleration is still a bit glitchy though I'm afraid (with or
without WebRender).

To enable, toggle 'layers.acceleration.force-enabled' as well as
'gfx.webrender.enabled'

edit: It's also working through my Nvidia 950m (through bumblebee), although
subjectively it seems to have a little more lag this way.

~~~
esaym
Well that stinks, I'm on Intel HD 3000

~~~
hexane360
I'm not saying you need at least a 5500, I'm just saying it works for me and
that's what I have.

------
madez
Why are they so obsessed with 60 fps? 120 fps looks considerably better, and
there are other effects like smear and judder that significantly decrease even
with significantly higher frame rates, say 480 fps [1].

[1] [http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-
hol...](http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-hole-fixing-
judder/)

~~~
kibwen
The WebRender folks are well aware that higher framerates are the future.
Here's a tweet from Jack Moffitt today, a Servo engineer (and Servo's
technical lead, I believe):
[https://twitter.com/metajack/status/917784559143522306](https://twitter.com/metajack/status/917784559143522306)

 _" People talk about 60fps like it's the end game, but VR needs 90fps, and
Apple is at 120. Resolution also increasing. GPUs are the only way. Servo
can't just speed up today's web for today's machines. We have to build
scalable solutions that can solve tomorrow's problems."_

------
ttflee
So, did it mean that Mozilla has implemented Core Animation counterparts in
each and every platform they supported?

------
lkbm
This is a great explanation. I also recommend Metajack's talk on WebRender in
Servo:
[https://www.youtube.com/watch?v=an5abNFba4Q](https://www.youtube.com/watch?v=an5abNFba4Q)

Also go read everything else Lin Clark has done. They're pretty great.

------
kadal
Re: HW Acceleration

Is this going to work at all on Linux?

~~~
metajack
We specifically target integrated GPUs, and several of the main developers
work on linux using Intel GPUs with open source drivers. There's every reason
to think this will be amazing on Linux.

~~~
alex_duf
This is making me happy! Thanks!

------
lokimedes
Nicely done article, but please please stop using analogous terms like
quantum, rocket, atoms, jet, etc... These terms actually means something in
the real world. Working in the limbo between web dev and science exacerbates
how silly this is. Find your own terms please.

------
markdog12
Just tried it on iMac 27" late 2015, High Sierra, latest nightly with
WebRender enabled. Went to [https://codepen.io](https://codepen.io). Froze the
screen for 30-40 seconds, then just black. Had to hard reset.

------
ngokevin
I don't think it would help WebVR (maybe in the future rendering DOM as
textures, but it'd be limited to only 2D), but today WebVR applications goes
through a different pipeline, running purely on WebGL.

~~~
mncharity
Some of us are running alternate WebVR stacks. :) On a Vive, playing with the
3D demo, it looks like I'm getting ~50 minimally-readable lines of text, with
no motion, and no lens correction (integrated graphics). Should be ~80ish with
corrected chromatic aberration. Which is pretty good for the low-angular-
resolution and PenTile Vive. It's comparable to the usual hand-tuned pixel
fonts rendered flat (part of that is PenTile pain).

------
Ygg2
Interestingly enough I tried this on Windows 10 and it seems to hide some
extensions and more importantly my min/max/close button.

Also fonts look slightly different, like I am browsing through using my Linux
machine.

------
OddMerlin
Great write up.

------
Tehnix
Anyone know if they used any specific tools for the graphs/figures? They
remind me of XKCD style graphs, really like those for technical yet informal
explanations!

~~~
hsivonen
Wacom Cintiq using Pixelmator according to
[https://mobile.twitter.com/linclark/status/86746972399853977...](https://mobile.twitter.com/linclark/status/867469723998539777)

------
pfedigan
Great Explanation! I wonder how much this will impact my gaming experience
while browsing websites.

------
ino
Any thoughts on weather fill rate could become an issue?

~~~
Rusky
It shouldn't- they're not doing anything crazy in the pixel shaders, and they
make excellent use of early Z culling.

------
hasenj
This is great! I'm really impressed by the technical achievement, and hope
everyone takes lessons from it to put performance and FPS at the forefront of
their concerns when developing applications.

Sadly, seeing the state of the industry, people will just use this as an
excuse to continue write more and more sloppy code that would perform terrible
even on the newest and faster WebRender version.

In other words, this version might run current websites at 60 fps. But wait a
few years, and it will become the norm that a lot of websites render at 10fps
or less.

There should also be a way to _punish_ developers who fail to run their sites
at 60fps on WebRender, similar to how browsers will start to punish sites that
run without https.

For example, if a site fails to run at 60fps for a few seconds, show some kind
of alarm on the address bar that this site is very slow and might crash the
browser.

~~~
critiqjo
No, not "crash the browser" (coz browsers are not supposed to crash no matter
the site's content), but that "this site might drain your battery quickly" or
something along those lines.

