
Don’t use JPEG-XR on the Web - robin_reala
https://calendar.perfplanet.com/2018/dont-use-jpeg-xr-on-the-web/
======
PaulHoule
From time to time I've run an analysis on the use of alternative image formats
on the web and it never comes out to be worth it.

New formats are not supported on all browsers so you wind up having to support
the mainstream formats as well as one or more new formats. Rather than
benefiting from reduced storage costs, your storage costs get multiplied.

Performance is also a problem, particularly when people add "yet another
polyfill" to get an image to work without native support.

Once I get into eyeballing images closely I also end up questioning if the
compression gains are real. For instance, one I read a comment on the article
here

[https://calendar.perfplanet.com/2018/is-avif-the-future-
of-i...](https://calendar.perfplanet.com/2018/is-avif-the-future-of-images-on-
the-web/)

and looked closely I couldn't unsee that the AV1 had very different artifacts
than the other compressed images. Not necessarily worse, but definitely
different. Unlike the poster I liked the way the suit rendered but definitely
there was a lot of blocking around edges that the poster interprets as
"pixelation on the flag".

Like many things, the costs of switching image formats are definite, but the
benefits are questionable.

~~~
cosmie
> From time to time I've run an analysis on the use of alternative image
> formats on the web and it never comes out to be worth it.

I just checked the Google homepage, and even the New Year's Eve Google Doodle
currently being shown is a good ol' GIF, rather than a modern format such as
WebP. And this is with me using a desktop Chrome browser that definitively
supports WebP.

> Like many things, the costs of switching image formats are definite, but the
> benefits are questionable.

One easy way to get some of the benefits without incurring the switching costs
is to leverage Cloudflare Polish[1]. I've used it at a few places, and have
had good experiences. Although the majority of gains came from optimizing
images that were poorly optimized to begin with, rather than the incremental
improvements that occurred from enabling WebP support.

[1] [https://support.cloudflare.com/hc/en-
us/articles/36000060737...](https://support.cloudflare.com/hc/en-
us/articles/360000607372-Using-Polish-to-compress-images-on-Cloudflare)

~~~
maxst
> I just checked the Google homepage, and even the New Year's Eve Google
> Doodle currently being shown is a good ol' GIF, rather than a modern format
> such as WebP. And this is with me using a desktop Chrome browser that
> definitively supports WebP.

new-years-eve-2018-4995722058399744.2-law.gif

299 972 bytes

new-years-eve-2018-4995722058399744.2-law.webp

293 636 bytes

new-years-eve-2018-4995722058399744.2-law.apng

251 516 bytes

Interesting...

~~~
masklinn
Meanwhile using ffmpeg's default settings it's 63k as h.264 and 45k as h.265.

~~~
londons_explore
Starting a video in most browsers is pretty expensive (it usually loads up
various GPU features for hardware video decoding).

On some old and buggy machines, that might cause the browser to crash too.

Not something Google wants to risk for a doodle - that's why all non-trivial
doodles require a click to load.

~~~
masklinn
> Starting a video in most browsers is pretty expensive (it usually loads up
> various GPU features for hardware video decoding).

OTOH it might be cheaper (relatively) on phones as they can offload the work
to hardware components and ramp CPU down (or avoid ramping it up).

------
SimeVidas
It’s great to see a summary at the top of the article.

However, the summary (and the article) leaves out a relevant question that
immediately comes to mind: Are other image formats decoded _off_ the main
thread or is it just that decoding JPEG-XR is just more taxing compared to
other formats, thus negating its advantages?

~~~
JohnBooty
Yeah, this was my thought as well.

I'm fairly certain the article may be poorly worded. Certainly standard JPGs
are decoded on the CPU side as well, are they not?

To the best of my knowledge, I don't think any browsers are doing JPG decoding
in hardware; I think they use the GPU for primitives and compositing, etc.

I would look forward to being corrected if I'm wrong!

~~~
pornel
The CPU portion of JPEG decoding is relatively cheap (just Huffman + RLE) and
easy to parallelize (DCT blocks are independent). Conversion and upsampling of
YCbCr can be done on the fly on the GPU, and this even saves memory compared
to using raw RGB/RGBA bitmaps.

Other formats use more expensive entropy coding, more block sizes, smoothing
filters, and inter-block prediction which forces them to be decoded mostly
serially.

~~~
dylan604
Isn't the JPEG instruction set built into the CPUs now anyways? I would be
shocked to find out they are not seeing as MPEG2 has been part of the CPU
since the Pentium days.

~~~
userbinator
There is no "JPEG instruction set" as such --- and what you may be referring
to by MPEG2 is MMX, which is just a SIMD extension that certainly helps with
JPEG and MPEG decoding, among other things.

~~~
dylan604
Interesting. I just remember the days of the add-on hardware boards for real-
time MPEG2 encoding for DVDs that were pretty much obsoleted when some sort of
upgrade to the CPU was made. I officially went 100% software/CPU encoding in
2005. However, it looks like something more specific from Intel called Quick
Sync came out in 2011.[0] This is a dedicated core on the die. Do browsers get
access to that level of hardware?

[0][https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video](https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video)

------
Solar19
I have several somewhat orthogonal thoughts on this:

1\. This is exactly what I've feared and suspected about JPEG-XR, webp, and
JPEG-2000 (which Safari ever so quietly supports). I especially wondered about
JPEG-XR after reading how Microsoft had done some great things with GPU-based
hardware acceleration for JPEG decoding in IE11:
[https://blogs.msdn.microsoft.com/ie/2013/09/12/using-
hardwar...](https://blogs.msdn.microsoft.com/ie/2013/09/12/using-hardware-to-
decode-and-load-jpg-images-up-to-45-faster-in-internet-explorer-11/)

I was puzzled by the fact that they never mention JPEG-XR and have never
published a similar post about hardware acceleration for the more modern
format. They've been incredibly lazy and unfocused on XR for years. It's
effectively dead now.

2\. The author of the don't use it post keeps saying it gets "software
decoded". This implies that this is unusual. But it isn't. Does he think JPEGs
are not software decoded? They are in Firefox, and probably every other
browser but IE11 and Edge. Someone correct me if I'm wrong. Browser makers
have been surprisingly inert on using GPUs for image decoding. libjpeg-turbo
is used by Chrome and Firefox, and possibly Safari. It's pure CPU, albeit with
some SIMD. That's "software decoding". Hardware acceleration normally means
GPU, or a fixed function unit/ASIC like you see for some video codecs.

3\. One percent? This was all about a -1% drop in some metric? What's the
error margin?

4\. He implies that there's a lot of JS in this SPA. This seems like a
possible confound that won't necessarily apply to rigorous web developers who
don't overload on JS (who are admittedly extremely rare in 2019). Sure, they
showed that XR was using more CPU, but so do all post-JPEG formats.

5\. Which leads to my last point. What about webp? He strangely never mentions
it. It's like only JPEG and XR exist. webp performs very poorly in CPU
efficiency compared to JPEG. What are trivago and Cloudinary seeing on that
front? Is webp less taxing than XR?

~~~
cflat
A few clarifying points: * the author specifically said -1% Conversion. If
this were AWS, that would mean a daily reduction of $6million in revenue! *
the author specifically says this is for IE users. This implies two things: a)
webp is not an option and b) likely we are also talking about lower class of
hardware (mom & dad). I would expect WebP had older hardware probably has the
same kind of performance tax, but it's harder to identify because at least
with IE/Edge there is an implied age-of-hardware

------
dahart
> -1% negative impact on core business metrics including conversions in our
> test of JPEG-XRs [...] Let’s not use JPEG-XR on the Web.

Is this an overreaction, or a subversive call to improve the decode perf of
JPEG-XR? I'm certainly on the performance-is-a-feature bandwagon myself, but
1% ux impact is pretty small. And shouldn't we assume that decode perf will
improve in the near future? Is there any reason to assume that this problem is
permanent and unfixable?

~~~
mediumdeviation
> but 1% impact is pretty small

The best you can say here is that it’s within the margin of error, in which
case the effect is so small it’s literally unmeasurable. In that case the null
hypothesis still wins, and you should not switch.

> Is there any reason to assume that this problem is permanent and unfixable?

IE11 as tested in the article is not getting anything other than security
patches. The users who use IE11 are probably also the users most affected by
poor performance since they most likely have the oldest hardware.

~~~
dahart
Interesting response, thanks. Your comment gives me even more questions,
though!

> The best you can say here is that it’s within the margin of error

What is preventing JPEG-XR from becoming faster and a net ux positive,
tomorrow or in the future? Isn't the best I can say that today's results might
be a bug or fluke or a quick and dirty implementation, and not a permanent
problem?

> IE11 as tested in the article is not getting anything other than security
> patches.

The title and conclusion both said we should avoid JPEG-XR on the web, not
that we should avoid it on IE11.

You're right, the article focused on the problem with IE11. Is it already a
net positive on Edge? (The article didn't say.) Should the shim to make it
work in the old browser that fewer people use _really_ override the potential
benefits on the newer browser that more people are already using?

Does it seem reasonable or unreasonable to make future web decisions depend on
an old browser that's on life support today and has small and declining market
share?

------
throwanem
Apparently only IE and Edge support JPEG XR, so the format not being used on
the web will happen on its own anyway.

~~~
andreapaiola
Well... With Edge phasing to Blink noone will support JPEG XR

~~~
wereHamster
The Edge team could still decide to integrate their JPEG XR implementation
into Blink. That would mean Chrome would gain support for JPEG XR.

~~~
05
What makes you think the changes would be accepted into mainline?

~~~
CydeWeys
They don't have to be; Microsoft could maintain that as a separate fork/add-in
to Blink. In theory it sounds like it should be highly modular, so that
shouldn't be a huge ongoing maintenance burden.

I don't know why they wouldn't just switch over to WebP in preference to that,
though. It's better to have fewer of these formats around and the simplest
thing to do is use the one that already has support.

~~~
stordoff
> They don't have to be; Microsoft could maintain that as a separate fork/add-
> in to Blink

Note that parent said "Chrome would gain support for JPEG XR", which I believe
is what 05 is replying to.

------
subfay
This piece gives a hint where Trivago as a company went with Holacracy. The
lack of focus is so obvious and seems to let individual developers create some
work which is not just poorly worded but also nonsense:

Why should anyone test a format which is used by just two browsers with a very
small market share and which will be never adopted by the company with the
biggest market share (because Google has WebP). JPEG-XR is DOA and nobody
should care and waste time on this (except the OP gone astray).

I like that there is a tl;dr right at the beginning though.

~~~
jbreckmckye
I'm not sure I'd be quite so blunt, but I do agree with your argument in the
main: this is so low on the list of priorities that I can't help but wonder if
someone at Trivago thinks they've run out of any actual revenue-generating
development work.

It indicates to me that either the inmates are running the asylum - that devs
are being given too much leeway to pursue personal interests - or (and?) that
Trivago's day to day work is proving so dull that developers in some teams are
inventing R&D work just to stay sane.

~~~
subfay
Yeah my tone was a bit to harsh and I didn't want to insult anyone but I am
still wondering that nobody at Tribago gave him this feedback. However and
hopwfully this useless piece of work leads him to some other more relevant
stuff.

------
mgamache
Hardly surprising. JPEG is a very mature technology with deep support and
optimizations going all the way to hardware support built into many (most?)
cpus. If the real world savings are ~ 20%, it probably won't be worth it.

------
jbreckmckye
Exotic file formats, like exotic JavaScript language constructs, tend to
perform poorly because they are not optimisation targets. And they are not
optimisation targets because - obviously - no-one uses them. To my knowledge
there is no particular intrinsic reason that JPEG-XR decoding cannot be moved
off-thread.

The solution to this problem is to start using these new technologies, so the
browser vendors can prioritise work to make them performant. It is not to
write articles that entrench that neglect and tie us to flawed, ancient
technologies from the 1990s.

It gets worse. If there's one kind of meme that survives a long time in the
development world, it's performance memes. Rumours about poor perf invariably
end up haunting technologies with FUD long after any issues have ceased to be
relevant.

~~~
inferiorhuman
Yep, it's kinda like Github and SSL/TLS. When they went SSL only it was a bit
painful, but things have largely caught up and improved overall.

------
rhacker
Going a little further than just JPEG-XR - definitely do performance tests for
new releases. We had a bug that affected one page in our app. It made people
cringe using the feature set that invoked it, but essentially something in
angular was firing, and the thing it did caused a re-fire. It just kept
looping when on these very specific pages in the app. I finally found it and
killed it. But definitely don't go to production with a client CPU murdering
bug. It can't be measured the same as network latency, etc... so you really
need to use your apps.

I was surprised to see a post from Trivago - I might have to book my next
hotel through them.

------
k__
If the compression is really much better than with JPEG, there is probably a
point where the connection is so slow, that JPEG-XR would be faster than JPEG,
because the file-size would out-weight the decoding performance.

------
microcolonel
This is a little bit wrong in that the other CPU-bound tasks during loading
are generally in the main thread, whereas image decoding is usually on a
different thread (so, depending on how many images you have, and how many
CPUs, it may not increase overall page load time in reality).

Now, overall time to decode still matters. If it's a gigantic hero image above
the fold, then it could take a noticeable amount of time to decode, but it's
probably still a win.

With the only good JPEG 2000 encoder, its quality at small file sizes is
easily better than webp, but in practice it seems virtually nobody uses JPEG
2000, because the open source encoders are not really that great (probably
because the patents are a guessing game, although technically the known
holders are willing to grant royalty-free licenses).

------
Yizahi
At this point in time I suspect we do need to use anything not invented by
Google, if only to slow down arrival of googlenet.

------
t0mbstone
Regardless of the findings, I think it's funny that they think they were able
to "statistically verify" a -1% impact on conversions...

That's a hilarious thought to me. How can a 1% change in a sales funnel be
considered statistically relevant?

------
dfc
The author says that JPEG-XRs are decoded on "the software-side on the CPU" a
couple of times that it seems like it might be a term of art but it sounds
very awkward to me.

Is the TLDR actually "jpeg xr implementation is poorly optimized and
negatively affects browser performance which decreased conversions."

