
Studying Lossy Image Compression Efficiency - dmmalam
https://blog.mozilla.org/research/2013/10/17/studying-lossy-image-compression-efficiency
======
clarry
I think Mozilla is right to tread carefully and not commit to yet another
format for a minor improvement. In my opinion our web clients are bloated
enough as is, and they're only getting bigger. Once you add support for a
format and it gains mainstream traction on the web, it'll be really hard to
remove, and we'll be stuck with software that has to support so many things
for reasons that do not make sense in the near future.

As for the study, I have to admit I did not read it. But the first thing to
note is that computed metrics (SSIM, PSNR) are not reliable approximations of
perceived quality. Optimizing for either will produce a codec that pleases the
computer, not the eye. This is why the tuning of image and video codecs for
psychovisual effects is such a laborious, manual process -- you have to have a
human viewer to assess the quality.

In the long term I think it would be good if a new still-image format was
derived from the intra portion of some reasonably advanced video codec, as
they all beat JPEG handily. This is essentially what WebP is, but if I recall
correctly, Google didn't really give VP8 the love it needed when they acquired
On2. Instead they rushed with a "release early, release often" mentality,
presumably hoping the community would jump the boat first and do the grunt
work for them later (this was during html5 video wars when some thought Theora
sucks and others thought h.264 is too encumbered -- VP8 was Google's
resolution). The details are shady but the curious should search for the
debate around early VP8 (I think the x264 folk did some extensive notes).
Needless to say I think, VP8 (and WebP) wasn't and isn't the game changer
Google would've hoped.

To avoid patents, a reasonable choice would IMHO be to look at Xiph.Org's
upcoming video codec Daala. Curiously Mozilla has the right man in house since
Monty moved from Red Hat to Mozilla like, almost like yesterday... Daala
however isn't quite ready yet as the Xiph guys are busy researching out new
ideas and unicorns. But the results so far are very interesting to say the
least; anyone interested should check out Monty's demo pages.[1]

As others have said, JPEG is "good enough" for most of us, and even JPEG is
used rather inefficiently, often with too high quality. So there's no
compelling reason to push for a new format here-and-now. Give it some time and
commit once there's something significant I say. In the future we'll thank
ourselves for not having deployed all that junk over minor increments.

[1]
[http://people.xiph.org/~xiphmont/demo/](http://people.xiph.org/~xiphmont/demo/)

~~~
pornel
Mozilla hasn't used PSNR. They've used PSNR-HVS-M, which is more
sophisticated.

In test of objective metrics SSIM was one of the best (0.8 spearman
correlation with human scores)
[http://www.ponomarenko.info/tid2008.htm](http://www.ponomarenko.info/tid2008.htm)

IHMO the study has been done very well. They've even eliminated potential
differences coming from colorspace conversion and chroma subsampling. If
anything, this study favors WebP that can't do full-resolution color.

~~~
brigade
From what I can tell, for Y-SSIM and IW-SSIM they appear to be encoding color
images but only measuring the luma component, which is invalid and renders the
results meaningless. Unless they're actually encoding greyscale images for
those sections without mentioning it?

BTW HEVC-MSP is limited to 4:2:0 only as well.

~~~
joshmoz
Study author here. For Y-SSIM and IW-SSIM we convert the images to grayscale
(MATLAB 'rgb2gray') before running the scoring algorithms. We aren't running
the algorithms on luma planes. See published code and methodology.

~~~
brigade
How are you measuring filesize? Looking at compression_test.py, it looks like
you encode a color image and use that filesize. But chroma takes bits to
encode, and if you're only measuring greyscale (which is basically luma
depending on what formula rgb2gray uses) then you're penalizing better chroma
quality.

If you use a greyscale/luma-only metric, you have to encode greyscale from the
start, not convert a color image after encoding. Otherwise you're measuring
chroma bit cost, but not chroma distortion. Thus, invalid.

Actually, is PSNR-HVS-M greyscale-only as well? In that case, only the RGB-
SSIM results would be valid...

And actually, thinking about it even more, converting to greyscale at the tail
end just for Y-SSIM and IW-SSIM is fundamentally flawed (if it doesn't result
in exactly the same bits as the luma plane), as now you're measuring
distortion from an image that the codec was _never asked to encode!_

------
pornel
"JPEG-killers" we have are disappointingly unambitious. That's skating to
where the puck was 20 years ago.

They're not aiming to be RAW killers and OpenEXR killers. They're not making
anything new possible, they're just polishing the same old thing.

Photographers are increasingly avoiding JPEG because of limited dynamic range
it has. WebP (due to its video heritage) has even lower dynamic range than
JPEG and can't preserve full resolution of chroma channels.

JPEG-killers add simple alpha channel that 17-year-old PNG has, but nobody
even thought about adding alpha with blending modes (e.g additive, to be able
to create lights, or per-channel to be able to represent subpixel AA).

After all these years there still isn't a format that can save a Photoshop
layer and represent non-trivially blended transparency correctly (closest you
can get is images wrapped in SVG with filters, uhh).

~~~
mistercow
>After all these years there still isn't a format that can save a Photoshop
layer and represent non-trivially blended transparency correctly (closest you
can get is images wrapped in SVG with filters, uhh).

There are other ways you can hack around it. One is to save the JPEG at double
width and save the alpha channel as a gray scale image on the right, and then
use css masks for webkit, canvas for firefox, and CSS filters for IE to add
the alpha channel when the page renders. A similar technique can be used to do
transparent video:
[http://jakearchibald.com/scratch/alphavid/](http://jakearchibald.com/scratch/alphavid/)

For better results, you actually want to pad the right side of the non-alpha
image to a multiple of 8 so that you hit JPEG block boundaries. Then you
repeat the edge pixels so that you don't get artifacts at the edges.

Handling artifacts from transparent pixels turning non-transparent is a little
trickier, but one strategy is to apply a gaussian blur (radius 8) to the
original image, discard the alpha channel, and then fill all completely
transparent pixels in the original with the blurred version.

------
kibwen
(Warning, unsourced hearsay ahead!)

Backstory here, gleaned from eavesdropping on Mozilla employees and Gecko
engineers: Google and Facebook are leaning on Mozilla to implement WebP in
Firefox, while Mozilla's been resisting due to (in their eyes) the lack of
measurable improvement over JPEG. Furthermore, as joosters alludes to
elsewhere in this thread, most JPEG libraries set the default compression
level way too conservatively, and most users of the libraries don't bother to
ever deviate from that default. Merely increasing the compression level from
the default can yield a 10% decrease in image size without degrading quality
_whatsoever_ as measured by the standard metrics mentioned in the OP.

But there's the rub: the standard metrics kinda suck! Nobody's yet come up
with a really solid algorithm to compare the quality of different image
formats and compression ratios. So all this research we're doing may very well
be useless and misguided. :P

(I'd love it if someone who actually knows what they're talking about can
pitch in. Corrections welcomed!)

~~~
RexRollman
I personally keep expecting to see a schism develop between Mozilla and
Google, considering they compete on two fronts: web browsing and mobile OS.
Hopefully, nothing more than just healthy competition will happen.

------
joosters
When comparing against JPEG, it's always worth pointing out that most JPEG
compressed images aren't optimal and you can reduce their size further with no
quality loss -
[http://www.kokkonen.net/tjko/projects.html](http://www.kokkonen.net/tjko/projects.html)
jpegoptim will improve the compression in almost all cases.

This highlights a big problem with work on better image compression - we can
already do better, but _no-one bothers_. If you're trying to introduce a whole
new codec, you've got a big hurdle to jump.

~~~
gillianseed
>If you're trying to introduce a whole new codec, you've got a big hurdle to
jump.

Agreed, I think the big hurdle is simply that jpeg is 'good enough' for web
use, and that a transition to a new standard lossy image format just isn't
'worth the hassle' so to speak.

As for webp, I personally use it for lossless encoding instead of PNG on
personal files these days, but I don't really use it for lossy as I don't
think it makes a big enough difference versus jpg and jpg is supported
everywhere.

That said the current webp implementation is based upon VP8, so I suppose the
next implementation will be based upon VP9 which would likely make it
competitive with the HEVC based image codec, and unlike the HEVC based one it
isn't patent/royalty encumbered.

~~~
ZeroGravitas
The WebP team said they wouldn't revise to match VP9. The improvements mostly
didn't apply to still images so it wasn't worth it.

~~~
brigade
Bigger transforms and more intra modes certainly apply to still images. JPEG's
8x8 transform is pretty small given how large images are nowadays, and VP8
only has a 4x4 transform.

~~~
ZeroGravitas
Yes, they mentioned that as one clear win:

[https://groups.google.com/a/webmproject.org/d/topic/webp-
dis...](https://groups.google.com/a/webmproject.org/d/topic/webp-
discuss/qmASWczhS0c/discussion)

------
tangue
Mozilla misses one important of WebP : Alpha channel. Today PNG 24 is the only
way to achieve transparency with a 24 bits palette, resulting in unnecessary
huge files. As a webdesigner, WebP would replace PNG 24, not jpeg.

Anyway, I have this strange impression that Mozilla is more akin to spent
ressources dismissing the format rather than implementing it or finding an
alternative.

~~~
pornel
I've developed _lossy_ PNG encoder that creates 3-4 times smaller PNG files:

[http://pngmini.com/lossypng.html](http://pngmini.com/lossypng.html)

~~~
ksec
My thoughts exactly. So why isnt this being used instead.

~~~
pornel
It should be!

It's _so_ hard to spread knowledge about image optimization. There are still
developers who use (non-animated) GIF images.

Photoshop has still makes JPEG optimization _an option_ :( It probably made
sense on 8Mhz Macintosh, but today it takes user longer to tick that checkbox
than it takes to optimize a JPEG.

------
ralfd
Discussion on reddit:

[http://www.reddit.com/r/programming/comments/1ottdn/mozilla_...](http://www.reddit.com/r/programming/comments/1ottdn/mozilla_releases_study_on_lossy_image_compression)

I sometimes wonder why something gains traction on reddit and not on hacker
news or vice versa.

------
mappu
A few years back i compared SSIM on Lenna for a few image codecs:
[http://code.ivysaur.me/srv/ssim_0.png](http://code.ivysaur.me/srv/ssim_0.png)
comparing Photoshop's JPEG encoder, imagemagick's jp2k encoder, WebP, and a
single frame from x264 (which came out ahead at all file sizes, as expected).

You have to be significantly better than JPEG to really make a compelling
argument as the successor, though. IPv6 is undoubtedly better than IPv4 and
the upgrade started over a decade ago.

------
darkstalker
What happened to JPEG 2000?

~~~
pornel
Patents, and weak results of wavelets.

------
anon1385
I was a bit surprised to see Mozilla admitting that HEVC exists without
smearing it. If only they could be this impartial in their video comparisons.
Instead we had to endure the embarrassing spectacle of them trying to argue
that Theora was competitive with H.264[1].

[1]
[http://weblogs.mozillazine.org/asa/archives/2009/06/theora_v...](http://weblogs.mozillazine.org/asa/archives/2009/06/theora_video_vs.html)

~~~
gsnedders
Mozilla has never made any statement that Theora is competitive with H.264;
various people (both who work for Mozilla and around the community) have made
such statements, but they don't speak for Mozilla.

