

Corrected "real world analysis of Google’s webp versus jpg" - Terretta
http://michael.terretta.com/corrected-real-world-analysis-of-googles-webp

======
barrkel
There's something in all this that I suspect demonstrates a possible flaw in
Google's traditional data-orientation, as opposed to human judgement.

What the colour gradients (1Mcolors) example in the referred to article
([http://englishhard.com/2010/10/01/real-world-analysis-of-
goo...](http://englishhard.com/2010/10/01/real-world-analysis-of-googles-webp-
versus-jpg/)) shows is that WebP, at the limits of quality, prefers to blur.
The problem is that most real-world scenes are of objects that have texture,
and an artificial texture is psychologically more pleasing than blurring away
all these high-frequency small detail patterns. Dark Shikari’s article
(<http://x264dev.multimedia.cx/?p=541>) shows this really clearly: in a real-
world landscape, WebP loses badly to _both_ jpg and x264 with too much
blurring.

Thing is, if you just do the "obvious" quality metric and do a subtraction of
the encoded picture from the original, and sum up all the differences, that
doesn't encode the psychological aspect.

A blurred image might indeed do well here, because the blur colour will be
somewhere in the middle between the highs and lows of the high-frequency
pattern, so the error won't be too large.

But a psychologically-oriented encoding may have encoded a pattern that
resembles the original frequencies, but not precisely aligned, so the highs
and lows may not coincide exactly between encoded image and original. It may
thus simultaneously give a worse difference on a subtraction, but please the
eye more.

~~~
kevinpet
I had an internship working on measuring accuracy of fractal image compression
algorithms many years ago. I remember the same thing -- measured accuracy was
poor, but perceived quality was high.

------
HardyLeung
I was also able to reproduce the original post's JPG distortion result using
Pixlr (not just GIMP). Here's the 1920x1200 JPG compressed at 0% quality
(resulting file is ~60KB) and subsequently uploaded to Imgur.

<http://imgur.com/dZN2B>

At such a high degree of compression, Jpeg wouldn't be able to get to any
level of quality because it boils down to a pixelation in the size of 8x8 (or
16x16). You'd definitely needs a different kind of compression to preserve
quality at such high level of compression. Realistically, many image editors
offer default compression at 80% so the artefacts are less visible. I don't
know how WebP cocmpare to Jpeg at the 80% setting.

------
jmillikin
(edit: these settings are not optimal. See followup post)

I don't know what magical encoder the author is using, but I was easily able
reproduce the original post's results (for JPEG, I didn't try WebP). Steps:

1\. Download the original image from <
[http://fc07.deviantart.net/fs21/f/2007/275/f/4/f4e91d2565442...](http://fc07.deviantart.net/fs21/f/2007/275/f/4/f4e91d25654426ac.jpg)
>

2\. Open in your favorite image editor.

3\. Save it with whatever optimization settings you want, until it gets down
to the goal size (in this case, 44.5KiB).

4\. Compare to the original.

For example, I used GIMP, with these settings < <http://imgur.com/SIasU.png> >
, which produced this result < <http://imgur.com/8Sclu.jpg> >. Although the
author is using a different encoder, I doubt it's _this_ much better than
GIMP's. (edit: with better settings, GIMP's result is <
<http://imgur.com/8aEPJ.jpg> >).

~~~
Terretta
It could be that the original author, and your replication of his results, is
not illustrating a JPEG compression issue, but an issue using GIMP's JPEG
compression to compress a 1920x1200 image into a 45KB JPEG. This is certainly
an "edge case" compression test.

The author updated comments on reddit saying he hadn't been clear, but he was
talking about full 1920x1200 in the 45KB file. That's a different story, so I
repeated my tests at full resolution.

Using Photoshop's "Save for Web and Devices" option, Photoshop JPEG
compression (at "quality zero") introduces a moire pattern, but still does not
generate the extreme banding issue shown on the blog, and replicated in your
results.

Here is what Photoshop generates in 44,962 bytes, at 1920x1200 resolution:

[http://michael.terretta.com/update-corrected-real-world-
anal...](http://michael.terretta.com/update-corrected-real-world-analysis-of-
googl)

I apologize, I'm not familiar with GIMP or why its results might show such
dramatic banding.

~~~
jmillikin
After further testing, it appears that the "Save EXIF data" and "Save XMP
data" options are very expensive at these resolutions/file sizes. Disabling
them nets a few dozen KiB. Additionally, setting "Smoothing" to full makes the
image smaller at the expense of blurriness.

Updated version: < <http://imgur.com/8aEPJ.jpg> >

"Save for web" is probably a shortcut to remove those extra data. I wonder if
Photoshop is also adding additional smoothing, beyond GIMP's maximum? That
would explain why the PS image's background is smoother, but the inner details
(like specular highlights) are not as distinct.

~~~
jjcm
The EXIF/XMP data is highly insightful, I didn't even think about how much
space those would take up. I updated the original blog post with the
information you provided here, and cited you for it. Hope you don't mind.

~~~
Terretta
I mentioned in comments on your original article: I wasn't able to appreciably
beat the BMW or portrait images.

Metadata would help explain this, since as a percentage of file size, it'd be
a lot more detrimental on the first image at 45K than the BMW at 575K.

But sure seems there's something else going on with the first image's circular
gradient; PS seems to be dithering its blocks in a moire pattern while GIMP is
showing strong banding even on files w/o metadata.

~~~
oozcitak
Metadata is most certainly the case here. The breakdown of metadata for the
original image
([http://fc07.deviantart.net/fs21/f/2007/275/f/4/f4e91d2565442...](http://fc07.deviantart.net/fs21/f/2007/275/f/4/f4e91d25654426ac.jpg)):

    
    
      EXIF (APP1) -> 5328 bytes
      Photoshop IRB (APP13) -> 5896 bytes
      Adobe XMP (APP1) -> 26284 bytes
      Adobe (APP14) -> 12 bytes 
      Total -> 37520 bytes (36.6 kB)
    

That's 36 kB of metadata, leaving only 8 kB for compressed image data. This
explains the banding seen in the English Hard post. The BMW image on the other
hand
([http://fc01.deviantart.net/fs51/f/2009/313/1/c/Jesus_Mobile_...](http://fc01.deviantart.net/fs51/f/2009/313/1/c/Jesus_Mobile_by_DivineError.jpg)):

    
    
      Photoshop Ducky (APP12) -> 15 bytes 
      Adobe (APP14) -> 12 bytes
      Total -> 27 bytes
    

The existence of the "Ducky" section means that the image was "saved for web"
in Photoshop.

EDIT: Added numbers for the BMW image.

~~~
Terretta
I got GIMP, and used the "Save as JPEG" output to see if it's including
metadata.

According to jhead[1], GIMP is not including metadata in the saved JPEG.

At quality "13", the file was 46302 bytes. Using the "purejpg" option on
jhead, it dropped to 45592, or -710 bytes.

This 45592 bytes is exactly, to the byte, what the original article found.

[1] <http://www.sentex.net/~mwandel/jhead/usage.html>

------
pilif
is it possible that the faulty image was saved as a 256 color png by accident?
That would for sure explain the look of the gradient

~~~
jjcm
It wasn't. It seems that the image compression algorithm that GIMP uses for
JPG just has a higher tendency to band at lower levels of quality. I just
retried it to make sure that there weren't any mistakes, then attempted to
make it look better using the advanced options to no avail. You're welcome to
try it out though, the lossless images are here: <http://jjcm.org:8081/webp>

I'm in the process of updating the images on the original blog post (I'm the
author) using photoshop's jpg compression algorithm, and it should change the
results a bit.

------
olegkikin
Compressing 1920x1200px images to 45K is not "real world" in the first place.

------
jacquesm
I'm having a betamax moment around all of this.

------
hackermom
There's an important detail to this subject that seems to be unknown to most
people writing about this debacle: JPEG compression is subject to the software
performing the compression; how well the image is analyzed, and how well the
resulting product is coded. No two JPEG compressors are alike.

