Hacker News new | past | comments | ask | show | jobs | submit login
Studying Lossy Image Compression Efficiency (blog.mozilla.org)
66 points by dmmalam on Oct 20, 2013 | hide | past | favorite | 43 comments



"JPEG-killers" we have are disappointingly unambitious. That's skating to where the puck was 20 years ago.

They're not aiming to be RAW killers and OpenEXR killers. They're not making anything new possible, they're just polishing the same old thing.

Photographers are increasingly avoiding JPEG because of limited dynamic range it has. WebP (due to its video heritage) has even lower dynamic range than JPEG and can't preserve full resolution of chroma channels.

JPEG-killers add simple alpha channel that 17-year-old PNG has, but nobody even thought about adding alpha with blending modes (e.g additive, to be able to create lights, or per-channel to be able to represent subpixel AA).

After all these years there still isn't a format that can save a Photoshop layer and represent non-trivially blended transparency correctly (closest you can get is images wrapped in SVG with filters, uhh).


>After all these years there still isn't a format that can save a Photoshop layer and represent non-trivially blended transparency correctly (closest you can get is images wrapped in SVG with filters, uhh).

There are other ways you can hack around it. One is to save the JPEG at double width and save the alpha channel as a gray scale image on the right, and then use css masks for webkit, canvas for firefox, and CSS filters for IE to add the alpha channel when the page renders. A similar technique can be used to do transparent video: http://jakearchibald.com/scratch/alphavid/

For better results, you actually want to pad the right side of the non-alpha image to a multiple of 8 so that you hit JPEG block boundaries. Then you repeat the edge pixels so that you don't get artifacts at the edges.

Handling artifacts from transparent pixels turning non-transparent is a little trickier, but one strategy is to apply a gaussian blur (radius 8) to the original image, discard the alpha channel, and then fill all completely transparent pixels in the original with the blurred version.


(Warning, unsourced hearsay ahead!)

Backstory here, gleaned from eavesdropping on Mozilla employees and Gecko engineers: Google and Facebook are leaning on Mozilla to implement WebP in Firefox, while Mozilla's been resisting due to (in their eyes) the lack of measurable improvement over JPEG. Furthermore, as joosters alludes to elsewhere in this thread, most JPEG libraries set the default compression level way too conservatively, and most users of the libraries don't bother to ever deviate from that default. Merely increasing the compression level from the default can yield a 10% decrease in image size without degrading quality whatsoever as measured by the standard metrics mentioned in the OP.

But there's the rub: the standard metrics kinda suck! Nobody's yet come up with a really solid algorithm to compare the quality of different image formats and compression ratios. So all this research we're doing may very well be useless and misguided. :P

(I'd love it if someone who actually knows what they're talking about can pitch in. Corrections welcomed!)


I personally keep expecting to see a schism develop between Mozilla and Google, considering they compete on two fronts: web browsing and mobile OS. Hopefully, nothing more than just healthy competition will happen.


When comparing against JPEG, it's always worth pointing out that most JPEG compressed images aren't optimal and you can reduce their size further with no quality loss - http://www.kokkonen.net/tjko/projects.html jpegoptim will improve the compression in almost all cases.

This highlights a big problem with work on better image compression - we can already do better, but no-one bothers. If you're trying to introduce a whole new codec, you've got a big hurdle to jump.


>If you're trying to introduce a whole new codec, you've got a big hurdle to jump.

Agreed, I think the big hurdle is simply that jpeg is 'good enough' for web use, and that a transition to a new standard lossy image format just isn't 'worth the hassle' so to speak.

As for webp, I personally use it for lossless encoding instead of PNG on personal files these days, but I don't really use it for lossy as I don't think it makes a big enough difference versus jpg and jpg is supported everywhere.

That said the current webp implementation is based upon VP8, so I suppose the next implementation will be based upon VP9 which would likely make it competitive with the HEVC based image codec, and unlike the HEVC based one it isn't patent/royalty encumbered.


The WebP team said they wouldn't revise to match VP9. The improvements mostly didn't apply to still images so it wasn't worth it.


Bigger transforms and more intra modes certainly apply to still images. JPEG's 8x8 transform is pretty small given how large images are nowadays, and VP8 only has a 4x4 transform.


Yes, they mentioned that as one clear win:

https://groups.google.com/a/webmproject.org/d/topic/webp-dis...


Another potential problem is that JPEG encoding and decoding is very mature. I remember this being an issue for color spaces JPEG 2000 in earlier versions of OS X (no idea about more recent versions). No matter what you did as a developer, it never seemed to be possible to get colors to look the same way once they'd been passed through the system-provided JPEG 2000 encoder.

If you get all modern browsers to agree to support a new codec, but they all render them slightly differently, nobody is going to use it.


Mozilla misses one important of WebP : Alpha channel. Today PNG 24 is the only way to achieve transparency with a 24 bits palette, resulting in unnecessary huge files. As a webdesigner, WebP would replace PNG 24, not jpeg.

Anyway, I have this strange impression that Mozilla is more akin to spent ressources dismissing the format rather than implementing it or finding an alternative.


Years ago I wrote a little proof of concept burying an alpha channel in the JPEG application data area and pulling it back out in the client browser to render JPEG with alpha.

It worked well, for some browsers. Now I expect it would work well on most of them, though I haven't looked at it in years.

It was received with zero interest. That may mean no one needs JPEG with alpha.

You can read more at http://jim.studt.net/jpeg-alpha/


That is pretty nifty! I also liked your graphic explantation with the asrtronaut.


> I have this strange impression that Mozilla is more akin to spent ressources dismissing the format rather than implementing it or finding an alternative.

Mozilla is acting responsibly here. It's not just effort to add the format to their browser. The format will spread to entire ecosystem, requiring websites to support it (e.g. can you upload photo or avatar in WebP?), image viewers, image editors, rich text editors, etc. When Facebook implemented Chrome-only WebP users complained, because they couldn't use images anywhere else.

Also take into account that bandwidth doubles about every 5 years[1], so if WebP is giving 20% gain, it only advances the web by 1 year. What WebP makes possible today will be possible anyway next year, and we won't have to spend years re-building interoperability that JPEG has built over 20 years.

[1] http://www.akamai.com/stateoftheinternet/


Were they acting responsibly when they implemented Theora into Firefox even though it has inferior quality too? Are we now saddled with a new video format spread to an entire ecosystem and websites required to support it? Are users pissed off that they "Save As" Theora files but can't use them in other apps?

WebP isn't just about efficiency over JPEG, the biggest benefit comes from replacing PNG and GIF, which people use for alpha transparency and animation respectively. These are practical, immediate problems that today's web developers face, can they wait a few more years until Xiph approved codec does better?


> Were they acting responsibly when they implemented Theora into Firefox even though it has inferior quality too?

Yes, because of the IP/patent issues, which do not apply to GIF (nowadays), PNG, and JPEG.


The issue of widespread usage of alpha transparency and animated images do apply however. Rather than hold up WebP over a future imagined codec that doesn't exist pending research, a never-going-to-implement-anyway HEVC, or a doesnt-have-all-the-features JPEG, why not work with the WebP folks to fix any deficiencies?

It's highly reminiscent of fight over the ACA in Congress. "Repeal it". Rather than work with making the bill better or fixing the problems, just filibuster.


> Rather than hold up WebP over a future imagined codec that doesn't exist pending research, a never-going-to-implement-anyway HEVC, or a doesnt-have-all-the-features JPEG, why not work with the WebP folks to fix any deficiencies?

The bitstream format of VP8 (which WebP is a derivative of) is frozen.

> It's highly reminiscent of fight over the ACA in Congress. "Repeal it". Rather than work with making the bill better or fixing the problems, just filibuster.

Come on. Google is not a legislative body.


Neither of those mean Mozilla can't work with Google together on a next-gen spec. Simply saying the current status quo is ok (jpeg + gif/png) is a disservice. Pushing JPEG XR isn't a solution either, it doesn't support animation for example.

What I'm saying is, rather than say "no", be constructive. I see Mozilla saying "No", but I don't see them actually proposing anything that solves the other problems WebP is trying to solve. No studies have been presented on lossless, transparency, or animated encoding efficiency for example.

WebP offers huge immediate savings on PNG/GIF use cases today, and this is especially important on mobile. Developers need these options now, not 2 years from now.


> I see Mozilla saying "No", but I don't see them actually proposing anything that solves the other problems WebP is trying to solve. No studies have been presented on lossless, transparency, or animated encoding efficiency for example.

Mozilla supports (and developed) APNG, which has existed for a lot longer than WebP. Google has thus far not implemented APNG (bug here, open since 2008 [1]). You might want to be careful about casually throwing around words like "NIH" when it comes to animated image formats…

The fact is that both browsers are perfectly free to not implement standards that they think are harmful, without having to come up with some better solution right away. Whenever we implement something on the Web, if it gets traction we are stuck with it forever. So we should move carefully.

Don't you think we should be making sure standards are sizable improvements over the status quo instead of just accepting every marginal improvement? If WebP is indeed worse than JPEG, then authors have to choose between better quality (JPEG) and alpha channels/transparency (WebP). That's not a very good decision to force them to make.

[1]: http://code.google.com/p/chromium/issues/detail?id=1171


Is WebP really "worse" than JPEG, or just about the same, or not a substantial improvement? The real question is, what if you don't compare WebP to JPEG, but instead, compare it to APNG, then how does the calculus come out?

If I prepare two animated/transparent images of 100 frames today, what is the filesize of APNG vs WebP?

To me that's the real question. My social network streams are filled full of animated GIFs now and they are huge. The G+ mobile client uses WebP for these and the savings are substantial.


> Is WebP really "worse" than JPEG, or just about the same, or not a substantial improvement?

I don't know. Those are subjective terms. You can draw your own conclusions from the study…

> The real question is, what if you don't compare WebP to JPEG, but instead, compare it to APNG, then how does the calculus come out?

I don't know off the top of my head. I agree that this should be studied.

What I'm quibbling with is the notion that we should just rush to adopt large proposals like WebP if they're a marginal improvement in any area. Like I said, on the Web we are stuck with mistakes forever if they gain traction. Maybe the story would have been different if WebP had been just the WebP-lossless-animated portion. (I don't call the shots, except in Servo, but I would personally have no issues with adopting WebP-lossless-animated.) But WebP is tied together with the lossy compression that seems to have issues relative to JPEG. "Two steps forward, one step back" is less attractive than "two steps forward".

Google says WebP versus PNG is 28%, so I would expect the results for WebP-animated versus APNG to be similar. IMHO, a 28% improvement is not at the level of "so good we must rush to standardize this now, warts and all".


Cutting bandwidth usage on mobile clients by 28% seems like a pretty good win to me. If the gains were 5%, I agree it would be iffy, but ~30% is nothing to shake a stick at.

This whole issue seems sort of like SPDY. HTTP-NG went on for a more than a decade and produced nothing. Google rolled out SPDY, and suddenly we have an HTTP 2.0 spec being discussed. Sometimes the perfect is the enemy of the good, and vendors rolling out implementations forces the community to act and put resources on an issue.

Even if WebP doesn't end up as the ultimate new replacement for JPEG/PNG, the fact that Google, Facebook, and others are rolling it out will likely force some movement to get serious about finding a replacement. Ideally, one doesn't roll out an implementation until there is a consensus spec agreement, but the real world usually doesn't match up the the ideal.

Look at how asm.js's rollout forced V8 to adapt. Even if V8 isn't adopting asm.js, Mozilla shipping it forced the V8 team to spend more time optimizing those paths.

I don't particularly care if WebP is the final image standard or not. What I don't want to see is Mozilla sitting on their hands waiting for a solution to fall into their lap, and putting all their resourcesin Daala.

I expect standards working groups to actually have working members, actively engaged in putting forth positive proposals, not people who just sit around and raise objections. That was one of the problems ARB had for years. If that doesn't happen, then you end up getting steamed rolled by proprietary proposals (e.g. Cg, CUDA, et al)


They're aware of that as a startup they fund asked them to look at WebP again because it worked so well for their use-case of icons with alpha. I believe that is why they reopened the bug.

It's not covered by this study though.


I've developed lossy PNG encoder that creates 3-4 times smaller PNG files:

http://pngmini.com/lossypng.html


My thoughts exactly. So why isnt this being used instead.


It should be!

It's so hard to spread knowledge about image optimization. There are still developers who use (non-animated) GIF images.

Photoshop has still makes JPEG optimization an option :( It probably made sense on 8Mhz Macintosh, but today it takes user longer to tick that checkbox than it takes to optimize a JPEG.


Does it beat PNGOut or OptiPNG?


It's complementary. You can optimize lossy PNG files with these tools to get even bigger savings (that's what http://tinypng.org does).

But if you just compare file sizes then pngquant2 wins by a wide margin.


>Today PNG 24 is the only way to achieve transparency with a 24 bits palette, resulting in unnecessary huge files.

That's not necessarily true. If you only have completely-transparent and completely-opaque pixels color type 2 will work, and if your total unique colors (RGBA) number less than 256 you can still do plaetted.


I think Mozilla is right to tread carefully and not commit to yet another format for a minor improvement. In my opinion our web clients are bloated enough as is, and they're only getting bigger. Once you add support for a format and it gains mainstream traction on the web, it'll be really hard to remove, and we'll be stuck with software that has to support so many things for reasons that do not make sense in the near future.

As for the study, I have to admit I did not read it. But the first thing to note is that computed metrics (SSIM, PSNR) are not reliable approximations of perceived quality. Optimizing for either will produce a codec that pleases the computer, not the eye. This is why the tuning of image and video codecs for psychovisual effects is such a laborious, manual process -- you have to have a human viewer to assess the quality.

In the long term I think it would be good if a new still-image format was derived from the intra portion of some reasonably advanced video codec, as they all beat JPEG handily. This is essentially what WebP is, but if I recall correctly, Google didn't really give VP8 the love it needed when they acquired On2. Instead they rushed with a "release early, release often" mentality, presumably hoping the community would jump the boat first and do the grunt work for them later (this was during html5 video wars when some thought Theora sucks and others thought h.264 is too encumbered -- VP8 was Google's resolution). The details are shady but the curious should search for the debate around early VP8 (I think the x264 folk did some extensive notes). Needless to say I think, VP8 (and WebP) wasn't and isn't the game changer Google would've hoped.

To avoid patents, a reasonable choice would IMHO be to look at Xiph.Org's upcoming video codec Daala. Curiously Mozilla has the right man in house since Monty moved from Red Hat to Mozilla like, almost like yesterday... Daala however isn't quite ready yet as the Xiph guys are busy researching out new ideas and unicorns. But the results so far are very interesting to say the least; anyone interested should check out Monty's demo pages.[1]

As others have said, JPEG is "good enough" for most of us, and even JPEG is used rather inefficiently, often with too high quality. So there's no compelling reason to push for a new format here-and-now. Give it some time and commit once there's something significant I say. In the future we'll thank ourselves for not having deployed all that junk over minor increments.

[1] http://people.xiph.org/~xiphmont/demo/


Mozilla hasn't used PSNR. They've used PSNR-HVS-M, which is more sophisticated.

In test of objective metrics SSIM was one of the best (0.8 spearman correlation with human scores) http://www.ponomarenko.info/tid2008.htm

IHMO the study has been done very well. They've even eliminated potential differences coming from colorspace conversion and chroma subsampling. If anything, this study favors WebP that can't do full-resolution color.


Right, I probably came across not quite as intended; sorry about that. I didn't mean to say the study is hosed (which I can't say because I didn't read it!), the remark about objective metrics was a more general one. Yes, SSIM is one of the better metrics available, and obviously it's useful for certain things.

However it still doesn't accurately represent preceived quality; time and time again certain psy optimizations for instance are shown to reduce SSIM while perceived quality goes up. Conversely, a codec can optimize for SSIM alone and score well on a test that ranks by SSIM; but the perceived quality could be better. For this reason e.g. x264 has an option to disable psy optimizations that hurt the objective metrics.

There are other things that can hurt metrics a lot while doing nothing perceptible, though these are rather like bugs or errors in testing methodology, and as such would likely be noticed in a well conducted study. See also: https://wiki.xiph.org/Notes_on_testing_theora#General_points...


From what I can tell, for Y-SSIM and IW-SSIM they appear to be encoding color images but only measuring the luma component, which is invalid and renders the results meaningless. Unless they're actually encoding greyscale images for those sections without mentioning it?

BTW HEVC-MSP is limited to 4:2:0 only as well.


Study author here. For Y-SSIM and IW-SSIM we convert the images to grayscale (MATLAB 'rgb2gray') before running the scoring algorithms. We aren't running the algorithms on luma planes. See published code and methodology.


How are you measuring filesize? Looking at compression_test.py, it looks like you encode a color image and use that filesize. But chroma takes bits to encode, and if you're only measuring greyscale (which is basically luma depending on what formula rgb2gray uses) then you're penalizing better chroma quality.

If you use a greyscale/luma-only metric, you have to encode greyscale from the start, not convert a color image after encoding. Otherwise you're measuring chroma bit cost, but not chroma distortion. Thus, invalid.

Actually, is PSNR-HVS-M greyscale-only as well? In that case, only the RGB-SSIM results would be valid...

And actually, thinking about it even more, converting to greyscale at the tail end just for Y-SSIM and IW-SSIM is fundamentally flawed (if it doesn't result in exactly the same bits as the luma plane), as now you're measuring distortion from an image that the codec was never asked to encode!


There's nothing curious about Chris Montgomery moving to Mozilla - that's who most of the Daala team work for. Mozilla has quietly(?) employed almost all of the Xiph codec developers to work on Daala and Opus.


Discussion on reddit:

http://www.reddit.com/r/programming/comments/1ottdn/mozilla_...

I sometimes wonder why something gains traction on reddit and not on hacker news or vice versa.


A few years back i compared SSIM on Lenna for a few image codecs: http://code.ivysaur.me/srv/ssim_0.png comparing Photoshop's JPEG encoder, imagemagick's jp2k encoder, WebP, and a single frame from x264 (which came out ahead at all file sizes, as expected).

You have to be significantly better than JPEG to really make a compelling argument as the successor, though. IPv6 is undoubtedly better than IPv4 and the upgrade started over a decade ago.


What happened to JPEG 2000?


Patents, and weak results of wavelets.


I was a bit surprised to see Mozilla admitting that HEVC exists without smearing it. If only they could be this impartial in their video comparisons. Instead we had to endure the embarrassing spectacle of them trying to argue that Theora was competitive with H.264[1].

[1] http://weblogs.mozillazine.org/asa/archives/2009/06/theora_v...


Mozilla has never made any statement that Theora is competitive with H.264; various people (both who work for Mozilla and around the community) have made such statements, but they don't speak for Mozilla.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: