
Jpeg2png: Silky smooth JPEG decoding – no more artifacts (2016) - todsacerdoti
https://github.com/victorvde/jpeg2png
======
Scaevolus
Knusperli is similar:
[https://github.com/google/knusperli](https://github.com/google/knusperli)

Instead of trying to smooth the entire image, it reduces blocking artifacts by
finding plausible coefficients that reduce discontinuities at block boundaries
(jpegs are encoded as a series of 8x8 blocks). "jpeg2png gives best results
for pictures that should never be saved as JPEG", but knusperli works well on
normal photographic content, since it only tries to remove blocking artifacts.

> A JPEG encoder quantizes DCT coefficients by rounding coefficients to the
> nearest multiple of the elements of the quantization matrix. For every
> coefficient, there is an interval of values that would round to the same
> multiple. A traditional decoder uses the center of this interval to
> reconstruct the image. Knusperli instead chooses the value in the interval
> that reduces discontinuities at block boundaries. The coefficients that
> Knusperli uses, would have rounded to the same values that are stored in the
> JPEG image.

~~~
enriquto
> Instead of trying to smooth the entire image, it reduces blocking artifacts
> by finding plausible coefficients that reduce discontinuities at block
> boundaries

This is essentially the same thing, but using slightly different regularizers.
In both cases, an image is found that whose jpeg compression is identical to
the given one, by selecting the "most regular" image among the large family of
such images. Jpeg2png minimizes the total variation over the whole image, and
knusperly only along the boundaries of the blocks. The effect is quite similar
at the end.

~~~
Scaevolus
The effects are quite different, since reducing total variation also reduces
texture, which is important to perceived quality. Here's a direct comparison--
note how knusperli retains slightly more detail instead of oversmoothing:
[https://mod.ifies.com/f/200710_lena_decode_comparison.png](https://mod.ifies.com/f/200710_lena_decode_comparison.png)

~~~
enriquto
Good point. Unfortunately, the lena image is quite bad for comparison, there's
almost no texture (except for the blue kerchief). I wonder what would happen
for a heavily textured image; say, white noise. Would knusperly smooth it only
along the block boundaries and leave it alone elsewhere?

------
locofocos
"For pictures that have been reencoded to JPEG multiple times I recommend
shred."
([https://www.gnu.org/software/coreutils/manual/html_node/shre...](https://www.gnu.org/software/coreutils/manual/html_node/shred-
invocation.html))

That's the funniest thing I've read in a while.

~~~
Lanrei
That reminds me of a quote about chlorine trifluoride:

"...It can be kept in some of the ordinary structural metals—steel, copper,
aluminum, etc.—because of the formation of a thin film of insoluble metal
fluoride that protects the bulk of the metal ... If, however, this coat is
melted or scrubbed off, and has no chance to reform, the operator is
confronted with the problem of coping with a metal-fluorine fire. For dealing
with this situation, I have always recommended a good pair of running shoes."

~~~
derefr
I've always wondered if there was something you really _could_ do to
neutralize a reaction that "really wants to happen" like that. Presuming
you're in a well-equipped chem lab and have access to all sorts of non-mundane
substances... maybe you could pour liquid helium on the fire? (It'd splatter
from the Leidenfrost effect, but the flying Helium droplets wouldn't _harm_
anything in any chemical sense, "merely" a physical "being very cold" sense.
So, suit up against frostburn, and move away any glassware to ensure it isn't
shattered by the opposite of unequal heating.)

~~~
kortex
ClF3 will react until there is nothing left to fluorinate. In practice, this
means it'll react with most neutral elements, oxides, and chlorides. Certain
oxides might act to poison the reaction, slowing it down by absorbing free
radicals. Mainly, you wouldn't even want to be close enough to such a reaction
to be able to douse it.

------
ajb
This is a poor-mans version of the deblocking filter which you find in recent
video standards, but jpeg was to early to get it.

The advantage of those over this is that the encoder knows that the decoder is
going to run the filter, and it can use that in its perceptual model to choose
an encoding which looks most like the original _after deblocking_. Video
codecs also use the deblocked version as reference frames. So you see these
artifacts in mpeg-2 video and jpeg, but not more recent ones.

~~~
pornel
I'd call newer formats to have poor-man's deblocking, because they literally
apply blur to blocky images, instead of reconstructing an optimal gradient.

In newer codecs you can notice that sometimes blocks of 8 or 16 pixels get
blurred with a 4px-wide blur, which is barely enough to cover up the edges,
but not enough to mask square-ish shapes underneath.

------
0-_-0
Now we just need an adversarially trained neural network to fill in detail [1]
and we have a 21st century JPEG decoder!

[1] Implemented for a custom compressor:
[https://hific.github.io/](https://hific.github.io/)

~~~
lazyjones
And one day, a law enforcement officer will use it on a bad quality JPEG image
and one of the innocent persons whose photos were used to train the NN will be
arrested.

~~~
kevin_thibedeau
Nexus-6 just needs to get better at hiding from the law.

------
wtallis
This strikes me as conceptually similar to the problem of printing floating
point numbers. A naive algorithm will convert the binary floating point to
decimal and print that out, with potentially many trailing digits that are
more or less garbage. The more human-friendly algorithms figure out the
shortest/simplest decimal that would be rounded to the same binary floating
point quantity, and prints that instead.

~~~
Kednicma
You and the rest of this thread's participants may be interested to know that
the problem was solved a while ago; you want the "Dragon4" algorithm from "How
to Print Floating-Point Numbers Accurately" [0]. Indeed, the problem is
difficult, and having implemented the algorithm, I can confirm that it has
earned its reputation. Numeric algorithms are hard and the only reward is
accuracy.

[0] [https://lists.nongnu.org/archive/html/gcl-
devel/2012-10/pdfk...](https://lists.nongnu.org/archive/html/gcl-
devel/2012-10/pdfkieTlklRzN.pdf) or use your search engine

~~~
nextaccountic
Nowadays I think you want Ryu

[https://www.reddit.com/r/ProgrammingLanguages/comments/930pc...](https://www.reddit.com/r/ProgrammingLanguages/comments/930pch/ryu_a_new_algorithm_to_quickly_convert_floating/)

------
webmaven
_" jpeg2png gives best results for pictures that should never be saved as
JPEG. Examples are charts, logo's, and cartoon-style digital drawings."_

Interesting value proposition.

There may be equivalents for other misused file types/formats. The obvious one
is the inverse case of photos saved as GIFs, but I imagine audio and text (eg.
PDF malapropisms?) are possible too.

~~~
jandrese
I just tried it with a text heavy screenshot, a common scenario where the JPEG
encoding produces a poor result with lots of speckle around each letter.

The result is very good. The speckling is effectively eliminated without
smearing the letters.

JPEG Encoded, 25% quality: [http://jubei.ceyah.org/overcompressed/crop-
jpg.png](http://jubei.ceyah.org/overcompressed/crop-jpg.png)

JPEG2PNG: [http://jubei.ceyah.org/overcompressed/crop-
converted.png](http://jubei.ceyah.org/overcompressed/crop-converted.png)

~~~
indigo945
I find the text on the reference decoded image easier to read, despite the
colorful speckles surrounding the words. JPEG2PNG doesn't smear the letters a
lot, but does appear to slightly deform them. Consider the leading "a" in "add
license, readme", whose upper half seems to be closer together after the
JPEG2PNG run.

However, now I'm wondering whether my impression that the reference decoded
text is easier to read is a form of overfitting. I have spent the past 20
years reading low-quality JPEG screenshots on the internet, maybe the
perceptual centers of my brain have developed an unblocking algorithm?

~~~
stordoff
That's a really interesting suggestion. I'm seeing something similar - the
jpeg2png output looks cleaner and at first glance more pleasant to look at,
but the reference seems both sharper and easier to read. It feels somehow more
easier to ignore the more obvious flaws in the reference, and being used to
them sounds like a plausible explanation to my layman ear.

------
crazygringo
This is really cool, but in other comments I'm reading that good JPEG encoders
and decoders _depend on knowing the other_.

That you can design a better JPEG encoder for a known reference decoder (which
decoded version best matches the original), and you can likewise design a
better JPEG decoder for a known reference encoder.

But now that there are _both_ encoders _and_ decoders trying to be more
efficient and non-blocky respectively... it feels like a mess. Even more so
because a decoder never knows which encoder was used, and an encoder never
knows which decoder.

Is there any solution to this mess? Is there a best practice here? Since most
JPEG's are probably viewed on the web, do all browsers share the same decoder,
and so effort is best spent on the encoder? Or even while encoder improvements
are made specifically for the algorithms browsers use, can improvements to
decoders still be made that remain improvements, instead of e.g. "overfitting"
and actually making things worse?

This just feels like a bit of an "improvements race" on both sides which
leaves me a little dizzy in trying to understand it...

~~~
IgorPartola
The obvious improvement is to use a better format than JPEG. Maybe something
that can include information about the encoding process so the decoder can use
that instead of guessing. Or maybe even a non-lossy format so we can focus
more on data compression and less on visual compression.

------
ris
Interesting project, but its name sells it incredibly short.

------
karmakaze
For the first image I much prefer the jpeg decoded image to the de-noised one.
The de-noised one just looks like it was sent through a low-pass/median filter
and loses contrast, detail, and texture. If you flip between the full-sized
images the jpeg2png looks like vaseline on the lens.

The naming is also odd. I don't imagine it would do well with text (which are
the sorts of images I associate with png) that was jpeg encoded then decoded
with it.

~~~
KarlKemp
It says it is _specifically targeted_ at images that should have been png in
the first place.

------
etaioinshrdlu
The interesting thing is that advanced _encoders_ might create JPEGs that
produce odd results with this decoder.

~~~
a_e_k
Yes, this is definitely an interesting question. Earlier this year I
implemented a version of trellis quantization for some compression experiments
that I've been tinkering with in my spare time. My code considers more
possibilities than just rounding to the nearest quantization step when it
deems that the bit rate savings in after entropy encoding may be worth the
additional image loss (e.g., it might even completely zero a coefficient).
That would violate this decoder's assumption that the original DCT coefficient
must have been within a limited range around the quantization step.

I know that mozjpeg [1] features trellis quantization for JPEG encoding. I
wonder how this decoder would do with that?

[1] [https://github.com/mozilla/mozjpeg](https://github.com/mozilla/mozjpeg)

~~~
ache7
It goes fine with mozjpeg. I tested a little, didn't find any problems.

------
ur-whale
Surprised no one has tried to build a GANN to do this (remove JPEG artifacts).

Denoiser in ray tracing are quite a success, and this strikes me as quite
similar.

In fact, now that I think of it, you probably don't even need a GANN, since
you can create an infinite training set for quasi-free.

~~~
ache7
I've seen these projects on github. I don't remember the names, only that
waifu2x upscaler has an NN filter for JPEG artifacts.

------
pents90
It's long past time to retire Lena as the canonical test image in CG.

~~~
crazygringo
Seriously. People wonder why there aren't enough women in CS.

Things like the canonical test image being a sexy "come hither" look, coming
from a Playboy centerfold, definitely don't help.

And it's not enough to say "well the rest of the centerfold isn't shown". Come
on. We all know the source of the image.

And "tradition" is far less important than CS being a welcoming environment
for all people.

Computer science is a profession. Let's use images that are appropriate for a
professional setting.

~~~
nisuni
As a woman in CS I couldn’t care less about Lena.

Let’s focus on real things, not on this phony issues!

~~~
brnt
Passing an indifference test should however hardly be the right bar to clear
for a field of research.

~~~
nisuni
My point is: people (often, males) like to speak in the name of all women when
it comes to Lena.

It’s offensive, it keeps women out of CS, it’s a symbol of the patriarchy,
etc...

As woman, I am saying this is not the case, at least not for THIS woman: I’ve
never felt excluded because of Lena, my CS studies were not impeded by Lena in
any way, I have actually used Lena’s image for a small project.

Of course I would accept someone saying that the image offends someone (better
if this point is substantiated by evidence) but I absolutely reject people
speaking in the name of all women, an saying that Lena offends all women. This
is not true!

~~~
brnt
Finding a person not offended does not mean an image does not contribute to a
particular environment, or is an example of it. The image need not be
offensive to most or even many for it to be true.

Also, be careful not to attack a strawman: the issue is of course not _just_
the image.

As a man I experienced plenty of unpleasant conversation about women,
conversation which might be referred to as men-talk or some such. Also, I'm
not competitive at all, while many men would identify that as manly behavior.
One of the few women in my university program confided that this (usually
unfounded) self-assuredness was very annoying and a huge turn-off, a feeling
we both shared, but would have been regarded as an essential ingredient to
participating in that program.

Of course, such issues are also present in the reverse. My wife had similar
conversations but about men amongst some female colleagues. Fortunately, we've
met enough people to not have to settle for such juvenile views/conversations.
I can absolutely see how a woman would find it very difficult to be
comfortable in such an environment however, when I didn't even feel part of
that mildly macho culture.

Just because something is 'anti-women', does not mean it can only bother
women. Our cultures associate many things to gender, which in my view is a
leftover of centuries past and we would do well to remove that sort of
association. Maybe then, a picture of Lena wouldn't represent a (sub)culture
so accurately and be therefore an indicator of the problem.

~~~
nisuni
I have my evidence that Lena is not harmful to women, at least to this woman.
It’s one data point, but better than zero.

You are making a much stronger claim without evidence. How is Lena harmful?
How many women did Lena drive out of CS?

Do you have some proof, some data, besides neo-Marxist crap such a
intersectionalized mysoginistic oppression?

It reminds me of that video that showed that college students are outraged by
stereotypical Mexican Halloween costumes, but actual Mexicans are fine with
it.

Are you fighting for women, or for your ego?

~~~
brnt
If you disqualify my experience but hold your own as a valid data point, I
guess we are done.

