
RAISR Sharp Images with Machine Learning - wsxiaoys
https://research.googleblog.com/2016/11/enhance-raisr-sharp-images-with-machine.html
======
romaniv
I would not compare a fancy NN algorithm to Bicubic interpolation, even as an
example. Bicubic sucks. If someone really wants to get a decent high-res
upscale of an image via "conventional" means (like GIMP) they would probably
use something like Lanczos algorithm for upscaling -> Anthropomorphic
smoothing for denoise -> unnsharp mask for clarity. And that's a simple
pipeline, you can go even fanciers with masks, layers and edge-detection
kernels.

~~~
lotyrin
They aren't. They compare to techniques "A+" from R. Timofte, V. De Smet, and
L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-
resolution"
[http://www.vision.ee.ethz.ch/~timofter/ACCV2014_ID820_SUPPLE...](http://www.vision.ee.ethz.ch/~timofter/ACCV2014_ID820_SUPPLEMENTARY/)

and "SRCNN" C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep
convolutional network for image super-resolution”
[https://arxiv.org/abs/1501.00092](https://arxiv.org/abs/1501.00092)

Their result seems not to score as well in quantitative metrics (signal to
noise ratio), but is faster. (And seems to have good qualitative results, in
my opinion.)

~~~
amelius
Indeed, I just skimmed through the paper, and the results don't seem much
better than A+ and SRCNN (even worse in some instances).

~~~
ska
Yes, they aren't notably better, but they are a lot faster. Maybe useful.

I really wish people wouldn't call these algorithms super-resolution, as they
already had a useful meaning.

~~~
TTPrograms
I think superoscillation is a better description of the phenomena in
E&M/optics.

------
lotyrin
Those looking for quantitative comparison of this against other techniques,
it's on page 21 of the paper (
[https://arxiv.org/abs/1606.01299](https://arxiv.org/abs/1606.01299) )

Edit: and to see for yourself in more detail there's
[https://drive.google.com/file/d/0BzCe024Ewz8ab2RKUFVFZGJ4OWc...](https://drive.google.com/file/d/0BzCe024Ewz8ab2RKUFVFZGJ4OWc/view)

------
hatsunearu
[https://github.com/nagadomi/waifu2x](https://github.com/nagadomi/waifu2x)

Here's a previous project that attempted this as well.

[https://arxiv.org/abs/1501.00092](https://arxiv.org/abs/1501.00092)

------
d--b
This is obviously something where machine learning is a great tool, knowing
about the way images usually look, it's easy to imagine deep learning methods
doing a great job at this.

However, an obvious application of this will be to apply these methods to
security image footages, in order to improve the recognizability of the people
shot. You know like in Hollywood movies where the detective asks the geek "can
you improve the details of that picture?" But the general public (and by this
I mean cops, lawyers and jury people) will not understand that the details are
actually made up. The person in the security footage may be unrecognizable
before treatment, but after, it would look as "the most likely person to have
been there."

What could go wrong?

------
amelius
Are there any machine-learning approaches to superresolution which take
context into account? By that I mean filling in details that can only be
guessed by context.

For example, when the algorithm detects that the image is of a field of grass
(but individual blades of grass not discernible), then in the superresolution
output it paints in these blades.

~~~
sillysaurus3
One of my favorites is Neural Enhance: [https://github.com/alexjc/neural-
enhance](https://github.com/alexjc/neural-enhance)

I was playing around with it and the results are quite interesting. It's
pretty much exactly what you describe.

The nice thing is, if you have Docker installed, you can run:

    
    
      docker run --rm -v `pwd`:/ne/input -it alexjc/neural-enhance --help
    

and everything magically works. Compared to other machine learning projects,
neural enhance is probably one of the easiest to get started with.

~~~
amelius
That looks amazing! Thanks.

------
abritinthebay
This is very clever, and an interesting read, but kind of disappointing
results.

They don't look _much_ better than doing say... upsample to twice desired
size, apply basic sharpen/structure filter, downsample to desired size.

A little better? Yes. But some (like the horse) look pretty weak

~~~
IshKebab
They're never going to look as good as you might be expecting from films. That
would require fabrication of information that doesn't really exist.

~~~
londons_explore
The best methods for this, using Generative Adversarial Networks, do in fact
fabricate new realistic-looking information.

I think it would be great to input a blurry version of a cityscape, and have
it add numbers to mailboxes, litter to the streets, cracks to the sidewalk,
etc.

------
plaguuuuuu
I wonder how it compares with Waifu2x

------
sengork
Does this mean we can finally get a form of Zoom-and-enhance seen on CSI?

------
canada_dry
I'm imagining the day when old black and white films and home movies are
restored to be almost crystal clear.

Can't be more than 18mo away by the sounds of it.

~~~
visarga
I was thinking of perfect restorations of old classical music recordings. If
we decode the sound of a violin, we could re-synthesize it in studio quality.

------
kazinator
From a qualitative glance at the images, I believe I could come close to the
same results with some basic filtering, like a gussian blur followed by a
quantization to a smaller number of colors, then a little bit of blur.

The image of the horse head statue looks like an "oilify" effect.

Mind you, a pretty good oilify effect.

This filter is clearly useful independently of what it achieves in relation to
what it "says on the box".

In creative image processing, I just have to like what it does, whatever that
is (and I do).

------
bcheung
Is there an executable form / source for this we can play around with?

