I hate to be a debbie downer but the results don't look particularly great to me. For comparison, see "AffGAN" [1] and "LAPGAN" [2] for better (IMO) super-resolution results using GAN-like techniques. Granted, all three papers are applied in somewhat different settings (different input/output resolutions, different datasets), so direct comparison is difficult.
Are there any turnkey, easy to use software packages that use LAPGAN or AffGAN for image enhancement? Or are these techniques purely in the research realm at this point?
Does research always have to yield a result that's better than any other approach? In my understanding it's worthwhile to pursue different ways of approaching a problem even if some of them don't work as well as others (but you don't know that beforehand).
You're not the only one who feels this way; I wasn't very impressed by the adversarial-network synthesized moving GIFs, that everyone went crazy over a few months ago, either.
Kind of alarming to see this being proposed as an investigative tool. Isn't the entire point of this area of image processing that the network is creating plausible information where there is none? That's great for creating higher-resolution versions of entertainment assets, but it would seem categorically inappropriate for forensic science.
Exactly. It's not extracting information, it is (as the Boston Marathon link says) hallucinating the additional information. It's an artist's impression, not CSI's magic 'enhance'.
> it would seem categorically inappropriate for forensic science
It would be absolutely wrong use, as it substitutes some information that the software has learned before and combined instead of the non-existing one in the original pixels.
If the software was trained with the picture of the innocent person, it would produce it instead of the picture of the really guilty person. It can only guess and only guess based on with what it was trained with.
It's bad enough when people don't notice that they filled in the gaps based on pure speculation and prejudice, it will be even worse when the same happens with the "authority" of a computer.
Maybe there could be a moral obligation for the designers of the training set: trained with the right data, if you say "enhance" once too often the model might overfit into a scene from Star Trek ("after shooting, video footage shows man in command uniform firing at random redshirts, people in medical uniforms miraculously unharmed").
that's also why 1080i effective resolution is more than simply 540 lines of resolution (1080i for a sitcom has a higher effective resolution than 1080i watching a a sporting even, which is also why I think ESPN chose to transmit it 720p, at least for some time, as opposed to 1080i, but could be wrong about that)
Definitely glad to see Waifu2x mentioned here, in the age of increasingly common paywalled full-resolution "imagery" (ahem..), it's a godsend to anyone who isn't using a monitor from the early 00's.
It's also nice to just add a little extra quality to older... imagery. Works surprisingly well for non-2D stuff too.
>Given an image, we wish to produce an image of larger size with significantly more pixels and higher image quality. This is generally known as the Single Image Super-Resolution (SISR) problem. The idea is that with sufficient training data (corresponding pairs of low and high resolution images) we can learn set of filters (i.e. a mapping) that when applied to given image that is not in the training set, will produce a higher resolution version of it, where the learning is preferably low complexity. In our proposed approach, the run-time is more than one to two orders of magnitude faster than the best competing methods currently available, while producing results comparable or better than state-of-the-art.
>A closely related topic is image sharpening and contrast enhancement, i.e., improving the visual quality of a blurry image by amplifying the underlying details (a wide range of frequencies). Our approach additionally includes an extremely efficient way to produce an image that is significantly sharper than the input blurry one, without introducing artifacts such as halos and noise amplification. We illustrate how this effective sharpening algorithm, in addition to being of independent interest, can be used as a pre-processing step to induce the learning of more effective upscaling filters with built-in sharpening and contrast enhancement effect.
Forgive me if I've missed something here, but these where only trained against synthetic images (images that where scaled down using various formula). Due to this, I'd expect this to not work as well as it could on actual images taken by sensors.
Do any datasets even exist where the images are at sensor pixel level?
That way the model would 'know' about imaging effects (I can't think of any specifically mechanical effects that could be in play here right this second) etc?
In fact I wonder if any imaging sensor vendors run R&D trying to come up with novel neural net based debayering approaches - this could be a cheap way of bumping image quality/perceived resolution.
I wonder, could you craft a shader for tree-foliage from this?
Given the Background, and the leave texture + alpha, instead of rasterizing, anti-aliazing and then using z-baked lightsources and probe reflections to light it semi-correctly, what would a neural net implementation look like?
Would you even notice the mistakes in a constant flickering scenery like this?
There's the waifu2x project [1] which also uses neural networks for super resolution. There's also the MPDN extensions project [2] which has various kinds of image scaling methods. It depends what kind of algorithms you want to play with really.
Amazing, but also a bit scary. This will surely be used for retroactive identification from existing photographs, and will be a free gift for authoritarian law enforcement. Of course such extrapolative technologies are subject to challenge, but criminal juries have a tendency to accept forensic claims at face value notwithstanding their actual scientific reliability. It's partly because of this that if I ever found myself on trial for a crime I didn't commit I'd probably waive my right to a jury trial - laypeople are far too easily fooled.
The results are synthesised/generated - there is no way to use this for face recognition from low res images because the result, while plausible, is not real
I wonder if something like this could be used to boost the fidelity of radio signals that are picked up by SDR (software defined radio).
Here are some manual techniques people currently use to hunt signals on SDR.[1] A lot of what they do is visual, and enhanced visual fidelity of potential signals would definitely be a big help, if it worked.
[1] https://openreview.net/pdf?id=S1RP6GLle
[2] https://arxiv.org/abs/1506.05751