
Extreme imaging using cell phones: SeeInTheDark [pdf] - jwise0
http://graphics.stanford.edu/talks/seeinthedark-public-15sep16.key.pdf
======
notlisted
Nice PDF. Some cool new stuff, smart decisions & remarks (noise, hotpixels,
matching frames, etc).

If you have an iOS device, check out the app Average Cam Pro[1]. I've had it
on my iPad for several years now. It does an awesome job taking noise-free,
detailed pictures (as long as you don't hold it in your hand) by taking up to
100 pictures and merging them. You can adjust exposure etc. after the fact.
Great stuff.

I have yet to find a good replacement for my Android phone. The closest thing
to [1] is Multiple Exposure Camera [2] which has a terrible interface and no
exposure setting, but some other nifty settings to remove moving objects etc.
(note: just don't use the 0s interval or it crashes)

[1] [https://itunes.apple.com/us/app/average-camera-
pro/id4155778...](https://itunes.apple.com/us/app/average-camera-
pro/id415577873?mt=8)

[2]
[https://play.google.com/store/apps/details?id=com.conslazy.m...](https://play.google.com/store/apps/details?id=com.conslazy.multipleexposurecamera&hl=en)

~~~
ricw
I just installed average camera pro. Very blurry. Not useful for indoor
photography. Does not allow for adjustment of the shutter speed. Don't buy.

It's Nothing like what the paper describes.

~~~
notlisted
Operator error. Like I said, it's not for handheld use (the only way to get
blurry shots).

Correct way:

\- Settings: Set the number of shots to 32 or higher. 1s interval. 3s pre-
start timer

\- Point at scene. Lean it against an object (don't touch/move) or mount on a
tripod.

\- Select exposure/focus/whitebalance with longpress (I select a light part of
the scene, exposure/gain can be adjusted upward afterwards).

\- Lock exposure/focus/whitebalance (could be an L in the middle, or text on
your screen left, boldface is locked in)

\- Shoot.

\- After it's done (don't touch!) adjust the exposure if needed.

====

If you want even higher quality images, you can subtract banding noise (also
mentioned in the PDF by the way).

In order to get the banding noise image for your camera:

\- set shots to 128, delay 5 seconds

\- point lens at light source (e.g. on table, don't move)

\- hit shoot, put piece of white paper over the lens

\- wait ~2mins (till it stops shooting).

\- Select N in middle, "save image as noise fingerprint".

Then take another shot of a scene. Once final pic shows, hit N, now say
"remove banding noise".

------
ChuckMcM
That was a great read, back in the early 80's at the Image Processing Lab at
USC there was a project to turn, what was then, a new "CCD" imager into
something that could collect images that were useful for image analysis. At
the time these "TV" cameras were equivalent to a really low end web camera.
What we came up with was basically the first half of this talk, taking frame
after frame and integrating it into the frame buffer with a filter function.
Sad that I can't find the technical reports in Google Books.

------
andmarios
Very impressive! I have an app on my android that follows the same basic idea
(accumulate frames) but doesn't come anywhere near this.

Unfortunately the author, in a reply to a youtube comment, stated that this is
work he did for Google and doesn't know if the company has any plans for this.

~~~
puzzle
He's Marc Levoy and he knows Google's plans very well; he just can't talk
about them on YouTube, since it's the stuff that gets announced on stage at
I/O and similar events.

------
nullc
There is an old free software package ALE (
[https://unix4lyfe.org/ale/](https://unix4lyfe.org/ale/) ) that does some of
the processing like this. It was much easier to get working if you had linear
light images... and registration of really noisy images was quite hard on it.

------
nmstoker
Here's a (v short) video by the paper's author showing the app in action with
there manuscript that features early on in the paper:
[https://youtu.be/S7lbnMd56Ys](https://youtu.be/S7lbnMd56Ys)

------
br1n0
In php is quite simple to merge images by pixel, I also tried to delete moving
object
[https://github.com/br1n0/phpPhotosMerger](https://github.com/br1n0/phpPhotosMerger)

~~~
tambourine_man
Nice job

------
ricw
Is this what makes the google pixel phone camera so outstanding? it would
explain why a technically inferior camera (slower aperture, no optical image
stabilisation) outperforms both apple and Samsung flagship phones.

------
mparlane
Mirror [85 MB]: [https://dl.dropboxusercontent.com/u/9910153/seeinthedark-
pub...](https://dl.dropboxusercontent.com/u/9910153/seeinthedark-
public-15sep16.key.pdf)

~~~
jjcm
Another mirror in case it goes over the dropbox limit:
[http://files.jjcm.org/seeinthedark-
public-15sep16.key.pdf](http://files.jjcm.org/seeinthedark-
public-15sep16.key.pdf)

~~~
mparlane
Thanks, it did :)

~~~
logicallee
I just want to say, I only came for the comments, but clicked after seeing
people report so many downloads that it went over your rehosting limit.

The pictures in the PDF are absolutely stunning. I encourage everyone to have
a look.

If anyone needs a quick spoiler (a pair of comparison images I copied from the
PDF into a gallery for you) here you go:

[http://imgur.com/a/THJbf](http://imgur.com/a/THJbf)

------
trhway
Hubble - does it take one long exposure or similarly multiple short ones?

~~~
dandelany
A bit of both. Hubble can have single exposures that are hundreds of seconds
long - but remember it's in low orbit, so eventually the subject will fall
beneath the horizon. Most Hubble observations stitch together exposures from
multiple orbits. Ultra Deep Field is an extreme example which comprises ~a
million seconds of exposure over 400 orbits:
[https://en.m.wikipedia.org/wiki/Hubble_Ultra-
Deep_Field](https://en.m.wikipedia.org/wiki/Hubble_Ultra-Deep_Field)

~~~
xioxox
In astronomy you need multiple exposures to get rid of cosmic rays, which
appear as bright lines or dots. A median filter can be applied to the
exposures. HST uses tools called drizzlepac and multidrizzle to combine
multiple dithered images to better make use of the telescope's native
resolution which would otherwise be degraded by the detector.

------
Aoyagi
>Need long exposure, but cell phones have no shutters.

Mine have... 1020 and 808.

------
dsjoerg
excellent! now someone make an app out of this.

------
hughes
Sounds interesting, too bad the server is dead

