
Color Night Vision (2016) [video] - Tomte
http://kottke.org/17/04/incredible-low-light-camera-turns-night-into-day
======
qume
There is much discussion here regarding quantum efficiency (QE). Keep in mind
that figures for sensors are generally _peak_ QE for a given colour filter
array element. These can be quite high like 60-70%.

But - this is an 'area under the graph' issue. While it may peak at 60%, it
can also fall off quickly and be much less efficient as the wavelength moves
away from the peak for say red/green/blue.

From what I can tell from the tacky promo videos, the sensor is very sensitive
for each colour over a wide range of wavelengths, probably from ultraviolet
right up to 1200nm. That's a lot more photons being measured in any case, but
especially at night.

Their use of the word 'broadband' sums it up. It's more sensitive over a much
larger range of frequencies.

I also wouldn't be surprised if they are using a colour filter array with not
only R/G/B but perhaps R/G/B/none or even R/IR/G/B/none. The no filter bit
bringing in the high broadband sensitivity with the other pixels providing
colour - don't need nearly as many of those.

Edit - one remarkable thing for me is based on the rough size of the sensor
and the depth of field in the videos, this isn't using a lens much more than
about f/2.4. You'd think it would be f/1.4 or thereabouts to get way more
light but there is far too much DoF for that.

------
amluto
It would be interesting to see how this compares to theoretical limits. At a
given brightness and collecting area, you get (with lossless optics) a certain
number of photons per pixel per unit time. Unless your sensor does
extraordinarily unlikely quantum stuff, at best it counts photons with some
noise. The unavoidable limit is "shot noise": the number of photons in a given
time is Poisson distributed, giving you noise according to the Poisson
distribution.

At nonzero temperature, you have the further problem that your sensor has
thermally excited electrons, which aren't necessarily a problem AFAIK. More
importantly, the sensor glows. If the sensor registers many of its own emitted
photons, you get lots of thermal noise.

Good low noise amplifiers for RF that are well matched to their antennas can
avoid amplifying their own thermal emissions. I don't know how well CCDs can
do at this.

Given that this is a military device, I'd assume the sensor is chilled.

~~~
grondilu
Theoretical limits are probably very high. For one, collecting area can be
pretty large. Obviously you don't want a square meter lens, but a foot-wide
one would probably be acceptable for a specialized camera.

Second, nocturnal animals like howls see very well in the dark. Granted, they
don't see colors but they still show that in theory seeing in the night is
physically impossible.

PS. regarding the collecting area, something I've been wondering for a bit :
isn't the number of photons a lens can capture proportional to the square of
the surface and not the surface itself? I know that sounds counter-intuitive
but with interference, quantum mechanics and stuff, I checked the math once
and I could not see where I was wrong.

~~~
greglindahl
I don't know what you mean by "surface", but the number of photos collected
depends on the _area_ of the lens. No quantum mechanics or interference
involved.

~~~
grondilu
I did mean area. And I thought the number of photons was proportional to the
square of the area because the probability amplitude is proportional to the
area. Therefore the probability should be proportional to the square of the
area, shoudn't it?

It also seemed to make sense to me otherwise we would not need to build large
telescopes, we could just build lots of small ones and fuse the images.

~~~
davrosthedalek
No, the probability is proportional to the area, not the amplitude. If a
sensor of size A sees N photons, a sensor of the same size next to it sees
another N. Fusing these two sensors together see 2N, not 4N. Otherwise, you
violate energy/momentum conservation.

~~~
TeMPOraL
I love this kind of explanations. You can skip a lot of hard math with a
simple counterexample that shows the absurdity of proposed theory.

------
joshvm
Better video from their website comparing to other cameras:

[https://www.youtube.com/watch?time_continue=328&v=c_0s06ORTk...](https://www.youtube.com/watch?time_continue=328&v=c_0s06ORTkY)

Anecdotal evidence on the internet suggests it's around £6k, but that seems
far too low.

~~~
edgartaor
What's the point of SWIR?

~~~
IIIIIIIIIIII
There was a discussion in another thread (forgot where I saw it) where this
exact same topic and video was discussed, and the summary is they seem to have
completely messed up their SWIR setup. Well, it's pretty apparent, that
technology cannot be as bad as what they show (showing nothing at all) or it
would not exist.

Here is SWIR used to see through smoke:

\-
[https://www.youtube.com/watch?v=GUUIgBut8RU](https://www.youtube.com/watch?v=GUUIgBut8RU)

\- [https://www.youtube.com/watch?v=keRxJg-
gjLE](https://www.youtube.com/watch?v=keRxJg-gjLE)

\- LWIR, MWIR, SWIR (long/medium/short wavelength infrared):
[https://www.youtube.com/watch?v=3pfzO26a21c](https://www.youtube.com/watch?v=3pfzO26a21c)
and
[https://www.youtube.com/watch?v=iV4hNzDJbF0](https://www.youtube.com/watch?v=iV4hNzDJbF0)

\- Overview SWIR cameras and applications:
[https://www.youtube.com/watch?v=Vi0x7D5u7Dk](https://www.youtube.com/watch?v=Vi0x7D5u7Dk)

Use cases for such cameras go way beyond night vision, see @5:13 in the last
video for a list.

------
rl3
One would think with all the money the military throws into imaging technology
that they would already have this.

For Special Operations use, it'd be nifty to have this technology digitally
composited in real-time with MWIR imaging on the same wearable device. Base
layer could be image intensification with this tech, then overlay any pixels
from the MWIR layer above _n_ temperature, and blend it at ~33% opacity.
Enough to give an enemy a nice warm glow while still being able to see the
expression on their face. Could even have specially made flashbangs that
transmit an expected detonation timestamp to the goggles so they know to drop
frames or otherwise aggressively filter the image.

Add some active hearing protection with sensitivity that far exceeds human
hearing (obviously with tons of filtering/processing), and you're talking a
soldier with truly superhuman senses.

That's not to mention active acoustic or EM mapping techniques so the user can
see through walls. I mean, USSOCOM is already fast-tracking an "Iron Man"
suit, so I don't see why they wouldn't want to replicate Batman's vision while
they're at it.

~~~
jamessb
> For Special Operations use, it'd be nifty to have this technology digitally
> composited in real-time with MWIR imaging on the same wearable device. Base
> layer could be image intensification with this tech, then overlay any pixels
> from the MWIR layer above n temperature, and blend it at ~33% opacity.
> Enough to give an enemy a nice warm glow while still being able to see the
> expression on their face

See the two images of an SUV (the top one has caption "Color Low Light Night
Vision Midnight Image Fused with thermal infrared RED ALERT FLIR Image") on
the product page [1].

[1]: [https://www.x20.org/color-night-vision/](https://www.x20.org/color-
night-vision/)

~~~
rl3
Nice. Any idea if it's digitally composited? I've seen image
intensification/IR fusion implemented before with analog techniques and wasn't
impressed; resulting image had a high amount of jitter and felt shoddy.
Perhaps it was just that manufacturer's implementation.

------
telesilla
Can someone wake me up in the future? When we have digital eyes, and we can
walk around at night as if it were day except the stars would be glittering.
Sometimes, I'm so sad to know I'll not live to know these things and I'm
incredibly envious of future generations.

~~~
avdempsey
Be proud to know you helped hold the world together long enough for our
descendants to get there.

~~~
Roritharr
I don't think there's a lot of people this applies to and the world doesnt
currently feel like it's being held together at all.

It rather feels like it's rapidly coming apart, but that's seemingly okay for
many people because you can just have a short position...

~~~
dwaltrip
Try taking a break from reading/watching the news for a few weeks.

------
akurilin
You can get somewhat close to that with a Sony a7s these days:
[https://vimeo.com/105690274](https://vimeo.com/105690274)

~~~
jacquesm
Impressive, for sure, but check the colors in the linked video as well (the
one you linked is mostly grayish) as the fact that it was just starlight that
made it work.

~~~
jychang
That grayishness is due to the S-Log tonal compression, not the sensor.

------
Silhouette
They list a lot of potentially useful applications on the product's own web
site. I wonder how long it will take for this sort of technology to be
commercially viable for things like night vision driving aids. High-end
executive cars have started to include night vision cameras now, but they're
typically monochrome, small-screen affairs. I would think that projecting an
image of this sort of clarity onto some sort of large windscreen HUD would be
a huge benefit to road safety at night. Of course, if actually useful self-
driving cars have taken over long before it's cost-effective to include a
camera like this in regular vehicles, it's less interesting from that
particular point of view.

------
colordrops
Two thoughts come to mind:

1\. It would be nice to see a split screen against a normal view of the scene
as it would be seen by the typical naked eye.

2\. Our light pollution must SUCK for nocturnal animals that see well at
night.

~~~
imaginenore
It's likely pitch black to the human eye, except for the stars. We have Sony
A7s, which goes to ISO 409000, and it already is close to night vision, way
more sensitive that my eyes, especially paired with a f1.4 lens.

This camera goes to ISO 5 million, which is awesome.

------
jacquesm
That really is incredible. I wonder how they keep the noise level down and if
the imaging hardware has to be chilled and if so how far down. Pity there is
no image of the camera (and its support system), I'm really curious how large
the whole package is. It could be anything from hand-held to 'umbilical to a
truck' sized.

Watch when the camera tilts upwards and you see all the stars.

~~~
anamexis
There's plenty of pictures of the camera and more info at the linked site:
[https://www.x20.org/color-night-vision/](https://www.x20.org/color-night-
vision/)

~~~
jacquesm
What an amazingly small package. Thank you, I totally missed the link.

So, if I get this right this uses residual light from the stars and other
sources to do the imaging, and judging by the speed of the update it's doing
that with so short an exposure that you'd never even realize that if it wasn't
mentioned.

That's one hell of a sensor, and probably quite a bit of on-board post-
processing.

There is a starlight video on that page:

[https://youtu.be/RbD9E6YyLbA](https://youtu.be/RbD9E6YyLbA)

~~~
cr0sh
They also mention something about the CCD being "large pixel pitch" or
something to that nature, to gather more light per imaging element. This would
explain the size of the camera, because we know that a 10 mega-pixel camera
can be small enough for a phone. Make the pixels much bigger, and the whole
array much larger (instead of something the size of a pencil eraser, for
instance, make it many times larger - perhaps 400mm per side or something).

~~~
jacquesm
It's not that big actually:

[https://www.x20.org/wp-content/uploads/2015/09/true-color-
ni...](https://www.x20.org/wp-content/uploads/2015/09/true-color-night-vision-
module-camera-1024x556.jpg)

Maybe 6" horizontal and 4" vertical and the sensor smaller than that.

[https://www.x20.org/wp-
content/uploads/2015/09/IMG_0041.jpg](https://www.x20.org/wp-
content/uploads/2015/09/IMG_0041.jpg)

Judging by the pitch of the pads at the bottom that's about 2" wide and 1.5"
tall.

~~~
Gravityloss
They mention it's large format so over 100 mm sensor.
[https://en.m.wikipedia.org/wiki/Large_format](https://en.m.wikipedia.org/wiki/Large_format)

~~~
frostburg
It's not actually large format. Fabrication would be impossibly expensive.

~~~
Gravityloss
You could have multiple 35 mm format sensors, a 3x3 grid for example.

------
19eightyfour
That is beautiful.

If they can increase the dynamic range to bring detail to the highlights it is
basically perfect.

I've never seen a valley look like that with a blue sky above with stars in
it. Truly incredible.

The 5M ISO rating is pretty funny. 1/40 f1.2 ISO 5M.

~~~
Animats
The dynamic range is so good that it will work in daylight. They even point it
directly at the sun. There's some blooming, but not much.

~~~
19eightyfour
Yes I was impressed by the sun shots. That was really incredible.

However at night the headlight fan on the road is detail-free. This could be
improved.

That was what I meant saying the DR could improve.

Thinking about it some more from a technical perspective ( and I know
absolutely nothing about the technology in this field ) it seems the camera
really suffers from strong point light sources ( headlights, streetlamps, so
on ), and I know that these sources often have specific known spectral
characteristics ( color temperature and so on ) -- and I'm wondering if a way
to improve the performance here is to somehow detect the spectral signatures
of a range of such light sources, and somehow cap the contribution these can
make to the brightness in the image...so preventing these highlights from
burning out the details, and increasing the apparent dynamic range -- even
when the camera is pointed down road ahead of a vehicle with headlights on.

------
cameldrv
They say it's hybrid IR-visible. I wonder if the trick is to use IR as the
luma and then chroma-subsample by having giant pixels to catch lots of
photons.

------
floatboth
"an effective ISO rating of 5,000,000"

Holy shit, my Canon 600D is pretty bad at 2500, goes to crap at 3200, and 6400
is an absolute noise mess…

~~~
Nrsolis
The Sony A7S Mk II can do an effective ISO close to 4096000.

[http://www.kenrockwell.com/sony/a7s-ii.htm](http://www.kenrockwell.com/sony/a7s-ii.htm)

EDIT: fixed the actual ISO.

~~~
sdp
That link says the ISO goes to 409,600, not 4,096,000.

------
dreamcompiler
This is an amazing device. I've taken night photos that look like frames of
this movie on my digital camera, but they require a 60-second exposure and a
tripod, and they're -- still frames.

~~~
doubleunplussed
With that knowledge, we can do a back of the envelope calculation for what
kind of lens it would take to make your camera take a video at 10 frames per
second that looked the same as its 60 second exposures.

Basically we need 600 times more light. Square root of 600 is about 25. So
you'd need a lens system with an initial aperture 25 times larger in diameter
than the one you used to do the long exposures.

Big lens, but if you want to brute force this kind of problem, that's the way
to do it!

~~~
greglindahl
Cheaper to use a telescope.

It's easily possible to grind your own big telescope mirror. A big lens is
like trying to build a big refracting telescope -- extremely expensive. The
largest refracting telescope ever made is 100cm (40 inches). Mirrors are up to
10 meters.

~~~
tigershark
Most of the telescopes are extremely dark compared to "normal" camera lenses.
The brightest that I remember was the so called "camera Schmidt" with f/2.8.
Normal Newtonians usually are in the f/4-f/5 range. The cassegrain and derived
are around f/10\. The big aperture in the telescopes is needed for the angular
resolution, certainly not for the brightness.

~~~
greglindahl
The general physics is that the more photons there are (big aperture and/or
bright source), the more opportunity you have to have high resolution.

~~~
tigershark
Yes, the resolution is given by the diffraction limit of the optics that is
dependent by the diameter. But resolution is different from brightness. If you
tried to take a video of the same scene with a telescope chances are that the
final result would have been darker, _even using the same camera_. Granted
that it would have had a much much better resolution (but a much much smaller
FOV)

~~~
greglindahl
So you think astronomers are too stupid to trade resolution for brightness?
Hint: downsample your image: you won't lose too much as long as the dark
current isn't too large.

~~~
tigershark
They have completely different use cases. In one case you are making a video
of a scene with a FOV of several tens of degrees. In the other you are
observing objects with a maximum size of a couple of primes and with features
measuring infinitely small fractions of arc seconds.

------
caublestone
My brother in law experimented with this camera a few years back on family
portraits. The camera picks up a lot of "dark" details. Skin displays pale and
veins are very defined. My nieces called it "the vampire camera".

~~~
sydd
Can you share any photos?

~~~
runeks
Here's a day time shot, which exhibits a bit of this "HDR"-effect:
[https://youtu.be/Y2zfG6kY7Ns](https://youtu.be/Y2zfG6kY7Ns)

~~~
BuffaloBagel
hey! seeder siddy, utarh.

------
fpoling
There are far infrared cameras that capture thermal radiation from 9-15 NM
band. They nicely allow to see in complete darkness. They do not use CCD but
rather microbalometers.

But they are expensive. 640x480 can cost over 10000 USD and cameras with
smaller resolution like those used in high-end cars still cost over thousand
USD.

~~~
fest
FLIR recently started selling their low-resolution uncooled sensors in small
quantities: * 80x60 resolution for $200 * 160x120 resolution for $240 (doesn't
seem to be in stock on Digikey though).

I've used the first one- resolution is quite low, but a few years ago you
couldn't even buy such sensors as a hobbyist (example image:
[http://faili.wot.lv/tmp/IMG_20160320_163431.jpg](http://faili.wot.lv/tmp/IMG_20160320_163431.jpg))

These are the same sensors which are used in FLIR smartphone-attached thermal
cameras.

------
teh_klev
Direct link to manufacturer or supplier:

[https://www.x20.org/color-night-vision/](https://www.x20.org/color-night-
vision/)

------
batbomb
So maybe Peltier on the sensor, heat sink attached to body, body hermetically
sealed. Sensors probably tested for best noise quality (probably a really low
yield on that).

------
drenvuk
This is incredibly cool. You can even see how other sources of light actually
have an effect on the environment as if they were their own suns.

~~~
jacquesm
Shutter time variation or mechanical iris closing depending on the amount of
incident light to change the exposure?

Otherwise with any direct source of light in an image it would immediately
overexpose (in some low light cameras that could even damage the sensor). It's
super impressive.

------
peteretep
Put one of these on a drone and you'll break a lot of people's assumptions
about their privacy

------
anovikov
Why can we see terrain much better than with a naked eye, but stars definitely
worse? There isn't even a hint of Mikly Way, which should be easily visible
with naked eye in a desert with zero light pollution.

Or this was shot during full moon, carefully avoiding it getting in sight?
Then it is not much better than (fully adapted) naked eye.

------
kator
Side by side comparison:
[https://www.youtube.com/watch?v=c_0s06ORTkY](https://www.youtube.com/watch?v=c_0s06ORTkY)

------
Cieplak
I wonder what the sensor is made of. I would bet on there being a fair bit of
Germanium in there.

PS: probably wrong about that, silicon's band gap is more suited to optical
spectrum, even though germanium has more electron mobility. I'm speculating
now that they're using avalanche photodiodes.

[https://en.m.wikipedia.org/wiki/Avalanche_photodiode](https://en.m.wikipedia.org/wiki/Avalanche_photodiode)

------
copperx
I've dreamed of such a camera for decades. I thought the technology was at
least 10+ years away. This is what science fiction is made of.

------
lutusp
Someone should contact this company and volunteer to redesign their website
([https://www.x20.org](https://www.x20.org)). They should also be told that
"complimentary" and "complementary" don't mean the same thing.

They have a great product, unfortunately presented on a terrible website.

~~~
giarc
Why should someone volunteer to redesign their site. They are a private
company, they can pay someone to do that. This isn't a charity.

~~~
lutusp
> Why should someone volunteer to redesign their site.

I didn't mean volunteer without pay. You must be aware that, for missions so
unpleasant as to be sanity-threatening, the military only accepts volunteers.
Same idea.

------
ChuckMcM
Here is the manufacturer's web site: [https://www.x20.org/color-night-
vision/](https://www.x20.org/color-night-vision/)

There is a 'shoot out' video on that page which compares themselves to other
night vision technologies. Pretty impressive demo.

------
AnimalMuppet
It occurs to me that this technology could do absolutely amazing things as the
imager for a space telescope...

------
breatheoften
Is that red rocks (just outside of Las Vegas). There's a lot of man made light
sources that really scatter light pretty far and in a lot of directions (the
Luxor spotlight comes to mind). I wonder if that could have an effect on this
camera's performance.

------
jsjohnst
Here's the manufacturer website on it: [https://www.x20.org/color-night-
vision/](https://www.x20.org/color-night-vision/)

------
jbrambleDC
I want to know what this means for observational astronomy. Can we put this in
the eyepiece of a telescope and discern features in nebulae that otherwise
look like gray blobs to unaided vision

~~~
vortico
Telescope cameras are already way past this. This camera has the advantage of
being cheaper and smaller.

~~~
tigershark
Not at all. Even when cooled well below zero and without any Bayer filter I
never saw something like this with less than 1/10s of exposure.

~~~
vortico
I suppose it's a matter of resolution. ARCONS is an IR photon counter with
44x46=2024 pixels. I don't know how that would correspond with a camera's ISO,
but you can't get much better than single photons. I'd imagine it's useless to
install in your ground telescope, but perhaps in a couple decades, the
resolution will be scaled up to compete with current optics imaging methods.

[https://arxiv.org/abs/1306.4674](https://arxiv.org/abs/1306.4674)

~~~
tigershark
Photon counters were not what I had in mind when you were speaking about
astronomy cameras. I remember that around 20 years ago we had 1/100th of
magnitude resolution with photon counters compared to 1/10th of the CCDs (SBIG
&co.)

------
interfixus
Why is the night sky blue? Is that really scattered starlight?

~~~
Hydraulix989
I can't think of any other light source.

------
bsenftner
Has no one considered a neural net post processor which has been trained on
daylight views? Seems like an obvious method, given Hacker News...

------
nnain
What a quandary: We see military weapons technology put to terrible use all
the time, and yet, so much technology shows up in (US) military use first.

~~~
tripzilch
I don't see the quandary. The US spends trillions (or something) on military
R&D. There's nothing special about military R&D per se that make it produce
better and greater technology than other reasons to do research. In fact,
because of the particular focus, it probably produces quite a bit of
technology that doesn't even have _any_ non-terrible use, even if you're
really creative about it.

It's a numbers game. If you throw more spaghetti at some particular wall than
anybody else or any other wall, then obviously that's where most of it will
stick ...

It's mostly that humans are real easy to convince to hate and want to kill
eachother, as long as the numbers are large enough you can forget they're
individuals. So obviously the other dominant lifeforms on this planet (super-
organisms, in this case multinational corporations) exploit this loophole for
control and power.

------
samstave
ELI5 what an iso of 5MM means?

~~~
hexane360
Basically it's a measure of how much light gets into the camera, in
lumens*s/m^2. It's defines the practical limit for how fast you can run a
video camera and get decent exposure.

~~~
samstave
How do you measure a lumen?

~~~
shouldbworking
Lumens are very misleading as they're entirely based on human vision
sensitivity.
[https://en.m.wikipedia.org/wiki/Luminosity_function](https://en.m.wikipedia.org/wiki/Luminosity_function)

Naturally, the closer to the 555nm green peak, and the more monochromatic your
light source, the brighter it appears to be. This is a part of why night
vision systems are green.

You measure a lumen by taking the area of your light source spectrogram in
respect to the luminosity function.

I did this years ago with some manufacturers LED light output curves to
reverse engineer electrical->photon efficiency vs cost from their data sheets.
The main purpose was to figure out how close we were to LED lighting taking
over.

I did that research back in 2012 and predicted 2015 as the point where LED was
more efficient and cheaper than CFL but I was off by a year or so. Hilariously
that paper got me a C- because it was only tangentially related to the
teacher's pet subject and never saw the light of day.

------
faragon
Is this real? :-O

------
egypturnash
Their website is a thing of beauty. It's straight out of the Timecube school
of design. [https://www.x20.org/color-night-
vision/](https://www.x20.org/color-night-vision/)

