Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: I shot high-res stitched panoramas in Iceland using a thermal camera (petapixel.com)
753 points by dheera on Jan 2, 2021 | hide | past | favorite | 95 comments



Hi everyone, OP here.

I used a Seek RevealPro which outputs TIFF files that contain 32-bit floating point values. I created a script that takes in a Seek TIFF and outputs an 8-bit greyscale integer TIFF:

https://github.com/dheera/iceland-thermal/blob/master/script...

I then panoramically stitched those 8-bit TIFFs together (some with the help of hugin, and some manually with GIMP) and distortion-corrected them as best as I could to a black-and-white visible light image. I also created an artistically-chosen color map for each image. For the final output image, hue and saturation are fully based on thermal data, and lightness is a combination of thermal and visible data. Scripts to generate the final output images, along with the panoramically pre-stitched data, and the color maps I used:

https://github.com/dheera/iceland-thermal

Also, my Instagram, which is mostly astrophotography of late, but I did post thermal images there before and I may do so in the future again if I can find another good set of interesting thermal subjects to do this in!

https://instagram.com/dheeranet/


That photo with the icelandic horses - how did you pull that off with a QVGA-resolution thermal camera + stitching? Animals tend to move...


Yeah so I tried to get that shot about a dozen times on various farms before I finally found some horses that seemed tired, lethargic, and didn't move too fast.

I also had to stand there for a good 15 minutes in the biting cold before they stopped being interested in me and kind of just stood there.

They still occasionally turned their heads though so I quickly snapped my visible light image with a DSLR in one hand on a tripod and scanned over the horse part with the thermal camera in the other, before patiently scanning the rest of the scene. The horses in the distance though still did move, as you can see in the separate thermal/visible images in the repo.

I had to do a similar thing for the erupting geyser. 2 quick thermal shots in rapid succession for the eruption in one hand, DSLR in the other, then scanned the rest of the scene. Took about maybe 20 tries at least.


Icelandic horses are traditionally kept outside all year round, and so will often stand very still for long durations when preserving energy.


Have you tried any other thermal cameras and how did they compare?


Not too many, for personal projects I'm mostly budget-constrained. I tried the Flir C2 but the thermal data was extremely low resolution, and returned it. I also tried a Flir Lepton 2 module since I could use an RPi and some gadgetry to do the panorama scanning and stitching automatically, but it too was low resolution (80x60).

The Lepton 3 seems to be decent at 160x120 though, which is probably good enough for a project like this, so if I were doing this again I'd probably use a Lepton 3 and a 2-axis servo scanner with a RPi, and maybe even combine it with an RPi HQ camera to automate the entire process into a single push button.

The Seek RevealPro is slightly annoying in that it doesn't have programmatic access so I had to press a button to take every piece of the panorama. It's a great general-use device though. I use it when camping or doing astrophotography in remote locations for some peace of mind since I can spot any potential wildlife and people threats from a distance, and I can easily identify whether a parked or seemingly abandoned car has been recently occupied. (My fears may be unfounded, but either way I'm much more at ease when I can see in the dark.)


I was recently looking at the Lepton. I would really like a to make an affordable monocular. It seemed like the image quality was quite low even for the 160. I'm not sure if adding some lenses for magnification would improve them or not. The refresh rate is pretty slow too. I'd love to experiment with it, but $200 is a lot for a project that might not be usable.

The Boson seems like it has great clarity and refresh rate, but at that price I might as well just buy a purpose made device (outside my price range).


I'm curious what kind of thermal camera he's using.

I bought a Therm-App Pro 640x480 a while back and it's quite good - and reasonably priced compared to Flir - along with a github repo for different false color palettes. But what's really weird is it seems to not be available anymore. I'm guessing export restrictions or massive demand for thermal cameras during Covid along with supply chain issues. IIRC it is also 25 frames a second, which is unusually high for thermal cameras due to export restrictions.

Edit: Here's a post on creating panoramic images with therm-app. https://www.flickr.com/groups/therm-app-users/discuss/721576...


I was wondering this too, the author added this info via a reply in the comments (of his site/article , not HN)-

> Author here. The camera was a Seek RevealPro which has 320x240 resolution. The thermal data in the panoramas are stitched together from raw data exported by the camera and represented as color (hue+saturation, as per scale in upper right, represents temperature data).

The brightness channel is piecewise-linearly combined with a corresponding visual image taken with a regular SLR to provide a visual reference of what you're looking at.


I think the ThermApp cameras had the ability to combine a visual photo, but I haven't used mine for quite some time so I may be wrong.


OP here. Seek RevealPro. I added a top-level comment with some more details and links :)


Thanks OP. Really great work on those Iceland pics. It’s an original concept I haven’t seen before - adding thermal data to panoramas of landscapes with geothermal heat. So cool. I’m a bit jealous I didn’t think of it first.

Sounds like you have a science background and field, but I’d encourage you to think of these as art and chat to a few galleries about an exhibition. At the very least it would be fun, and pretty amazing to see big blowups of your own imagery. They may be able to do a virtual exhibition during Covid.


> chat to a few galleries about an exhibition

I would love to! I'm a physics/EE major by training and career-wise I'm focused on ML and robotics, but I love art and especially using art to visualize science in ways that are understandable to the general public. (I'm also doing an astrophotography project to show the visual sizes of various nebulae. Your usual NASA photos don't really provide any sense of scale, and many people have the misconception that you need insane magnification to see them, but there are actually a whole lot of objects that are visually much bigger than the moon, just too dim to see.)

Being an engineer by background my network is mostly engineers -- I don't really have any strong connections with galleries and media, but I'll be on the lookout for them! I welcome intros if anyone would be so kind. :) TIA!


Thank you for all of these, awesome work.


I wonder how much imaging technology has been held back by ITAR.


Looks really nice. I see one for sale in the US. $4k though.


You can buy a FLIR Boson at 640x512 for that price. The cost of thermal cameras is roughly linear with the number of pixels. I realise that's not cheap, but you can get double the resolution (e.g. 4x pixels) for the same money.


Cunning. He took panoramas with thermal camera and simultaneous black and white photos and coloured the b&w photos with the data from the thermal camera. Interesting examples of insulation. I wonder how the result compares to the standard Flir auto blended images. Msx?


This would be better than FLIR's MSX, which generally suffers from a lack of precise alignment of the thermal and optical images, particularly for shots with a lot of depth of field.


Sounds like an opportunity to sell the algo to flir


Beautiful. Eerie. Great work. Thank you for sharing.

Your images look almost like they came out of James Cameron's vivid imagination. If you had told me these were computer-generated images of Pandora at night[a] from the upcoming Avatar sequel, I would have believed it.

[a] E.g., see https://www.google.com/search?tbm=isch&q=avatar%20pandora%20...


I have done a lot of work with high resolution thermal imagers and never seen any images that look like these so I was curious what was going on.

>I also simultaneously shot a black-and-white photo with a regular camera and a wide-angle lens for comparison. I then wrote my own program to color the black-and-white photograph with the actual thermal data, using a false-color scale which you see in the upper-right corner of every image.

Amazing drawings, but these are not actually thermal images.


Pure thermal would have been quite smudged. IMHO the B&W adds really nice context - and it sounds like it's not just a straight overlay, but he's recoloring the B&W pixels, which I think provides a crisper result.

I'm curious what resolutions you've worked with, if you don't mind sharing. High frame rate thermal cameras (above 9hz) are ITAR restricted. Hi-res above 640x480 is not common on the consumer market and is restricted to embargoed countries. Cooled thermal cameras are restricted too IIRC.


ITAR restricts the export of arms technology from the United States, but the big names in traditional digital cameras are all Japanese. Perhaps naively, I would have thought they'd have the expertise needed to build a high-quality thermal camera without US involvement. I suppose infrared imaging must be quite a bit different from visible light.


The device we had was 1.3MP--rough the same resolution as the composite image discussed in this article. The camera core was expensive, but the lens actually cost more than the sensor. Yes it was ITAR regulated.

As an example of what I would have expected, this product brief (not the device we were using, but similar) has a picture of the white house https://www.lockheedmartin.com/content/dam/lockheed-martin/m...


Thanks. Interesting link. I must admit I can’t help but admire Lockheed’s marketing with that whitehouse image. There’s a not-so-subtle message in there.

Sounds like super interesting work!!


That's the Capitol building though, not the White House?


probably - I have never actually been to DC


The company is Santa Barbara Focalplane.

They're right around the corner from FLIR and Seek Thermal is right down the street.

Lots of amazing tech in this little town.


I presume that was with 16 bits per pixel, which is not available in consumer cameras.


(That building is the U.S. Capitol rather than the White House.)


At my previous work I was involved with processing of data from 1024x800 cameras with 16 bit per pixel and nice optics. After proper calibration and trivial image processing they were not particularly blurry since they were able to resolve even tiny temperature differences.


By that metric, most of the NASA images you see are "drawings" as well. I worked on imaging systems for various NASA spacecraft, and most of the published images are false-color, albedo-adjusted, averaged mosaics of many imaging sources from different orbital passes over months or years. Images of people and everyday things are heavily-processed as well, using techniques like HDR and color correction. Photography has always involved this sort of technical manipulation.


> By that metric, most of the NASA images you see are "drawings" as well.

Yes, there is no debate about that.


Even biological vision involves some processing from raw light → mental perception.


OP here -- I added a top-level comment with some more links. In particular you can find the separate thermal and visible images on the repo: https://github.com/dheera/iceland-thermal


Thanks for the clarification! This makes sense, the reason your images look 'off' is that the normal thermal pallets (such as the one in your colorbar) are typically a roughly constant (or at least smoothly varying) luminance with the thermal information stored in the chroma. Your images have that multiplied by the visible light image intensity which gives a very surreal effect. Fun stuff


Don't commercial thermal imaging cameras also do a thermal/visible composite? Would those also not count as thermal images?


Why are Thermal cameras lagging in resolution? Visible wavelength cameras can be had upwards of 50MP for $5k.


Visible wavelength cameras rely on relatively cheap CMOS sensors. CMOS processes have been squeezed for efficiency for decades to make, for example cheap DRAM.

The technologies behind thermal cameras are very different because they need to be sensitive to wavelengths 10x longer than visible light (the silicon in CMOS is basically completely insensitive to light above 2 um, whereas thermal cameras need to image at ~10 um). The pixels are effectively microbolometers and rely on more exotic materials like Germanium.


Funny how once upon a time it was Silicon that was considered to be the rare material for semiconductor manufacture. Then it took over and Germanium was pretty much lost, though at 0.2V forward bias it has a lot of unique applications in sensing gear. It is also why you really want to use a Germanium diode for your crystal radio set!


Thermal cameras measure much larger waves (7-14 microns as described in the article), and their sensitivity to amplitude is high.

A major limiting factor in camera sensor resolution is signal to noise ratio. It is well known that if you bump up the "ISO" (light sensitivity) to make photographs at night, you see more noise. The higher the sensor resolution, the smaller each photosite and therefore the less signal (fewer photons) captured in the short time of one exposure.

Larger sensors can solve this problem but these are CMOS devices whose cost increases superlinearly with area (like microprocessors, see die yield). Most customers won't accept dead pixels.

Economics is also a major enabler of kilobuck 50 MP visible light cameras, miracles though they are. Industrial users aren't going to re-buy all their cameras every few years like DSLR users do.


I wonder if lower-iso long-exposure thermals would be useful, or if it would get too "motion blurred"


Sure but can you sell a thermal camera that takes e.g. one second per exposure? And costs even more than current models? I'm not sure there's much of a market there.


I think it's a combination of physics (wavelength is 20x larger than visual light, making optics and sensors trickier) and export control, as you could (literally) weaponize a high-res thermal camera to make an autonomous drone/missile targetting hot exhaust etc.


Because companys pay so much, so why make an cheaper product.

Mike did a real teardown on a Flir E4 many years ago, and saw (with an oscilloscope), even on the cheap camera with the low resolution, the chip has the full resolution. I think one week later the camera was hacked.

https://www.youtube.com/watch?v=NtqUE67BUDI


As I recall, IR sensors are also much more expensive to produce, as in, 1Kx1K is already a very large array and costs thousands of $. So any ordinary consumer is not likely to be using something with very high resolution.


I agree. The median income in the US is $35k, and many of these things are 5-10% of that.

I'd really like a monocular, but anything decent is very expensive.


Microbolometer pixels are big and expensive compared to CMOS pixels.


This is just slighly informed, please prove me wrong.

The major market so far seems to have been tactical/military, so they have focused on framerates rather than resolution. And then the US blocked high framerates for civilian use...

(edit: the US blocked civilian exports.)


I bought a HIK Vision Vulkan 35mm Pro [0] in the UK recently which refreshes at 50Hz, and I've seen others that can do 60Hz. One nice thing about the HIK Vision is that it comes with a documented API for controlling it via a wifi hotspot.

[0] https://www.hikvision.com/en/products/Thermal-Products/Comme...


My thermal camera[0] is 30Hz, which I assume is 30 FPS, and I'm a civilian.

[0]: https://www.bhphotovideo.com/c/product/1194041-REG/flir_431_...


> Export of this device described herein is strictly prohibited without a valid export license issued by the U.S. Department of State Directorate of Defense Trade Controls, as prescribed in the International Traffic in Arms Regulations (ITAR) ...

So I guess they only blocked civilian exports from the US.



Glorious. I'm jealous :).


The sensor needs a lot more energy. This produces two types of IR cams: actively and passively cooled. The passive cooling is easy to take around with you but only possible with a smaller sensor.

As for the price per pixel: I don't know from memory.


Nice results, thermal cameras aren't easy to work with, especially in terms of artistic result. I've seen problems with stitching resulting with artifacts in panoramas. As I understood (I'm not thermal cameras expert) it's due too the fact that sensor warms up -> thermal camera calibration doesn't correct the result enough -> stitching needs corrections.

To put it one dimension higher, in Pix4D we work on 3d models based on the thermal images: https://www.youtube.com/watch?v=x-jp9a1bpVU Example: https://sketchfab.com/3d-models/thermal-model-of-pix4d-headq...


These are beautiful.

Any thoughts on why the photographer reversed the spectrum (red cold, blue hot) for the fumarole? https://petapixel.com/assets/uploads/2019/07/image10-1.jpg (just looked cooler?)

Or why the distant horses have breath showing, but the closer ones do not? https://petapixel.com/assets/uploads/2019/07/image1.jpg (maybe a resolution/compositing issue?)


Photographer here ;)

I took some artistic liberty in choosing color scales for each photo (a) to make them look unique (b) to make people actually look at the color scale (c) because as someone with a physics background it bothers me deeply that people associate hot with red and cold with blue and even go so far as to call 6000K "cool white" and 3000K "warm white" when those are literally blackbody temperature equivalents. So I flipped it around on one shot :)


Also physics background, and I realized it looked right to me, and had to go back and look at the others...



If only the cost of thermal imagers could come down and be more affordable, more people might be interested!


That and the associated optics.

There’s some reason for encouragement however. I bought a Seek Pro, with 320x240 resolution, on sale for $350 earlier this year. The app is pretty terrible unfortunately and the camera physical construction is flimsy, but the imager seems to be of reasonable quality for the price point.

Couple of examples: https://imgur.com/a/ZGsOwUm


Thermal imaging resolution is restricted by arm control laws unfortunately. There are forces at work beyond market forces.


ITAR compliance assurance isn't free and certainly impacts operational costs, but the sensor I have isn't export controlled and still normally costs >$1K when you buy it from FLIR.


That's only for the US though, and I'd expect such a product to come from Asia.


I once rented an IR camera for roughly 100€. This is probably the best option for people who only need IR cameras on very few occasions.

Will we ever see chap(ish) IR cameras? Is there an alternative to germanium lenses on the horizon?


Zinc selenide lenses for laser cutters work very well in the spectral range covered by consumer IR cameras. They are dirt cheap on eBay, Ali Express, and other places.


I believe there are some metamaterial-based approaches for the actual microbolometers that should help drive costs down. Ostensibly metamaterials could also be used for lenses but I'm not aware of anything there, we're probably stuck with germanium for the time being.


Seek cameras are rather noisy though. More expensive, but much less noisy and still reasonable to buy as a expensive toy would be TE-Q1 from i3system: http://nicesuper3.cafe24.com/down/TE-Q1-compressed.pdf


Looks good! Any idea why they make it so difficult to buy their product?


i3system is based in South Korea, and I guess shipping thermal imaging technology is PITA due to various restrictions, not profitable enough to sell to individual customers. There seems to be EU reseller (thermalexpert.eu) but they sell only to companies (likely because VAT accounting when selling exclusively to companies is much easier).


That carboy is pretty hot.


It was a pretty hot yeast: https://i.imgur.com/YGhx1kr.jpg


Appears to be a still? Never done that. I do mead and wine.


Literally just making sure we could get sanitizer if we needed it. Maybe will get into the craft at some point.


I bought one on eBay a few years ago for about £300, but it sucked so I got rid of it. It used a traditional camera to draw most of the image (itself already quite low-res), and just highlighted even-more-low-res temperature data in blue and red.


I was just considering buying one. Would you share your thoughts?


The resolution was too low.

I didn't really have a use in mind for it, I just thought it was interesting and thought I'd find uses for it after I owned it, but the resolution was too low to be able to do anything useful.

Here's the only picture I have of it working: https://img.jes.xxx/1047

As you can see, you can't see anything.


> As you can see, you can't see anything.

Thank you.

I wanted to get one to (a) find creatures outside at night while camping, and (b) fix my house energy leaks. But that rez seems awfully low. So amazed at the OP for doing such an amazing job stitching images!


I've had decent luck finding and seeing creatures with the Seek RevealPro I used, although there may be better ones out by now.

I even used it to try to find a friend's lost cat once. I didn't succeed in finding the cat with it, but I did end up finding a few other stray cats and racoons. (For anyone wondering, the pet cat was found later and was taken to a shelter by some good samaritan.)


For (b) even a quite low res thermal camera can be useful. Compared to a laser thermometer, you have a much wider fov, and you can always get closer.



Yes I do covet them. Getting cheaper but still fairly low resolution. Keep hoping for a tipping point like digital cameras where cheap decent sensors will suddenly be everywhere. Not yet though.


I'd imagine demand for thermal imaging has skyrocketed due to covid, so its possible the tech will get better and cheaper.


I don't think cheap cameras are sensitive enough and you don't need an entire image that is better for seeing relative heat, you just need a single accurate reading. Those thermometers are only $20 and are being cranked out in bulk.


That's sick! Brilliant and creative idea. Those shots look beautiful.


I wonder why the church's second floor window is black (fifth photo from the end). Do they insolate floors too? But then why do the walls of the second floor and the roof leak?


> (Note: Glass windows are opaque to thermal cameras; the temperature measured here is the surface temperature of the window.)

This is not completely correct. Glass tends to reflect infrared radiation. For example if you view a car windscreen from the air with a thermal camera, you'll see a reflection of the sky.

It's more complicated than that, because you'll also detect the emitted blackbody radiation from the glass (for whatever temperature).


I think that yes they isolate floors too and it looks like the front of the house leaks more heat than then roof.


Fantastic images, never before did I see a more clear image of what 'continental drift' really looks like.


Love the beautifully haunting perspective.


Awesome stuff! Thanks for sharing it!

I'm used to thermal cameras being so lo-res you can barely tell what they are.


From what I know, it's hard not do do awesome pictures in Iceland. What a marvelous place...


Really amazing photos.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: