
Options for Enhancing Video of a Burglary? - jes
My home was burglarized last week. My neighbor&#x27;s security camera captured two 30s video clips of the individual, one at my front door prior to forcing it, and one with him leaving with a cloth grocery bag containing property he was taking.<p>The video shows his face, but it&#x27;s not as clear as I would like. I don&#x27;t know how to quantify its quality except to say its &quot;not bad&quot; but &quot;not great either.&quot;<p>I would welcome any suggestions for extracting and improving still images from the two clips. I&#x27;d prefer to outsource this to someone that has experience in this kind of work.<p>I don&#x27;t have a lot of confidence that the local police are going to invest a lot of time and effort in this, only because they are overloaded.<p>Any recommendations for providers or other actions to take would be appreciated.
======
brk
I work in the video surveillance space. The short version is that you are
unlikely to be able to do anything worthwhile with those videos.

If you want to contact me privately I can take a look at them and maybe offer
some specific suggestions.

~~~
jes
Thank you.

I would welcome any suggestions you might have. If you'll contact me at
john.sambrook@protonmail.com I'll send you the two clips. I don't see how to
find your email and I'm reluctant to ask you to disclose it publicly.

~~~
wizzwizz4
Might want to obfuscate that address a little – perhaps replace the @foo.bar
with "on ProtonMail's dotcom domain" or something, if it's worth it to stay
off unsophisticated scraper lists.

~~~
dylan604
Is this worth the trouble? I'm seriously asking. Seems like a ripe arena for a
cat&mouse game similar to adblockers. The script kiddies writing email
scraping tools surely have regex searching for "[at|@] * dotcom" type of
concepts.

Maybe I'm too cynical, but I pretty much assume that any email address is
"compromised" (as in no longer spam free not pwnd) as soon as it is created.
It may not be that bad, but I honestly believe the first time the email
address is used as an ID then it is.

~~~
lunchables
>Maybe I'm too cynical, but I pretty much assume that any email address is
"compromised" (as in no longer spam free not pwnd) as soon as it is created.

It's not binary, as in, either "compromised" or not. There are lots of
different spammers using different methods. Posting your email address on a
public website will dramatically increase the amount of spam you get.

~~~
arcticbull
It’s often about being the second easiest option, not the hardest.

------
fortran77
Sometimes, if you have multiple shots where the subject hasn't moved much, you
can combine them to get accurate subpixel resolution (i.e., resolution higher
than the camera has). However, this works best to subjects that are "fixed"
like license plates.

Here's a "practical" way to do it:

[https://www.dpreview.com/articles/0727694641/here-s-how-
to-p...](https://www.dpreview.com/articles/0727694641/here-s-how-to-pixel-
shift-with-any-camera)

And here's an example of a research paper about it:

[https://arxiv.org/pdf/1801.04590.pdf](https://arxiv.org/pdf/1801.04590.pdf)

The local police will spend no time and effort on it. And any video you
produce/enhance will not be able to be used as evidence by them.

~~~
ttul
This is what I was thinking as well. Superresolution techniques aren't going
to surface information that was not already in the image. However, combining
multiple sequential images of the same subject can have this effect.

The problem is that I'm sure the subject moves considerably from one image to
the next. You will have to isolate his face in each frame and transform the
face grabs so that they line up perfectly before the images can be combined by
averaging.

It's worth a shot!

~~~
CoolGuySteve
Yeah, it would be a pretty good weekend project.

Detect faces in each video frame, skew/stretch so that the area of each
detection is the same, apply this super resolution technique in OpenCV.

------
CharlesW
A company called MotionDSP makes a tool called "Forensic" that costs
$250/month:
[https://www.youtube.com/watch?v=u_B5PUYaIIA](https://www.youtube.com/watch?v=u_B5PUYaIIA)

~~~
jes
Wow. That seems like a pretty compelling tool. I may license for a month and
see what it can do.

Thank you for taking the time to help me with this.

------
behringer
I was able to save a group photo with this technique once:
[https://www.gimp.org/tutorials/Smart_Sharpening/](https://www.gimp.org/tutorials/Smart_Sharpening/)

~~~
samstave
It would be interesting to save a frame to a modern iPhone, edit it in the
photos app AND then apply this technique to the output of the iPhone edited
pic. I have been able to get a lot more detail from a photo in a dark setting
on the photos editing app on iPhone.

------
sansnomme
You might want to consider reaching out to ClearView AI. Ethics aside, they
probably can help you identify the burglars. With regards to the comments
suggesting super-resolution, just remember that the Second Law of
Thermodynamics prevents recovery of information that does not exist. Neural
networks and ML can give you a good guess at what the face may look like, but
you won't necessarily get more information out of it and if done incorrectly,
might even get a false positive. What neural networks are useful for here is
to extract the feature vectors from the face and perform facial recognition on
them.

~~~
tpmx
The key breakthrough here (for the typically crappy footage) will probably
involve

a) using many frames to ID the perp

and perhaps

b) working at a native H.264 I/P frame level with the neural networks, rather
than at a decoded framebuffer level.

~~~
mgamache
Why would the encoded frame be better vs the decided? Deblocking? Other
filters? I would think temporal filtering might help? Not an expert here.

------
nlh
This is a totally random shot, but I wonder if Pixelmator Pro's new ML Super
Resolution could be of help here:

[https://www.pixelmator.com/pro/machine-
learning/](https://www.pixelmator.com/pro/machine-learning/)

You could grab individual frames of the video and run them through this and
see what the results look like?

(no connection to Pixelmator - just played with the software and it looked
pretty darn cool)

~~~
drcode
There is a reason Google's initial DNN project was called "deep dream"\- It's
because it essentially uses "dreaming" to generate new content based on a
source database of unrelated images.

It is making up new image details (highly convincing and realistic new image
details) and hence this tech is completely inappropriate for handling criminal
evidence.

~~~
LeifCarrotson
It's definitely not appropriate for suggesting that a certain person is in the
video. Instead, what it can do is help (or poison) the imagination of a
viewer, by inserting what a person matching the few details that are in the
image/video would look like with average other details added.

Maybe your person has a darker pixel on the one side of their face you see:
might be a mole, a face tattoo, or a five-o'clock shadow, or maybe they've
just got a bit of mud on their face from falling in the dark. DNN will
'helpfully' provide that large facial moles and tattoos are less common than
facial hair in their database, and most of their models wipe mud off their
face before having their picture taken, so it will give your suspect a bit of
realistic salt-and-pepper growth. Or maybe it has a bunch of interesting
pictures of a guy with some facial hair and face tattoos that it can fit to
the video. Suddenly you find you've been burglarized by Post Malone.

It's only useful if you're trying to get an idea of what an average situation
looks like because you can't imagine what you're seeing in the noise.

------
hackpert
You might want to try out some stacking based resolution improvement methods,
like the ones used in astronomy. They take data from multiple "sources" and
combine them to produce a high resolution picture.

For example, check out
[https://www.astronomie.be/registax/](https://www.astronomie.be/registax/)

------
throwaway_tech
Beyond the video...Depending on what was stolen, if it is likely to be sold,
then monitor craigslist, offer up and FB garage/yard sale groups in your area.

~~~
pintxo
Second that. Actually got 4 tires/rims back that way (found them on ebay)

------
jerome-jh
With a video player/editor, locate the I-frames (or simply the best looking
ones, playing frame by frame), extract them, then use _any_ photo editor to
sharpen, adjust contrast/luminosity, enlarge, etc, and print them. It is not
rocket science and will give you something to show to the police/whoever, in
about an hour of work.

Before investing real money on this, be aware of the legal value of these
videos: do we really see the burglar forcing the front door? Taking your
property? Is your neighbour entitled to film that area? The answer strongly
depends on your local laws.

------
piracy1
You can enhance the overall image quality using:
[https://github.com/alexjc/neural-enhance](https://github.com/alexjc/neural-
enhance) on every frame. [https://letsenhance.io/](https://letsenhance.io/)

For faces: [https://www.geoffreylitt.com/2017/06/04/enhance-upscaling-
im...](https://www.geoffreylitt.com/2017/06/04/enhance-upscaling-images-with-
generative-adversarial-neural-networks.html)

~~~
solveit
I would strongly recommend against neural enhancing for identifying faces.
There's a good chance the face will get "enhanced" into somebody in the
training set that kinda looks similar to the blurry photo.

------
wizzerking
OpenCV has a bunch of image processing algorithms that I searched for, but
don't see mentioned CLAHE Contrast Limited Histogram Equalization _ I have
used to good effect to get better contrast Neural Network DeNoising
[https://papers.nips.cc/paper/4686-image-denoising-and-
inpain...](https://papers.nips.cc/paper/4686-image-denoising-and-inpainting-
with-deep-neural-networks.pdf)

[https://arxiv.org/pdf/1807.08176.pdf](https://arxiv.org/pdf/1807.08176.pdf)

I'm sure github has some repos on this

------
m3kw9
If you have the time stamp you could also reference other areas where he could
have walked and get more footage. But the faster you go and get these cottage
the better as some are not kept after a few days

------
cj
I've always assumed those scenes in TV shows where they pull up blurry photos
/ video and "enhance" them 10-fold was just hollywood.

Can this actually be done to any degree of usefulness?

~~~
s_kilk
Correct, if the information simply isn't present no amount of trickery will
conjure it up.

~~~
SiempreViernes
Well, these days there is plenty of trickery that will conjure up plausible
details to put in place of the information that isn't there. It won't be
_true_ information though.

~~~
bordercases
Sometimes enriching possibilities is good enough to break out of an
investigative minima but other restrictions prevent you from recapitulating
that evidence in your line of thinking. That's why parallel construction is
helpful.

------
throwlaplace
Put contact info in your profile or here so I can contact you. I do research
in exactly this area (super resolution for target detection/identification).

~~~
gpm
OP replied to another comment with john.sambrook@protonmail.com

------
jes
Here are links to the two videos:

[https://www.dropbox.com/s/jkyxlkh0aa5d24m/IMG_5402.mp4?dl=0](https://www.dropbox.com/s/jkyxlkh0aa5d24m/IMG_5402.mp4?dl=0)

[https://www.dropbox.com/s/oxgwpe6bn39p0re/IMG_5404.mp4?dl=0](https://www.dropbox.com/s/oxgwpe6bn39p0re/IMG_5404.mp4?dl=0)

~~~
rs23296008n1
First video shows he touched a few things. Perhaps fingerprints off those?
Those clothes would mark that guy depending on locale. Perhaps other cameras
saw him as well?

Also, now you know your camera isn’t doing the best, add other camera(s) where
it would have helped. Eg to capture face-on angle as you approach door.
Consider upping the resolution of the cameras you add.

On a more general note, rethink about how this guy found your place as a
suitable target. Mitigate what you can. He knew he had plenty of time. And he
seemed quite comfortable being there.

------
lisk1
Topaz Labs have some amazing photography enhancing tools that can help. They
are claiming they use AI for the enhance. Topaz Studio have this feature
ClearAI for me even on standard settings is doing an amazing job to clear the
photos. For batch processing you have to use it as a Photoshop plugin , the
frames of interest from the videos have to be extracted as separate jpeg
photos.

~~~
ubercow13
Using AI to enhance CCTV footage for the purpose of identifying criminals
seems somewhat unfair to the individuals whose photos were used to train those
AI models.

------
andrewtbham
This is not a practical recommendation by any means... but fast ai has a
lecture on image restoration with deep learning.

[https://www.fast.ai/](https://www.fast.ai/)

[https://youtu.be/9spwoDYwW_I?t=2913](https://youtu.be/9spwoDYwW_I?t=2913)

~~~
mackman
Oh god this is a terrible idea when talking about identifying a suspect.
You’re basically making an educated guess to construct evidence that could
then be used to falsely accuse someone. Please don’t do this.

~~~
andrewtbham
Yeh that is a good point. It would likely be a composite of training data
rather than the suspect.

------
akeck
It may or may not work, but try applying the below project to a few frames to
see what you get. Make sure to preserve the original footage on the original
media, though.

[https://github.com/alexjc/neural-enhance](https://github.com/alexjc/neural-
enhance)

------
c3534l
You can't get information from a photograph that isn't in that photograph in
the first place. Any software technique that purports to fill in those blanks,
like a neural network, is drawing a face in based on what it knows about faces
in general. The face you get may look realistically like a face, but it won't
look like the face of the person who robbed you, except perhaps by pure
coincidence.

~~~
theincredulousk
Video frames can be composited to improve clarity the same way it is done in
Astrophotography.

And your assertion that "any software technique...won't look like the face" is
incorrect. Again using multiple frames, it's entirely possible to infer what
clearer underlying detail would result in the less-sharp pixel values in the
video, given the training data establishing the relationship. It's a specific
technique for reconstruction, not just drawing in a realistic but synthetic
replacement like Photoshop's content-aware fill.

------
katyusha
[https://www.ghacks.net/2019/10/23/videocleaner-is-an-open-
so...](https://www.ghacks.net/2019/10/23/videocleaner-is-an-open-source-video-
enhancement-tool-for-forensic-purposes/)

------
ravenstine
The tools provided by Topaz Labs are incredible for enhancement and have free
trials.

------
theincredulousk
Not going to out OP's location by linking, but the actual video was posted in
Ring (so publicly available). OP, if you want experts to help, maybe consider
just posting the link.

------
spdustin
I wonder if the “stacking” software often used in amateur astrophotography
might help.

Or possibly using the Super Scale feature in Blackmagic Design’s DaVinci
Resolve?

------
mister_hn
You can try with ffmpeg to upscale video and improve quality. On GitHub there
are some tools that do a pretty decent work upscaling up to 4K

------
rotterdamdev
A better camera next time sadly. I prefer indoor cameras, better light
conditions help make better videos.

------
theincredulousk
Wow small world! I saw you post this on Ring. Hello from the greater Seattle
area :)

------
microdrum
Even if you can see his face clearly, how will you find him? It's just a face.

~~~
giarc
If police put out the clearest image possible, you still have to rely on those
that know him to be willing to identify him. Friends of criminals aren't
usually the type to turn each other in.

------
bilater
I can try and take a shot at it using my Pic.Hance model

------
theincredulousk
wow small world - I saw you post this on Ring. Hello from the greater Seattle
area :)

------
teddyh
You’re assuming that getting a clear face view would result in:

1\. Identifying the person

2\. The authorities finding this person

3\. The arrest of this person

4\. A conviction of this person

and possibly also

5\. You getting your things back.

In my cynical experience, _every single one_ of these five assumptions is
_extremely_ unlikely to actually happen. Note that each one of them depends on
all the previous ones.

~~~
bborud
That's not what OP asked and probably not the answer he/she really wants to
read. This being hacker news I had hoped the top comment would be something
that actually might help someone who wants to create better stills from video.

I see this on a lot of forums: people not answering the question but feeling
that they just _have_ to give their piece of advice that OP is having the
wrong problem.

Stop it. It isn't helpful. Especially to others who look for answeers to the
same question (but perhaps for entirely different reasons).

~~~
stronglikedan
But this isn't just a Q&A forum. It's also a discussion forum, and OP's post
contributed to the discussion. It's good info, and can save the poster a lot
of time, headache, and expectation.

~~~
bborud
The OP didn’t ask for legal or procedural advice. The question was a rather
straight forward technical one.

However, it came with context, which was a mistake because it drew focus away
from the actual question.

------
keanzu
Where does enhancement stop and tampering with evidence (18 U.S.C. § 1519)
start? With sufficiently powerful machine learning the video can be deepfaked
to have Elvis robbing you.

[https://www.law.cornell.edu/uscode/text/18/1519](https://www.law.cornell.edu/uscode/text/18/1519)

~~~
colejohnson66
Is it still tampering if you provide the original and the “enhanced” one?

~~~
dordoka
It will probably not be considered tampering, but at the same time, it will
probably not be accepted as a proof officially.

