
Webcam Pulse Detector - theschwa
https://github.com/thearn/webcam-pulse-detector
======
theschwa
The paper that this is based on is also pretty fascinating: Eulerian Video
Magnification for Revealing Subtle Changes in the World
<http://people.csail.mit.edu/mrub/vidmag/>

~~~
fulafel
Patented :'(

~~~
zapdrive
Well, in my opinion, a patent like this is well deserved. I could be wrong,
but the researchers have spent significant time on it and have
discovered/invented something really unique.

~~~
fulafel
that's the tragedy of it. something cool is invented and now it'll be fenced
off and buried for 25 years in all likelihood, and/or becomes part of a big
patent portfolio.

~~~
riquito
doesn't a patent imply that you must let other people use your idea in
exchange of a fair price?

~~~
shabble
ICBW, but no, I don't think it does, except it perhaps a narrow set of
circumstances.

Compulsory Licensing[1] is maybe what you're thinking of, but from what I can
tell seems to apply mostly to drug patents, or things {the,a} government wants
from you. It has a wider applicability in copyrighted forms of IP, I believe.

Patents that are part of some organised standard are often required to be
placed under 'FRAND[2] (Fair, reasonable, and non-discriminatory)' compulsory
licences to allow for interoperability whilst still allowing the patent holder
to receive (reasonable) royalties if they wish.

[1] <https://en.wikipedia.org/wiki/Compulsory_licensing>

[2] [https://en.wikipedia.org/wiki/Reasonable_and_non-
discriminat...](https://en.wikipedia.org/wiki/Reasonable_and_non-
discriminatory_licensing)

~~~
DannyBee
It would be more correct to say that "patents that their owners contribute to
some organised standard ..."

If I add a patented process to an important standard, but I don't own the
patents, you still don't get those patents just because they are part of some
important standard.

------
Osiris
There is already an Android app[1] that does something similar. You put your
finger up to the rear-camera and it detects your heart rate.

[1]
[https://play.google.com/store/apps/details?id=si.modula.andr...](https://play.google.com/store/apps/details?id=si.modula.android.instantheartrate&hl=en)

~~~
chewxy
It works differently. This one works by magnification of tiny changes (colour,
movement, etc)

See also: <http://people.csail.mit.edu/mrub/vidmag/>

------
dlgeek
Has anyone been able to get it to work? Whenever I try to open the stats
display, I get:

OpenCV Error: Assertion failed (p.checkVector(2, CV_32S) >= 0) in polylines,
file /tmp/buildd/opencv-2.3.1/modules/core/src/drawing.cpp, line 2064

~~~
Inufu
I have the same problem and have opened an issue:
<https://github.com/thearn/webcam-pulse-detector/issues/1>

Maybe somebody that knows more about OpenCV than I do can figure out what's
wrong.

~~~
dlgeek
I did a git pull this morning, and it's working now.

------
runeb
Give it a video feed from a political debate and live-tweet the politicians
pulse on different topics. I foresee a future of politicians sporting bangs
and bob cuts.

~~~
regularfry
They're already so heavily made-up for the cameras I would be surprised if
this worked.

~~~
evan_
Necks might not be, or hands.

This would probably be a bad metric though, you might accidentally be
selecting for sociopathic traits or something which might be undesirable in a
politician.

------
Swizec
Cool fact, a Slovenian company that is now SF based has made this tech into an
iPhone app a few months ago. -> <http://www.azumio.com/apps/cardio-buddy-2/>

I really like their apps and I'm quite proud that they originate from my neck
of the woods.

~~~
huhtenberg
It's a nicely designed app, but it doesn't work that well. Told me I had 60,
62, 61, 59... BPM when in fact it was over 75.

------
pstuart
Add some software to do expression analysis a la Paul Ekman [1] and one has a
remote non-invasive "lie detector".

[1] <https://www.paulekman.com/>

~~~
sagarun
Just like lie to me

------
pathdependent
I haven't downloaded this yet, but I certainly will. If it is accurate and
reasonably precise (yea, what does that mean?), I'd like to have it running in
the background, gathering data as I work, while also gathering statistics
about my work. I, like most of the people here, am a productivity junky. I
wonder if a heightened pulse could indicate mounting stress, which
necessitates a break. Sure, I have other machinery that could indicate
mounting stress -- like my mind -- but the quantitative, off-loaded metric
shifts responsibility in a way that could be helpful. "I'm not really
stressed," says me to my brain. "No, your heart rate is quite high," says my
computer to me.

~~~
evan_
You could get better results with a Bluetooth pulse monitor.

~~~
mkopinsky
That probably requires wearing something, potentially on your finger. Webcam
is much less intrusive.

~~~
shabble
Turns out that the fingertip pulse oximeter[1] I have works surprisingly well
on a toe.

Probably only applicable if you work from home though.

[1] <http://www.nonin.com/OEMSolutions/WristOx2-Model-3150-OEM>

------
dr_
Interesting. There was a Rock Health company that created an app using this
technology. The app is called Cardiio. I know the founders were from
MIT/Harvard but can't recall if they were the same people who conducted this
research.

------
hac
I've gotten this far trying to install this on Lion:

brew install gfortran

sudo easy_install scipy

sudo port install opencv +python27

export
PYTHONPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/:$PYTHONPATH

wget <http://openmdao.org/releases/0.5.0/go-openmdao.py>

sudo python ./go-openmdao.py

cd ~/Downloads/webcam-pulse-detector-master/openmdao-0.5.0

. bin/activate

cd ~/Downloads/webcam-pulse-detector-master && python get_pulse.py

Hope it's helpful to anyone else doing the same. OpenCV is segfaulting now,
but I may update this later if I get it.

~~~
hokkos
I got it working in ML with :

brew upgrade

brew tap homebrew/science

brew install opencv

brew install gfortran

sudo easy_install scipy

mkdir OpenMDAO

cd OpenMDAO/

brew install wget

wget <http://openmdao.org/releases/0.5.0/go-openmdao.py>

export PYTHONPATH=/usr/local/Cellar/opencv/2.4.4a/lib/python2.7/site-
packages/:/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/:/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/:$PYTHONPATH

sudo python ./go-openmdao.py

cd openmdao-0.5.0/

. bin/activate

#get the project in webcam-pulse-detector I used the github app

cd ../../webcam-pulse-detector/

python get_pulse.py

------
tocomment
This would be an awesome application for google glass! Does anyone know if
it's possible?

Imagine being able to look at someone and know their pulse. You'd have a
wearable lie detector.

------
tamersalama
I haven't been amazed for a while. Thank you.

------
leed25d
Great App for poker players

~~~
ndr
Imagine it integrated in Google Glass...

------
dyno12345
I want to point it at crowds and get an aggregate crowd-pulse rate! Measure
the excitement!

~~~
toomuchtodo
I expect to see this used by DJs, where the momentum of the music is
controlled by the crowd.

~~~
acgourley
Need way more light than that for a standard camera... now, maybe with an IR
camera...

~~~
toomuchtodo
Perhaps stage lighting could be used to illuminate the crowd...green lighting
perhaps? :)

------
jedanbik
Cool!

Reminds me of this question on Stack Overflow:

[http://stackoverflow.com/questions/12416772/is-there-an-
equi...](http://stackoverflow.com/questions/12416772/is-there-an-equivalent-
of-the-matlab-idealfilter-for-python-in-scipy-or-other)

------
lucaspiller
[Not a Python user] What is OpenMDAO and how do I install it on OS X?

~~~
AUmrysh
This might work: [http://openmdao.org/forum/questions/548/how-do-i-install-
ope...](http://openmdao.org/forum/questions/548/how-do-i-install-openmdao-on-
os-x-to-an-existing-virtual-env)

------
timothybone
This looks great for a music performance stunt -- the audience doesn't know
but the DJ rigged up a webcam as a trigger!

------
LAMike
what if someone created passwords based off of your brainwaves (assuming every
brainwave is unique as a fingerprint)

~~~
MasterScrat
The concept of "passthoughts" has been discussed recently.

[http://abcnews.go.com/blogs/technology/2013/04/passthoughts-...](http://abcnews.go.com/blogs/technology/2013/04/passthoughts-
brainwave-based-passwords-a-reality/)

~~~
vacri
It'll be interesting to see how robust that is against different levels of
alertness. I clicked through but couldn't see any info on using it on tired
people - and your EEG changes significantly with alertness level. Depends on
what they're measuring though.

------
hcarvalhoalves
How precise is this, do we have data? I'm guessing it depends on the refresh
rate of your webcam.

~~~
nl
It's precise enough for the application.

Webcams typically have a 15-30 fps framerate. Heartrates are typically 30-200
bpm (~= 0.5 to 3.3 bps).

~~~
afhof
It seems that the Nyquist-Shannon sampling rate would be too low for the
results to be accurate.

~~~
Qantourisc
No ? The frequency needs to be at-least double this seems the be the case.
However webcam noise and the algorithm might increase the need.

~~~
afhof
The webcam framerate should be at least double the heart rate to get an
accurate measurement, not the other way around.

~~~
nl
And isn't this the case?

15 fps (worst case framerate) vs 3 bps (worst case heart rate)

~~~
afhof
I dont believe so. Since the heart rate could be much higher, it would alias,
and the camera wouldn't be able to tell the difference between normal heart
rate and low heart rate.

This is what I suspect could happen, with two possible heart rates matching
the same set of samples:

[http://upload.wikimedia.org/wikipedia/commons/2/28/AliasingS...](http://upload.wikimedia.org/wikipedia/commons/2/28/AliasingSines.svg)

~~~
vacri
If your heart rate is higher than 3bps, you won't be sitting patiently in
front of a webcam waiting for lock-on.

Sedentary (calm, sitting) heartrates go from ~40bpm for extremely fit people
to ~90-100 for particularly unfit people. Higher than 100 is unusual for
sedentary people, and works out to ~ 1.5bps. The waveforms in the pulse itself
(note pulse, not ECG, which has much faster elements) speed up with the pulse,
but I would estimate (could be wrong) that your highest frequency significant
elements would be around 5hz for a 1.5bps heartrate.

Your example from wikimedia has a signal that is ten times faster than the
sample rate - and the nyquist limit says this would be aliased. My back-of-
the-envelope calcs above suggest a 5Hz signal at the high end against a 15Hz
sampling at the low end, which is a ratio of 3:1, which is enough to satisfy
nyquist.

Disclaimer: I am an ex-neuro tech and ex-sleep tech. I am used to sticking
electrodes onto people and studying them. The exact form of the pulse wasn't
big in my area, so maybe a cardio tech can chime in and correct me.

------
gluxon
What in the world? Is this magic?

~~~
AdamTReineke
On each heartbeat, blood is pushed and makes the skin nearly unperceivably
redder. By watching for and tracking this subtle change, you can extract a
heartbeat.

~~~
huhtenberg
> ... _unperceivably redder_

But why is he analyzing the green channel then? :)

~~~
mkopinsky
Look at the absorption spectra at
<http://omlc.ogi.edu/spectra/hemoglobin/index.html>. Green is ~510nm, red is
~650nm, blue is ~475nm. The difference between the absorption of Hb and HbO2
in the red range is the reason why pulse oximeters use red. In this case, we
don't care about differentiating oxy- vs. deoxy-hemoglobin, we just care about
the total absorption. Since more light is absorbed at green than at red, it
will show more of an effect with pulse.

I am not confident in this explanation, and still don't know why green over
blue.

~~~
acuozzo
> I am not confident in this explanation, and still don't know why green over
> blue.

I'm not sure either, but I can tell you that if you examine digitized (RGB)
35mm film -- across all 35mm film stocks -- the blue channel is the most
noisy.

So my guess is: He might have chosen green over blue due to the blue channel
being noisier.

~~~
mkopinsky
Is that noise inherent to film? Do webcams have the same noise?

~~~
jerich
The green channel has the least noise for Bayer-sensor digital cameras (which
is almost all of them).

50% of the sensor sites are green (25% each for red and blue), so there is
more signal given the same amount of noise.

~~~
mkopinsky
Why do they make them like that? Why not make the sensors 33/33/33?

~~~
jerich
From Wikipedia on the Bayer Filter: "Bryce Bayer's patent (U.S. Patent No.
3,971,065) in 1976 called the green photosensors luminance-sensitive elements
and the red and blue ones chrominance-sensitive elements. He used twice as
many green elements as red or blue to mimic the physiology of the human eye.
The luminance perception of the human retina uses M and L cone cells combined,
during daylight vision, which are most sensitive to green light."

The Foveon sensor does sample RGB at every pixel by using 3 layers, but it is
only used in a few cameras.

------
mikaelf
Why isn there a live demo?

------
jacobjbernstein
Awesome. Could open up a whole field of applications.

~~~
john2x
Like allow robots to detect if something is alive or dead. :D

------
nu2ycombinator
Is there a demo link for this application?

~~~
mikaelf
So strange that tye dont put up a live demo!

~~~
fomojola
Its a native app: hard to have a live demo for a native app.

~~~
addandsubtract
@shit_hn_says

