
Through-Wall Human Pose Estimation Using Radio Signals - signa11
http://rfpose.csail.mit.edu/
======
wcunning
I've noticed, recently, that literally every major chain store I go into
blocks my cell signal. I get full bars outside, perfect LTE, and about 5 ft
past the glass doors, I'm suddenly cut off. Specifically, Meijer grocery
stores and Home Depot are the worst offenders. Coincidentally they offer free
in store WiFi to track me with, so I turn off my wifi radio on both smartphone
and smartwatch before walking in the door... Note, this was not the case in
the exact same physical stores a couple years ago, and they weren't closed for
remodeling in the interim.

Does anyone know if this sort of Faraday cage technology for large buildings
is easily available for me to install in my own home? If so, what's it called,
what does it cost and where do I get it?

~~~
lgats
I believe you can achieve the same result in your home by adding a sheet metal
roof, metal cabinets, and refrigerators along every wall.

~~~
cabaalis
It's a rather unfortunate coincidence, isn't it? Certainly it's not their
fault that their buildings just happen to be constructed in such a way, and
their products arranged such that it blocks signal causing customers to not be
able to compare prices and have to use their approved internet connection. /s

I see this in the Targets in my town. I just expect signal to drop to zero.

~~~
davio
My Lowe's and Home Depot both have free Wifi. I use it all the time to use
their app or site to find the location of items in store (obv. can't talk to
humans).

~~~
russh
Good luck with that. I tried unsuccessfully, to buy an item at both Lowes and
Home Depot but nobody was able to determine where in the store the item was
even when I pulled the information up on my phone.

~~~
kbenson
One of the Home Depot's in my area had broken Wifi for multiple months. You
could connect, and go through the disclaimer, but nothing was reachable
through it. Particularly annoying because the other store, which is about the
same distance away, has working Wifi but horrible reception. Since I'm
occasionally in each, I had to remember to forget the Home Depot Wifi and add
it back in depending on when I wen tto different stores.

------
RyanCavanaugh
In a 5-4 decision in
[https://en.wikipedia.org/wiki/Kyllo_v._United_States](https://en.wikipedia.org/wiki/Kyllo_v._United_States)
, the SCOTUS determined that police can't use thermal imaging on a house
without a search warrant. I wonder what the vote would have been if the
justices could have seen videos like this.

~~~
JTbane
Sometimes I wonder is this could be mitigated by literally covering your walls
in aluminum foil. Maybe those "tinfoil hats" are on to something.

~~~
binarymax
Just get aluminum siding, which was very popular before people switched to
vinyl.

~~~
dsfyu404ed
Having pulled some of it off to switch to vinyl I wish it was still readily
available. It's orders of magnitude more durable than vinyl.

------
visarga
With this paper our last shreds of privacy are being taken away, but they
presented the news with nice background music, so there's that.

~~~
bufbupa
Meh, that seems a little sensationalist. Any significant metal would likely
throw this in a loop. You could even buy paint with graphite in it to block
weak RF signals outright. IIRC most military bases and data centers protect
against this sort of leakage.

~~~
Sammi
The potential for open/outdoor spaces is scary though. Radio "CCTVs" that see
everyone over a large area.

------
barbegal
What this video (or even the paper) doesn't show is the large antenna arrays
that are used to capture the heatmaps. A previous paper called "Capturing the
human figure through a wall" [1] shows the type of antenna array used and in
this paper they use two such antenna arrays to capture two planes to increase
the accuracy.

[1] [https://www.semanticscholar.org/paper/Capturing-the-human-
fi...](https://www.semanticscholar.org/paper/Capturing-the-human-figure-
through-a-wall-Adib-Hsu/2b2d03f8b96aa1e306fb941e0318d403efbde4be)

------
sudhirj
Love it. This is a game mechanic / movie superhero device that's been around a
long time, just love the fact that this does it so simply and with so much
promise. As sensitivity and the quality of the neural net increases, combined
with AR, this is going to make super soldiers out of SWAT teams.

~~~
renborg
but thermal imaging is already available without AI or WiFi..

~~~
namibj
You might want to take a look at Big Clive's video about the radar motion
detectors in Chinese E27 socket LED bulbs [0], which have become cheaper even
back in mid-2016 than good old PIR motion detectors. The latter are basically
a castrated Flir-style camera, the former closer to this.

[0]:
[https://www.youtube.com/watch?v=FgdXRLjYkc4](https://www.youtube.com/watch?v=FgdXRLjYkc4)

------
aeleos
It’s pretty amazing that they only trained it to create pose estimation from
radio signals, and that its ability to see through walls is only a byproduct
of them not blocking RF.

Given how much information is out there that we can’t sense I wonder if this
area will be a main source of AI advancements. Another example of something
like this is using videos of plants that vibrate slightly with sound waves to
recreate the original sound.

------
jamesough
Use ultra-wideband radio pulses, an array of a few thousand 3D printed UWB
antennas and a phased-array scanning beam. You'll get a machine you could fit
in the back of a truck that can scan a city block in real-time to mm
precision. Sweet dreams, citizen.

~~~
femto
Potentially not that simple, as there are fundamental limits on the
information capacity of a MIMO system, based on the surface area of the volume
occupied by the antenna array (in wavelengths squared). The capacity limits
apply whether you use the array for communications or radar, as both are
governed by Information Theory.

Not to say that it can't be done, but the maths exists to definitively prove
whether it can be, so no need for speculation and tinfoil. My gut feeling is
that you could scan a small volume from afar, but not a whole block. No, I
haven't done the maths.

~~~
jamesough
Yeah this is why you'd use UWB -- the capacity is insane. Also the raw
information theory calculation makes sense only if you have zero prior
information about the sample you're reconstructing. But 'people in buildings'
is a pretty strong prior. And it's not like you have to do any extra work,
just put a simulated raw signal through a neural net and regress to
groundtruth data.

But let's do the math, I'm curious!

------
Animats
The paper [1] doesn't show the antennas, but the text seems to indicate the
antennas are quite large, as large as the area being observed. Plus, you have
to have two perpendicular axes of antenna, so you have to be behind two walls,
or one wall and a floor or ceiling. That's a big setup.

It might be useful if you wanted to count people in a crowd going in and out
through a big opening, like a subway station or mall.

[1]
[http://openaccess.thecvf.com/content_cvpr_2018/papers/Zhao_T...](http://openaccess.thecvf.com/content_cvpr_2018/papers/Zhao_Through-
Wall_Human_Pose_CVPR_2018_paper.pdf)

------
miguelrochefort
I'll repeat it again.

\- Machines that see through walls are coming

\- Microscopic drones are coming

\- Mind-reading machines are coming

\- Private key cryptography will become obsolete

Fighting for privacy is backward. It's unsustainable. We must work toward a
post-privacy world. The sooner we do it, the smoother the transition will be.

~~~
beenBoutIT
Describing the exact same inevitable phenomena as "The Age of Transparency" is
probably going to elicit a better response from most people. "Post Privacy" is
negatively charged and makes it sound like more of a net loss as opposed to
progress.

~~~
Sammi
But they're the same thing. You can't get rid of the negative consequences
just by using a different term.

------
hammock
Wi-fi has been researched for this capability since at least 2015:
[https://www.youtube.com/watch?v=fGZzNZnYIHo](https://www.youtube.com/watch?v=fGZzNZnYIHo)

~~~
freyir
It’s a new paper from the same group.

In 2015, the tracker person was just shown as an amorphous blob on the screen.
In the recent paper, they display a stick figure modelling the movements of
the person’s limbs.

------
thedirt0115
Do the people that write these papers ever release their source code or
trained model? I'm curious about both other researchers reproducing as well as
being able to download something and play with it.

~~~
freyir
They’re using custom hardware to make this work.

This isn’t always clear from the press release. For example, they mention that
the technology works at “WiFi frequencies.” That’s true, but they’re sending a
radar signal using two huge antennas, not a Wi-Fi router.

------
hathawsh
Am I correct in assuming they are using a Walabot sensor as mentioned in
footnote 2?

[https://walabot.com/](https://walabot.com/)

EDIT: I see from section 3 that they created their own FMCW antenna array that
is somehow similar to a Walabot.

~~~
godelmachine
May I ask what's FMCW? Of course I can straightaway google but I would like to
hear an intuitive explaination.

~~~
hathawsh
I had to Google it too. :-)

[http://demonstrations.wolfram.com/FrequencyModulatedContinuo...](http://demonstrations.wolfram.com/FrequencyModulatedContinuousWaveFMCWRadar/)

------
sebringj
Through-Wall Human HeadShot Estimation Using Radio Signals...

------
concernedctzn
I wonder how similar it is to the tech used in this paper on gesture
recognition through walls:
[http://wisee.cs.washington.edu](http://wisee.cs.washington.edu) [2013]

~~~
barbegal
They are quite different. Wisee uses a single reciever and uses doppler shift
to identify the gestures whereas this "through wall" system uses an array of
transmitters and receivers that identify reflections from a position in space.

In layman's terms this system is more like a conventional camera but working
in the radio spectrum whereas Wisee is like a microphone that can pick up
changes in frequencies around it.

------
bufbupa
Finally, a legit purpose for my tinfoil hat!

~~~
amenod
I know you're joking, but that's an interesting idea... I wonder how would the
system "see" your head then? Unless it is trained on people wearing tinfoil
hats (and aluminium suits) it might produce interesting results.

~~~
barbegal
Adding a tinfoil hat would make you reflect more which the system would pick
up even easier.

~~~
amenod
Probably true - but it wasn't trained on such dataset, so I'm curious how
robust it is.

------
zlynx
Just like people running open WiFi access points getting upset about anyone
reading their packets. They don't think about what they can't see.

A building that doesn't block radio and IR (and others) is essentially
transparent. Putting an emitter like a WiFi point inside is like turning on a
light.

What we need is more attention put to building better walls.

~~~
mathnmusic
Does it matter if our WiFi points are always on?

~~~
matte_black
Presence detection can easily toggle WiFi on and off based on if people are
present.

------
rl3
The defense applications for this are tantalizing. However, given the
staggering amount of room clearing that's been taking place in US military
operations for the past 15 or so years, I'd be shocked if there wasn't already
an existing capability to detect bad guys through walls in a CQB environment.

~~~
dsfyu404ed
>I'd be shocked if there wasn't already an existing capability to detect bad
guys through walls in a CQB environment.

Existing and exist in a state you can deploy are two totally different things
that often happen many years apart. Usually something exists as an impractical
wish list item for a long time before some other development pushes it into
the realm of being practical to use.

~~~
rl3
Yeah, it's just that western special operations kick in so many doors on a
daily basis that one would think such tech would be fast-tracked.

------
packeted
Awesome. I was playing Batman Arkham City yesterday and this is "Detective
Mode" in the real life!

~~~
TremendousJudge
Yeah, I wonder what they are going to use this technology for. "Helping
earthquake victims" probably

------
thirduncle
Countermeasures?

~~~
aeleos
I bet some RF absorbing material or something that reflects RF at weird angles
would probably break it. Or an active RF transmitter.

------
enervate
I guess now we know how that beginning scene in altered carbon could work

------
gaius
There are no legitimate uses of this technique. It only applies if you lack
the consent of the target. If have that, there are many easier and better ways
to monitor them.

~~~
murdockq
A personal home security system could make better use of this instead of
motion sensors in each room.

------
bitL
We need multiple home robots to fake our presence now!

------
ttul
Amazing what a bit of military funding can achieve.

------
lolc
Wow great work! I'm stunned. Not really by the pose-recognition, that's just a
bonus.

------
gohwell
Does it only detect moving bodies?

~~~
sudhirj
Depends on the training data and the room config. The system sees a difference
in an empty vs occupied room, movement doesn’t seem to be strictly necessary.
The heat map video looks more like radio spectrum signatures, which might even
be working on still frames - which would imply that movement isn’t necessary
at all. It would help the AI, though. Easier to classify moving bodies, the
speed and gait is a signal.

------
godelmachine
I would like to get more research of this sort.

Would someone kindly point me towards it?

Thanks.

~~~
Jach
I built and extended this for a college project years ago:
[https://ocw.mit.edu/resources/res-ll-003-build-a-small-
radar...](https://ocw.mit.edu/resources/res-ll-003-build-a-small-radar-system-
capable-of-sensing-range-doppler-and-synthetic-aperture-radar-imaging-january-
iap-2011/index.htm)

It's not a bad place to start. And cheap if you want to build your own too
(some third party even sells the "cantennas" so you don't have to fiddle with
tuning that part (which is kind of infeasible without a network analyzer
anyway)).

~~~
godelmachine
Thanks Jach

------
maxander
We could be using this technology to see tumors and other pathologies through
patients’ skin. Instead we’re using it to give soldiers and spies x-ray
vision.

~~~
gus_massa
No we can't use this radio signals to see tumors, because science is not magic
and wishful thinking will not solve the technical problems. Let's try to name
a few:

The radio signals are absorbed by wet salty meatbags, so you can find them
using radio signals, but you can't see thru them. If you raise the power to
compensate the absorption, you will cook the meatbags, like a microwave oven.

If you were lucky to get a signal at the other end, the radio signal can
distinguish between meat-bone-air-water, but not between tissues like meat and
tumors that have a somewhat similar composition. A tomography use contrast to
try to distinguish them. A magnetic resonance is better, but it use a huge
magnet (and in some cases it use contrast too).

And also, the radio frequency that they are using has a wavelength of an inch
approximately. So roughly it can resolve things that are an inch long, so it
must be a big tumor to be visible. Perhaps you can try with a higher frequency
that has a smaller wavelength.

And if I understand correctly, this device needs a lot of calibration, so you
need version of the part of the body without the tumor and a version with
tumors of different sizes and positions to calibrate the device and then try
to use it in the real subject.

------
dznodes
MIT's new anti-fapping radar!

------
himom
Almost like The Eraser (1996), just add squad railgun rifles and X-ray-like
targeting.

------
con22
we have nothing to hide now!

------
ggg9990
Look at the list of names on this paper. We better fix our immigration system
for high skill individuals.

~~~
jjoonathan
Devil's advocate: "let's fund PhDs with respectable salaries instead of green
cards."

~~~
hello_1234
We are talking about MIT. Their acceptance rate is around 6%. I am sure they
get plenty of applications from the smartest American undergrads. Assuming
that they choose the best and the brightest, it makes sense that they would
have students from all over the world. Americans don't have a monopoly on
intelligence.

~~~
throwawayjava
You are both correct.

Many Americans choose not to go into academia (at every level) because the
other options are much more lucrative. It is true that many of America's best
minds do not pursue a PhD in large part because of the paltry stipend.

It is also absolutely true that the one top CS PhD program I know of does not
compromise in admissions and does not suffer from a lack of highly qualified
applicants.

 _> Americans don't have a monopoly on intelligence._

I think it's possible to agree with this point while still believing that
American CS Ph.D. programs -- especially those outside the top N -- are not as
attractive to domestic applicants as they probably should be.

