
DensePose – Dense Human Pose Estimation in the Wild - calahad
https://github.com/facebookresearch/DensePose
======
bsenftner
I find this highly interesting from a personal perspective, as I am an author
of a Personalized Advertising patent[1] from the early 2000's. I was unable to
capitalize on my work, and ultimately sold the patent. Interesting to note,
the month the patents' expired (due to aggressive early filing dates of my
original patents) I noticed Facebook's AR Kit was released, which touched
multiple aspects of what is protected by the global patents.

This tech is key for Personalized Advertising where consumers are inserted
into still and video adverting in place of current spokespersons and side-by-
side with celebrities. Advertising is about to get surreal and the fake news
consumers are about to get exploited something unbelievable. "Deep Fakes" for
porn is kid stuff compared to what this tech opens: Pandora's Box if you ask
me.

[1] [https://patents.justia.com/inventor/blake-
senftner](https://patents.justia.com/inventor/blake-senftner)

~~~
searine
>This tech is key for Personalized Advertising where consumers are inserted
into still and video adverting in place of current spokespersons and side-by-
side with celebrities.

What a boring dystopia.

Seriously though, if a company ever did without my permission I would sue the
pants off of them.

~~~
pavel_lishin
Even assuming that there isn't already some precedent that a checkbox allows
them to use your uploaded likeness in whatever they want, is your premise here
that you have more time and money to spend in a legal battle with Facebook or
Google?

~~~
bigiain
And there are _still_ people complaining that the GDPR is not a good idea...

If this becomes "a thing", I fully intend to use my UK citizenship and send
GDPR boilerplate deletion requests to all the data brokers, social networks,
and digital advertising services I can find.

~~~
taejo
> use my UK citizenship

Better hope it becomes a thing before March.

------
unit91
A few years ago, I would have thought this technical feat was amazing, and
stopped there. Now I think it's creepy. Ah how times have changed...

~~~
rojobuffalo
Is there any other upside to this besides novelty and amusement? It's easy to
imagine a dozen applications where this would threaten human privacy and
safety.

~~~
pavel_lishin
> _Is there any other upside to this besides novelty and amusement?_

* Automated injury detection. You've got a warehouse, you've got cameras, now you've got an instant alert when one of your workers appears to be injured but out of sight of other workers. You've got street cameras, now you've got automatic detection of someone having a heart attack and laying down on a sidewalk. (Dystopian application: "homeless person detected, deploying zap-drones") Hospitals and old folks' homes could use this, too.

* Lifeguard Assist programs - automatic detection of drowning-like behaviors. (Of course, over-reliance on this would be bad...)

* Children separated from parents might be easier to detect in places like malls, etc. (I'm going to stop listing obvious parenthetical dystopian applications)

~~~
applecrazy
I see a huge possibility of replacing expensive mocap hardware and software
using this tech, allowing for video games and VFX to become more accessible
and use commodity hardware.

I’ve been eyeing things like the Kinect and iPhone X face tracking for this
kind of task (for a fun side project I’m working on), but it would be great if
I could track at least position and pose of multiple actors in a scene using
just a standard webcam or camcorder.

------
jaxbot
CC-Non commercial is a disappointing license to find. The project is very cool
otherwise.

~~~
black_puppydog
I wonder how they arrived at Creative Commons' CC-BY-NC as the license. These
licenses are not meant for code but for artwork, Creative Commons actually
discourage the use of their licenses for code [1]. I recently noticed the same
with the FastPhotoStyle code [2] by nvidia, so I'm wondering if there is
something that draws their legal departments to this license?

[1]: [https://creativecommons.org/faq/#can-i-apply-a-creative-
comm...](https://creativecommons.org/faq/#can-i-apply-a-creative-commons-
license-to-software)

[2]:
[https://github.com/NVIDIA/FastPhotoStyle](https://github.com/NVIDIA/FastPhotoStyle)

~~~
levesque
If the dataset it was trained on is CC-BY-NC, I'm pretty sure the model also
has to be CC-BY-NC. However I think this is not respected, or even considered
by most people.

I'd go with limiting how competitors can use it as the main deciding factor.

~~~
bigiain
<cynical thought>Note that two of the three team members are from Facebook...

------
ricardobeat
Would it be possible to map the first detected pose to a 3D model, scale and
deform it to match the pose, and then use each next pose to manipulate the 3D
model (vs generating all the vertices again)? This should result in smooth
animation, without artifacts, and joint limits might even help with position
estimation.

------
natch
I'm too much of a newbie to figure this out but maybe someone here can tell
me: Do they provide the final trained model? Or just a precursor model and
code that trains a final model given bring-your-own data?

~~~
icebraining
The site says "The dataset will soon be available on this website!", but
apparently it's been saying that for at least four months.

~~~
natch
Yes thanks. I was asking about the trained model though. I realize you can get
the model by training with the data, but I don't believe for a minute that any
data they release will be anything but a small fraction of what they trained
their real model with.

------
acd
It can probably be used to identify potential criminals on the way the pose
walk threatening pose. Police can then screen them. Like the movie Minority
report.

~~~
milesokeefe
Do you mean using gait analysis to identify humans and match them to a known
criminal database or do you mean finding suspect criminals based on some
“criminal” way they walk? If the latter, I don’t think that’s really based on
anything more than current cultural profiling.

~~~
pavel_lishin
"Take that pep out of your step, citizen!"

------
bsenftner
Makes me wonder how much better they have in closed source to release this
openly...

~~~
cdibona
It is closed, the non-commercial license means almost no one can use it.

------
icebraining
Previous thread (I hadn't seen it either):
[https://news.ycombinator.com/item?id=16289057](https://news.ycombinator.com/item?id=16289057)

------
bigiain
I _assume_ this would make it fairly easy to do bone-length estimation and
comparison, leading to a way to uniquely identify someone from a video feed of
them walking - even without any facial features...

Now I wonder how much of that tech is already deployed...

~~~
infectoid
Gait analysis has been around for awhile. I guess this technique just makes it
even easier.

What's crazy to think about is Gait Analysis from orbit
[https://www.schneier.com/blog/archives/2008/09/gait_analysis...](https://www.schneier.com/blog/archives/2008/09/gait_analysis_f.html)

------
lunch
I can imagine this tech being using in some pretty interesting/scary ways:

* Generating avatars in Facebook's VR land from photos you're tagged in

* Recognizing a person IRL from photos they're tagged in

------
simonvc
No-one's said it yet so i will (but not my idea..) This is going to be super
useful for a future Oculus AR headset.

Basically, imagine the current oculus go headset, but with cameras on the
front, and instead of showing you the actual world, it shows you a game, based
on the existing current world, but morphed to look like Starship Troopers or
something.

~~~
simonvc
The Diamond Age - Neil Stephenson, touched on this.

~~~
neohaven
I think you mean Syndicate. ;)

------
davidw
There are a variety of technical things where I think "wow, that's pretty
clever and/or innovative - hats off!", and then there are things like this
where I'm like "OMG wizardry!". But that's one of the things I love about the
field of "computer stuff": there's so much interesting stuff I don't know.

------
ambicapter
Is there any reason this would be restricted to human pose estimation, as
opposed to, say, rangefinding of moving objects of a specific roughly-known
size?

------
lwansbrough
I can't tell - is this able to extrapolate pose information into three
dimensions, or can it only project onto two dimensional scenes?

~~~
gugagore
It's something in between the two possibilities you describe.

Each human pixel in the image is labeled with an index and two coordinates: x,
y (u and v are the traditional names, but think of 2D x, y coordinates)

The index specifies which patch that pixel is on, and the x, y coordinates
specify where in the patch the pixel is on. This is for a pre-specified set of
patches that cover a human mesh. See
[https://github.com/facebookresearch/DensePose/blob/master/no...](https://github.com/facebookresearch/DensePose/blob/master/notebooks/DensePose-
COCO-Visualize.ipynb) for more detail.

So, no, it does not extrapolate the full mesh, but also for all human pixels,
you are getting 3D information.

------
tastythrowaway
So is this basically 3-D rotoscoping?

------
calahad
This just popped up in my github feed, looks interesting.

------
tempodox
Leave it to FB to come up with the most creepy and invasive tracking and
surveillance.

