
AI Can Transform Anyone Into a Professional Dancer - tuckermi
https://news.developer.nvidia.com/ai-can-transform-anyone-into-a-professional-dancer/
======
cheschire
I could see this being used in the same way that auto-tune type effects are
added to singers voices. The editing of music videos could include a tool like
this to make sure all of the dancers behind the main one are moving _just_
right.

And just like auto-tuned voices, it will come off as janky and fake.

~~~
valtism
The autotune you refer to as "janky and fake" is when autotune is used with
intention to create that sound as an artistic syle. Autotune is used across
most pop (and other) music, but in such a way that that you never realise it
is there.

It is similar to special effects. People complain about how bad and fake they
look, but this is only for the effects that are bad enough that they are
noticible. People don't realise the sheer amount of special effects being used
in scenes they never realise they are being used for.

~~~
ssijak
There is nothing "artistic" in overusing autotune. It is just a tool to "mask"
that that person can`t really sing at all and is overused in
popular/commercial genres where there is an abundance of untalented musicians.
I can`t imagine anyone saying that obvious autotuned songs sound good.

~~~
apengwin
Kanye West used autotune to great effect in 808s and Heartbreak, one of the
seminal albums of this century

~~~
deadbeef404
Very true about using it to great effect in 808s and Heartbreak! I remember
hearing one of those tracks and finding it fairly nice to listen to. Then I
heard him live on SNL and his singing voice was close to non-existent.

------
tommoor
Oh man, the first company to put this in a mobile app is going to blow up.

(I guess it might take a few years for the performance to get there)

~~~
abledon
Isn’t there a patent / copyright or something for taking all the algorithms /
structures in the tech stack that this relies on? (GAN , densePOSE , etc)

Is any tech that is published in arxiv just free game immediately? Seems
unfair to the researchers

~~~
MasterScrat
It's their choice to publish.

------
tachyonbeam
Funnily enough, I think this technology would be better demoed by people doing
more natural motions like walking around and doing basic gestures, or dancing
in a style that is more fluid. This type of dancing is made to look
intentionally "unnatural" (ie: you rarely see people moving like this in your
daily life), which makes it a bit difficult to tell how much of the
strangeness/unanny-ness comes from the dancing style vs imprecision in the
algorithm.

~~~
phalangion
That might be intentional. We're used to seeing people walk, so it would be
easy to point out flaws in transformed video of people walking. With the
unusual movements, we attribute it to the moves because we're not used to
seeing them.

------
bsenftner
Nice save way not to say "New Deeper Fakes - now full body!". This is hardly
just about dancing. This can be used to create anyone doing anything. The
quality will improve, far beyond the quality levels of the hands now and the
other interpolated body parts far past the quality necessary for games.

~~~
learnstats2
Yep.

The real consequence of this is that video footage is no longer a reliable
source.

------
TheCoreh
Very impressive!

I'm a bit disappointed though that they didn't also include results for a
synthetic source video with "impossible" poses (e.g. joints bending backwards,
stretching, separating from the body or performing full rotations). That would
have been pretty interesting (though perhaps a bit unsettling) to see.

~~~
StephenMelon
I'm really impressed that the shadows behind in the window are reasonably
realistic too.

------
richdougherty
I loved watching the movements in the "Detected Pose" corner. I felt like I
could see the forms of the dance more clearly. I wonder if ML could learn
aesthetically pleasing dance forms, then perhaps we could get some generative
choreography!

~~~
flattone
Interesting. If we could grab audience wide fmri at the next nutcracker for
training, right ?

~~~
flattone
Edit _for labeling_

------
schaefer
The title seems poorly worded.

Using AI to transform anyone into a professional dancer might include using AI
to process live video (webcam) of someone dancing and then giving them some
feedback for improvement. In a word: coaching.

However this is using AI to produce composite videos of people dancing.

~~~
cowpewter
We actually kinda already have the first, in the form of the Dance Central
games for the Kinect. You dance, the software detects which of your limbs
aren't moving the right way, and it display visual feedback (highlighting the
limb in red, reducing your score, etc) to show you what you're doing wrong, so
you can perform the dance more correctly next time.

It's not good enough to produce professional dancers, but it has definitely
improved my dancing as someone who just dances for fun.

------
mc32
So this is a cool demo --and it has applications in cinema, MVs, etc. But,
this is being presented as something which could allow Jane Q User to portray
herself as an accomplished dancer --just transfer a style onto herself.

Maybe I'm in the minority, but I think if we take this idea and walk with it,
it has the potential to trivialize actual accomplishment. Maybe I'm
overthinking it.

~~~
djsumdog
I think that's all it's intended to be right? A cool demo. And it looks pretty
amazing for what it is really.

We're not going to see The Running Man style/quality fake videos any time
soon, and the media kinda runs with this an exaggerates; making people wonder
if camera footage may one day no longer be considered evidence.

We're far from that. At the most, the quality of the transfers here is about
the same as what you'd see with Deep Fakes (celebrities imposed on top of
pornographic models using computer vision and AI algorithms).

~~~
colordrops
Are we really that far?

~~~
bsenftner
We ARE that far. Any deep fakes you see in the wild are the hobbyists applying
the tech to the lowest common denominator. There are professional state
sponsored groups creating serious fake video, easily manipulating presence of
people at news worthy incidents.

~~~
colordrops
I think you misunderstood my comment. The comment I replied to said we are far
from realistic fakes. I was saying "are we really that far from realistic
fakes?", not "are we really that far along?"

------
Kagerjay
So things can AI do now

\- Mimic a target's body motions (this link)

\- Mimic a target's facial expressions (deepfakes)

\- Mimic a target's voice (lyrebird AI, etc)

related video, digital animation puppeteering

[https://www.youtube.com/watch?v=YiOByO8J7xg&t=2s&list=LLI462...](https://www.youtube.com/watch?v=YiOByO8J7xg&t=2s&list=LLI462thar4CFee8GIqXwEPQ&index=143)

Its not perfect by any means, but we're seeing a new age of CGI. Once
perfected, I wonder how the entertainment industry will change as a result
(Faster rendering times, less time to make scenes, puppeteering, not needing
expensive famous actors or stunt doubles, digital identity copyrights, etc)

~~~
piyh
I'm worried about the social ramifications and moving us one step closer to
"post-truth". Not much we can to do stop it at some point, but if a certain
pee tape came out in the next year, there'd always be a reasonable doubt to
its veracity.

~~~
SeanAppleby
Yeah, I don't think people are lending enough weight to how big of a deal that
is.

We're heading into a world where it would not be very hard to bombard the
public with a large number of long form videos of highly convincing videos of
anyone in the world ranting on any topic and acting out anything they want,
and we would have borderline no idea if it was legitimate.

Combining that with our media climate and already runaway problem with
monetary and political incentives for fabricated stories seems really
dangerous.

You could make a video of Neil Armstrong and Nasa execs talking about how they
faked the moonlanding, or even much more nefarious fake content confirming
conspiracy theories for political ends.

What will we use as a scalable filter to know what is actually going on, and
how will we keep that content from manipulating public discussion?

~~~
Kagerjay
We've had digital and video manipulation for years, it leaves behind pixelated
artifacts though and can be spotted (E.g. see captain disillusion on youtube).

But yes there's going to be a big market for tracking fake data sources in the
future. We're already seeing tools to track fake twitter accounts, fake
instagram followers, fake amazon purchase reviews, this is an ongoing trend.

------
ggm
Its a bungee jump into uncanny valley: exciting, but how many times would you
pay, and who has more fun? you, or your friends watching?

------
lovingdancer
As a dancer & instructor of over 18 years, I think this technology is
fascinating. I actually think it would be most effective as a teaching tool
for my students. Often times, since the kids are so focused on the physicality
of the steps, I find a disconnect between the visual and physical experience
as they train, i.e. the kids don't realize that the steps/movements they make
are in attempt to create visual shapes and lines. They run around the studio
'feeling themselves' (precious), but at the end of the year on-stage, the
choreography suffers from this visual connection.

I appreciate that the detected poses and motions create clear pictures for
what different parts of the body are doing. Particularly for ballet, if I had
access to this technology (in a way that was user friendly), I'd love to see
the difference between ballet styles (Vaganova, Cechetti, ABT, ect). I think
it would be much clearer from a students' perspective, to see the stylistic
difference in lines, shapes and movement.

This AI reminds me of Happy Feet, where they took Savion Glover's movement and
choreography and applied it to the animation penguin. It doesn't seem too far-
fetched. And lastly, for those who say this seems unnatural--dancing is
unnatural to the body, hence the training and years put into it. So having an
AI applied to it will only make it look more unnatural.

Artistically, this can be debated (as it has been), but in search for 'real
life application,' I'd love to get my hands on this as a teaching tool.

 _sorry for the long post--this is my first time on this site--my boyfriend
sent this to me & warned me that if i blabbed too long, this post would not be
successful._

------
pelario
As many other comments have said, the title is misleading; the key quotation
is:

"(...) allows anyone to portray themselves as a world-class ballerina (...)"

Moreover, after AlphaGO took away Go from us, I started to wonder "what is
left" for humans, and I believe that we are centuries away to have machines
that achieve world class dancing level. My reasoning is than in things like
Go, image or speech recognition, it is easier to "encode" the information for
the ML to actually learn. On the other hand, encoding the movements of
professional dancers is already quite difficult. Consider for example in the
video linked here, the whole human body is mapped into ~20 points. Sure, this
may be enough to portray someone as a dancer. But good luck making a dancing
robot.

So, maybe I quit my programming career to become a dancer, it is less likely
to be a job that the machines will take away ;-)

edit: grammar

~~~
Hendrikto
> I believe that we are centuries away to have machines that achieve world
> class dancing level

People said the same about Go. There are AIs that can compose enjoyable music
already.

~~~
goatlover
Robotics is a bit harder than board games.

------
tjr225
AI can make anyone appear to be a professional dancer. There is a difference
between what is real and what is fake.

~~~
flattone
Good thing we often accept versions of reality as far away as snap filters.

------
paraschopra
Here's an unpopular opinion: such applications aren't going to trivialize art.

Like competitive sports, art is all about display of human ability under
constraints. This is why even in the age of photographs, we still value hand-
painted canvases. Such techniques are simply going to make people more
discerning between real effort v/s automated means of generating the same
outcome.

Rather than thinking AI-assisted style transfers are the end of art, we should
think that these are new tools for artists to do even more interesting stuff.
See this upcoming tool for example:
[https://runwayml.com/](https://runwayml.com/)

~~~
hellofunk
I entirely agree with you. It's one thing for computers/AI to emulate the
creative work that humans have already achieved, essentially copying, or
porting, or manipulating prior art, but it's something else entirely to
genuinely create something new and fresh and connects with people emotionally,
and I have yet to see any evidence that AI is close to this.

~~~
krapp
Most modern art isn't made to great something new and fresh... it's mass
produced pastoral stuff like Thomas Kinkade or connected to a multimedia
franchise (book covers, movie posters, game art, etc.) and a lot of that is
certainly formulaic and derivative.

Maybe AI isn't able to copy human technique well enough yet but whether it
succeeds or fails will have little to do with whether or not it creates work
that resonates emotionally like classic art, because that's no longer the
purpose of the vast majority of art that people encounter.

And I would argue that human beings, for the most part, copy other human
beings anyway. Working within a "genre" and using cultural references and even
recognizable techniques are all essentially copying or at least adapting what
came before.

~~~
hellofunk
I guess it depends on how broad a definition of "art" we are using.

------
MPSimmons
I'm sure that this will, at no point, be used for evil.

~~~
trukterious
Indeed. It's still in the uncanny valley, but then evil is uncanny valley too.
Goodness has always been linked in our minds with beauty and grace, including
grace of movement.

------
dangerface
Need to send this to theresa may

~~~
robaato
Exactly my thought!

[https://www.theguardian.com/politics/2018/aug/30/theresa-
may...](https://www.theguardian.com/politics/2018/aug/30/theresa-may-busts-
out-dance-moves-africa-trip-guides-scouts-nairobi)

------
DrNuke
We should think of something like a blockchain to mark all this sh*t as fake
though, because in five years time there will be no way to distinguish reality
from invention and we will all be under constant blackmail from malicious
agents and rogue governments showing up at our door with whatever made-up
accusation they want.

~~~
eivarv
How would blockchain solve the issue?

~~~
DrNuke
Link a blockchain app made with IOTA (for example, but this is the best and
most manageable protocol I can see out there just now) to your unique ID in
the form of something like your heartbeat plus your physical location plus
your actual body shape, then perform a mini-transaction every 30 seconds or
every minute or the granularity you need, finally store the transactions
linked to that data as an alibi against any accusation that you were somewhere
else doing something fake an AI created to blackmail you.

~~~
eivarv
Or you could just be critical, and be aware of the limitations of media as it
pertains to representation of truth and reality.

Besides, techniques for identifying fakes is likely to lag closely behind the
techniques for producing them.

The first time we talked about "DeepFakes", someone I know pointed out that
there is nothing inherently new in this issue. Media has been faked to
manipulate the truth as long as there has been media.

Wether you trust the medium, a person, or a blockchain, trust is only as good
as the information you base it upon – and there's always ways to circumvent
it, or otherwise deceive you.

Also: There are some pretty big privacy issues (from what I understand) with
what you describe.

~~~
DrNuke
Unbiased courts or at the very least your legacy as an innocent person if you
get however smothered need a bullet proof you did not do what deepfakes
claimed you did. As for privacy, again, blockchain makes your data fully
available to you only and you only need to disclose the relevant bits against
the time-space accusation you face. Trust cannot be delegated but reliable
technical means would help a lot.

~~~
eivarv
Why not just put less trust in video?

The courts should know that videos can be doctored, just as images can be
photoshopped.

As the burden of proof lies with the accuser, it is they that would need a
bullet proof argument that you did in face do what the video claims you did –
and video is not alone in and of itself to establish truth.

As for the data gathering: the existence of the proposed data presents a
threat for the subjects (and society) in and of itself.

------
lainga
Oh boy, those hands at 0:14 are a no for me.

~~~
taneq
Technically impressive, this is. Canny, it is not.

~~~
bryanrasmussen
Is it uncanny then?

~~~
taneq
That... would be the implication, yes.

~~~
bryanrasmussen
Ok you meant the movements of the 'dancers' were uncanny, as opposed to the
technical prowess demonstrated? I thought that the not canny comment referred
to the tech, not the result, and it didn't seem sensible given other things to
describe the tech as uncanny.

------
alexcnwy
This is incredible!

I wonder if seeing yourself dance like this might speed up learning to
actually dance like this...

~~~
skookumchuck
I doubt it. Almost nobody learns to dance without a coach. It's because what
you think you see is not what the movement actually is. Much of dance is
playing with your perception of the movement (the moonwalk is the most obvious
example, and very, very few moonwalkers can pull off the illusion).

------
nikkwong
I'm itching to make this. Would this be a good intro ML project for a solid
software engineer (with a decently strong math background) or would it likely
be far over my head? Seems like reverse engineering it from the paper would be
tough, but maybe doable :p

~~~
currymj
it would be expensive in terms of hardware, you’d need to shell out for quite
a few GPUs or else budget a lot for expensive cloud instances.

plus this is just a very complicated thing, in that it’s gluing together
multiple new techniques to do various things.

some of the pieces that went into this work (like GANs) have lots of tutorials
online and might be a more manageable, and budget-friendly, place to start.
you could do something interesting on Google CoLab with free GPU time.

------
mirimir
No, not "transform into". It's "make look like, in a video".

I mean, who cares what you look like in some video? When you actually meet
people, they'll know that it's bullshit.

Now, if you could manage it in meatspace, _that_ would be cool!

~~~
onion2k
_who cares what you look like in some video?_

Everyone who watches television, movies, YouTube, etc. I know that's only _a
few_ people, but hey, it's a start.

~~~
mirimir
I suppose. But then, what would be the hiring criteria? Lowest bid? Nice ass?

And the focus here is "anyone", not professionals.

~~~
Jaruzel
The biggest worry here, is where instead of hiring 20 dancers for a music
video, they only hire one, and use AI to AutoDance[1] a bunch of low-cost
actors instead. This could destroy the jobbing dancer-for-hire market.

\--

[1] AutoDance™ - Like AutoTune. I hereby claim it as a term. ;)

~~~
mirimir
OK, I get that. But then, why bother hiring meat at all? Just CGI it entirely.

~~~
usrusr
CGI could do it for quite a while now, just not at a price competitive with
actual dancers. I'd think of this more as CGI automation than dancer
automation. For dancers, only the video production part of their market is
threatened, the stage part remains unaffected.

------
yedawg
Anyone want to get rich developing this for android /w me? haha

~~~
abledon
Can’t nvidia and the papers authors sue you or claim a takedown / charge
licensing or something ?!? How is it legal for people to take cutting edge
tech from universities for free and use it for their own software/profit?

~~~
currymj
same as it’s legal to use open source licensed software to profit. a lot of
academics don’t care, and their work might be built very strongly on other
work. there might or might not be a patent on this research, but I would guess
probably not?

~~~
abledon
I thought github projects were only ok to use if the author provided a LICENSE
file e.g. MIT or APACHE 2.0 , do these papers include a license clause at the
bottom? Usually no right , so it seems to be a ‘grey’ area

~~~
currymj
well, is it legal to do a "clean-room" implementation from the descriptions in
the papers themselves, without looking at any provided code?

(this should almost always be feasible, and is commonly done for non-IP-
related reasons e.g. someone might make a PyTorch version of something when
the original version was done in Tensorflow.)

i'm not a lawyer but I would have assumed "probably", unless there's a patent.
i mean, if it's not this would suggest it's illegal to attempt to replicate
scientific experiments.

also, in many cases even for the code provided by the researchers there is an
actual LICENSE file included, and it's often BSD or MIT. (Which sort of makes
sense -- these two permissive free software licenses are named after the
universities they came from. they reflect the academic CS culture around stuff
like this.)

------
browsercoin
Does this mean that we can manipulate videos programmatically in the future? I
don't see why not. Maybe we'll see games that are literally undistinguishable
from reality.

------
yummybear
My kids would pay any amount of my money to play this as a game.

~~~
bryanrasmussen
So would my kid, but that's because she does not seem to understand money.

------
amelius
Some time ago there was a submission saying that the adult industry uses
similar algorithms to put arbitrary faces on videos of actors.

------
thefounder
The title is clickbait...It doesn't turn anyone into a better dancer. It's
just about CGI stuff. A kind of future of emoji.

~~~
goatlover
Or the future of fake news.

------
ratsimihah
What is going here, the world is looking more and more like a movie!
Apocalypse soon?

------
rm_-rf_slash
Massively underwhelming read from the headline. This is basically just
deepfakes for dancing. When machine learning can actually _teach_ someone how
to dance, then we’ll have something interesting on our hands.

------
revskill
AI couldn't be used for critical task, in which mistakes are not allowed at
all. What's its real use cases ?

------
m0ew
Very impressive!

------
ch4ck
Looks as fake as dinosaurs in 1993.

------
mraison
> _Using NVIDIA TITAN Xp and GeForce GTX 1080 Ti GPUs, with the cuDNN-
> accelerated PyTorch deep learning framework for both training and inference_

> _the team based their algorithm on the pix2pixHD architecture developed by
> NVIDIA researchers_

Is it me, or is NVIDIA trying very hard to take credit for this UC Berkeley
paper? (they're almost taking credit for Pytorch as well). Sure, this kind of
work wouldn't be possible without their hardware, but in that case Intel could
probably take credit for most of science in the last few decades.

~~~
twtw
It's a company blog. It exists to highlight applications of the company's
products. I don't see why you are bothered, both of those statements are
factually true.

~~~
mraison
Normally I wouldn't be bothered, but in this case I saw people being misled
into thinking Nvidia did this work. Given Nvidia publishes a lot at computer
vision conferences, there's a higher than usual potential for confusion.

------
xiphias
It looks impressive, I just don't understand why computer games don't use
these techniques to feature realistic human bodies/faces.

~~~
saagarjha
You appear to be shadowbanned. Most of your (reasonable-looking) comments are
showing up as dead to me.

~~~
xiphias
Sounds interesting, thanks! I had a few informative but controversial comments
in the past... that may be the reason.

~~~
Jaruzel
Seconded, I just had to vouch for you in the above comment to un-dead it.

Aha, it was when you said this:

[https://news.ycombinator.com/item?id=15725493](https://news.ycombinator.com/item?id=15725493)

------
m0ck
This issue was already solved with invention of booze and soft drugs.

~~~
WA
No, this changed only the self-perception of one being a great dancer.

