
Synthesizing Obama: Learning Lip Sync from Audio - taytus
http://grail.cs.washington.edu/projects/AudioToObama/
======
tmuir
Does anyone remember about 10 years ago, maybe 15, there was a guy somewhere
in Europe that created something similar, as his graduate or doctoral thesis.
He demonstrated it by taking the video of Michael Jackson denying sexual abuse
allegations. However, he, the creator, was speaking into the system, and the
video and audio of Michael Jackson "saying" the same thing was playing in near
real time.

He explained his system at a fairly detailed level for a short 5 minute video.
He was breaking down the source clip into very small snippets of time, and
with some algorithm, giving each snippet a single value, or a score. Then, as
his realtime audio was analyzed in the same manner, the source snippet with
the closest score to the realtime snippet was output, both audio and video.
I've always wondered what happened to that project, and I've since forgotten
the creator or the name of his project.

~~~
rzzzt
Scrambled Hackz! I also remember running across this video.

------
doesnotexist
So I shared the video with a friend yesterday. To my surprise they were not
convinced. Which makes me wonder if it breaks down for very adept lip readers.

~~~
delinka
It doesn’t pick quite the right movement for some of the subtler sounds. I’m
still trying to quantify where it breaks down for me.

~~~
garblegarble
For me, I think it's that the movement of the rest of the mouth and the
muscles/skin in the face (and the light/shadow as a result) doesn't match up
with what I expect. It looks like the facial animation from a good videogame
cutscene

------
Mashimo
The future is scary.

* deepfake can superimpose a face on another face

* Adobe is developing a program to make a voice say anything (from a couple of hours of voice input)

* this can sync you mouth to the new voice generated by adobe.

Combine all three.

~~~
Spivak
Eh, it won't be so bad. People are already accustomed to the fact that text,
handwriting, and images can be easily faked. Voices can already be
convincingly faked with practice prior to software assistance and there are
lots of classes of videos can be faked with effort like security footage.

People aren't dumb and it'll only take a few good fakes to get people to
distrust videos not attached to a reputable source.

~~~
Stanleyc23
doesn't all the controversy around fake news support the exact opposite of
your claims that people are smart enough to only listen to reputable sources?

------
braindongle
I would love to hear a digital forensics expert weigh in on the doomsday
prophesies surrounding this sort of thing. Yes,digital fakery is going to get
more and more convincing, but we are not without weapons in the defense of
truth.
[http://ieeexplore.ieee.org/abstract/document/5487389/](http://ieeexplore.ieee.org/abstract/document/5487389/)

~~~
whataretensors
I don't think this will work in the long term. Adversarial learning should
learn any differences.

------
tootie
Here we go. The end of believing anything you see.

~~~
empath75
So— there is a way to defeat this, which is to track down the original source
video that was modified to match, but yeah, in a few years this is going to
wreak havoc on news reporting, intelligence, and the justice system.

People are going to have their lives destroyed by intelligence services over
this.

~~~
Barrin92
>People are going to have their lives destroyed by intelligence services over
this.

there's a paradoxical effect to it though. If these technologies become so
pervasive that they're not distinguishable from real footage they'll lose
their entire value, especially in a legal case. If you could arbitrarily
create fake dna tests there'd be no point to pay attention to the results.

~~~
polyomino
You’d think so wouldn’t you. AFAIK some courts still accept lie detectors as
testimony.

~~~
Barrin92
I didn't actually know that countries are still using them. I thought they
were a thing out of 80s American crime films. Consider me surprised

~~~
77pt77
I think they are quite common in Israel.

[https://www.timesofisrael.com/when-a-job-interview-turns-
int...](https://www.timesofisrael.com/when-a-job-interview-turns-into-an-
interrogation/)

> In Israel, too, the results of a polygraph test are inadmissible in criminal
> court but in civil court they are fair game if the person being tested
> agrees to it in advance.

------
js2
Relevant Radiolab episode:

[http://www.radiolab.org/story/breaking-
news/](http://www.radiolab.org/story/breaking-news/)

[http://futureoffakenews.com/videos.html](http://futureoffakenews.com/videos.html)

~~~
fgdelcueto
I immediately thought about that episode as well. We need to prepare for the
future of fake news on steroids when this gets even better. It's really scary.

------
thefounder
This is scary.

