
Fake Obama created using AI tool to make phoney speeches [video] - mychaelangelo
http://www.bbc.co.uk/news/av/technology-40598465/fake-obama-created-using-ai-tool-to-make-phoney-speeches
======
jnbiche
I was really suprised when the original paper for this didn't make it to the
top of HN:
[http://grail.cs.washington.edu/projects/AudioToObama/](http://grail.cs.washington.edu/projects/AudioToObama/)

Nor any of the follow-up articles I posted. Given onslaught of fabricated
"news" that spread around the last election, this type of ML technology is
almost guaranteed to play a role in the next one.

~~~
lightbyte
After watching the video from that link, I'm very fearful for what people are
going to do with this technology in the next decade. This combined with
Adobe's new "speech photoshop" will make excellent propaganda.

~~~
saganus
At first I was also fearful of people using this tech to make fake "evidence"
of things...

But now that I think about it, the most damage will come from making people
even more incredulous... of everything.

It will become even harder to use evidence to prove a point. Uninformed people
will just say "yeah sure, it's probably faked".

On the other hand... who could blame them?

~~~
folksinger
I'm not surprised that a forum full of people who work in the information
technology industry cannot see the epistomological failure of mass media, but
these problems are not new, and the idea that nothing communicated through a
screen can be trusted will most definitely cause more good than harm outside
of the very near future.

Democratic institutions not only functioned but were put into effect before
the advent of recorded or transmitted audio and video. Mass media has already
been distorting the truth for over a century.

"Trust nothing you read online" becoming commonplace is something to be
celebrated!

~~~
erikpukinskis
There's also another possibility: that media literacy will take off, and
children will learn to verify sources. This was the original proposal early
blogospherists: that ordinary citizens would learn to do journalism so as to
keep microcommunities informed.

This is in fact happening, to everyone's benefit. Only in matters of vague
mass policy (do we need giant walls? can markets sell health?) has it been
largely corrupted by corporate propaganda. Which is probably not terribly
worse than before. That scale has always been a big enough target to make it
worth the effort to corrupt.

------
nkrisc
I wonder if it's easier to make a fake speech of someone who talks the way
Trump does? Not making this a political thing, but I've noticed that he's not
very eloquent and frequently starts and stops in his speech and changes what
he's saying mid sentence. Might it be easier to synthesize that type of speech
pattern since any awkwardness can be hid in the erratic speech the real person
employs?

~~~
pjc50
I've seen claims that translators struggle with Trump because of this -
accurately translating what he says comes out as incoherent, which makes
people think that the translator is doing a bad job.

Although I think to some extent he _relies_ on fragmented speech. It's all
about emotional pattern recognition, rather than something that people engage
with rationally. Tony Blair also tended to do this, with long verbless
sentences.

~~~
Cenk
The Daily Show had a segment on this:
[https://www.youtube.com/watch?v=7qL1un6NPZA](https://www.youtube.com/watch?v=7qL1un6NPZA)

~~~
mnm2
Wow, that's like 100% scripted to please the target audience

~~~
ryanx435
yeah. The Daily Show has got a pretty standard formula. I read an article
about how they, and their spin off shows like John Oliver, basically use the
same script pattern for most of their shows in order to hypnotize their
audience.

Hold on, I'm going to try to find the article.

edit: well, the only source I can find after a quick search is this post on an
anonymous message board, but it covers the main points pretty well:
[http://img.4plebs.org/boards/pol/image/1479/14/1479145599055...](http://img.4plebs.org/boards/pol/image/1479/14/1479145599055.jpg)

a more detailed discussion is here:
[http://archive.4plebs.org/pol/thread/98690960/](http://archive.4plebs.org/pol/thread/98690960/)

~~~
somedangedname
"The SJW is the odious little Eichmann with his jackboots up on a fair trade
coffee table..."

Really??? That's rich coming from /pol

------
Diederich
When an open source package author releases a pile of source code, that person
typically also releases a cryptographic checksum of the code.

In the future, when the President (or CSPAN, or CNN, or Fox News, or whoever)
releases a segment (which they do all the time), they'll need to release (in a
public, 'timestamped' way) a cryptographic checksum of the content.

I have many of the same fears as people here about future fake news, where the
reality of something already comes as a distant second behind the outrage
produced. So even if we had this big pile of content and checksums, the
outrage echo chambers will still be going nuts.

But it's at least a partial technical solution to these problems.

(And I'm glossing over all kinds of other complications too, such as 'what
format' and 'where does it get stored' etc etc)

~~~
tudorw
I think it's reasonably certain some alternative network with these kinds of
features will emerge, validating authenticity of the contents origination was
a known requirement of theoretical networks , because it's hard, both
technologically, but more fundamentally the sociopolitical aspects are mind
boggling, loss of privacy, trust in the state, etc etc, this will take a while
to resolve, but someday people will be looking back and laughing :)

~~~
Diederich
I spend a non-trivial amount of time hoping that people, myself included, in
the future will look back at many things and laugh. (:

------
scarmig
It makes you wonder just how far fake content--and fake content involving real
people--can go.

Imagine a world where a service exists to which you can upload a dozen images
of someone, along with a voice clip. In response, it can generate all kinds of
videos--from the benign, to the person saying horrible racist things, to the
person starring in graphic pornography.

It seems technically feasible in the medium term. But how do we react to it?
Strict limits on the production or storage of these pseudo-artifacts? Criminal
penalties for distribution? A cultural rejection of pretty much all video and
audio evidence?

~~~
Balgair
Oh man, it's gonna get way crazy. Imagine the 'mean girls' in HS with this
tech. Imagine trying to be a male HS teacher with this around. We're gonna
have to regulate the shit out of the internet just to keep teachers in schools
and verify that they did not in fact cuss out a classroom and chug vodka.

------
rybosome
I'm truly frightened by this. We are still struggling with how to deal with
the "post-truth" world, and that's with the assumption that pictures are hard
to fake, video even more so, and audio nearly impossible.

Fake news is going to reach a fever pitch when "speeches" of Obama leak
saying, "We have to take all the white people's guns". And conversely, when a
genuine "grab 'em by the pussy" leaks again, a huge chunk of people willfully
will not believe it.

Seems possible that we could come up with technological and journalistic
solutions, given enough time, but it's moving too quickly.

~~~
emodendroket
From the material posted it seems that this is basically a high-tech,
convincing version of those late-night sketches where they have a fake version
of their mouths moving; it doesn't actually solve the audio problem.

~~~
cmahler7
would be pretty easy to use an impersonator or something like lyrebird.ai to
create the audio.

~~~
frgtpsswrdlame
Yes but you can also track down the original video to prove something is fake.
In order to have something truly convincing you need a video of Obama (or
anyone else) that no one else has.

------
EGreg
I have been saying for a while now that our current systems are all relying on
the inefficiency of an attacker.

Soon, video and audio of an event or speech be proof of anything.

The only way to prove identity will be to have a device which can do
challenge-response.

Without it, you won't be able to prove you're not a robot over the internet.

[https://news.ycombinator.com/item?id=14787882](https://news.ycombinator.com/item?id=14787882)

Forget "hacking elections". A botnet will be able to hack our trust in one
another ( _see CIA reputational attacks_ ), AI will be used to chat up girls
online better than any person ( _see fb AI sales bots_ ), and so on.

Computers can already beat us at Chess, Go, etc. How much different is humor,
honor and reputation once companies add one more breakthrough to deep learning
to model them?

An attacker that can make 100,000 jokes a second each of which is excellent?
The missing breakthrough is how to automate the "human judging" factor. This
is the problem when figuring out diets or treatments etc. Clinical trials take
a long time. Same with textbooks.

Once we figure out how to speed that part up, we are going to be able to make
AI that knows what's probably going to be funny ahead of time.

------
samcodes
Looking ahead 10-20 years, I don't see how anyone born from 2015 onwards has
any solid concept of reality the way I feel I know it. Between things like
this and AR, I feel like "real" vs "simulated" will seem to the AR natives to
be a pedantic distinction... Like how my parents still distinguish between
"having met someone" and "someone I have talked to on the internet."

~~~
emodendroket
> Like how my parents still distinguish between "having met someone" and
> "someone I have talked to on the internet."

I am not sure that that distinction is really gone.

~~~
cmurf
Where'd you two meet? On the internet.

~~~
emodendroket
I think it's a bit of a stretch to claim that because such an exchange is
plausible there is no meaningful distinction between online and in-person
communication.

~~~
jacoblambda
I really don't think there is much of a difference anymore. I remember meeting
people online and hanging out with them (talking, working on projects
together, playing games, etc) and it honestly didn't feel any different than
meeting someone in person and doing the same thing despite the fact that I
have never seen some of these people in person.

I find it especially difficult to differentiate a meaningful distinction
between the two when my interactions with people go from in person to online
or vice versa. We just do the same stuff we did normally, albeit maybe those
things occur in different settings such as at a computer instead of face to
face at a restaurant or wherever.

------
kmfrk
I really recommend the Adobe demonstration of their VoCo speech audio
"photoshopping" because of Jordan Peele's reaction:
[https://www.youtube.com/watch?v=I3l4XLZ59iw](https://www.youtube.com/watch?v=I3l4XLZ59iw).

More technology demos should have unscripted, sincere reactions like that.

~~~
yorwba
> More technology demo should have unscripted, sincere reactions like that.

I'd agree with you if you scratched _like that_.

See here for an opinion on this demonstration:
[https://news.ycombinator.com/item?id=13002787](https://news.ycombinator.com/item?id=13002787)

------
arnaudsm
The next generation of fake news is going to be fantastic.

~~~
avaer
I look forward to a time when journalists doing journalism is what's valued in
the news.

When technology to fake things is everywhere, maybe we'll pay for accurate
newspapers again. With something other than our willingness to be manipulated
by advertising.

~~~
dragonwriter
> When technology to fake things is everywhere, maybe we'll pay for accurate
> newspapers again.

We didn't pay for them _before_ , how could we do so _again_? Advertisers have
always been the main funding for newspapers and sensationalism (clickbait
before clicks) and propaganda have always been major influences on the medium.

Sure, things got worse for decades leading up to the arrival of internet news,
and continued to get worse after that, but the golden age of readers footing
the bill for accurate newspapers _never happened_.

~~~
avaer
This is kind of a slap in the face to anyone who's considered themselves a
journalist, as well as anyone who bought a newspaper for the news. It also
makes me wonder how the news industry ever got started if it never sold
"news", but only something wrapped in that veil.

I don't know anything about the history of news; do you have any good reading
material to recommend?

------
stevenh
The audio itself can also be faked now.

[https://lyrebird.ai/demo](https://lyrebird.ai/demo)

~~~
Beltiras
That would not fool anyone. Impressive nonetheless.

~~~
azinman2
It’ll only get better over time.... by them or someone else.

~~~
azinman2
Just imagine what will happen when state actors start to use this as part of
psychological warfare...

Future looks grim!

------
Oras
What worries me is that this is just a public research done in uni. What
should we expect from secret projects then?!

------
dheera
[http://starecat.com/content/wp-content/uploads/frequency-
of-...](http://starecat.com/content/wp-content/uploads/frequency-of-miracles-
time-graph-camera-invented-photoshop-invented.jpg)

Basically, we now have the video version of this graph. It came a little later
than Photoshop, but as with any technology, anything that is technology
possible will be implemented by somebody at some point in the future.

------
armenarmen
After the Falkland island war the British anarchopunk band Crass spliced
together and leaked a tape that made it seem like the attack was a false flag
and that Reagan wanted to start a war with the soviets in Europe

[https://en.m.wikipedia.org/wiki/Thatchergate](https://en.m.wikipedia.org/wiki/Thatchergate)

------
isaaclyman
This demonstrates an immediate need for a trustworthy video hosting service--
maybe with a companion app that records using some combination of
crypto/proprietary formats/trust networks/I don't know what else, this isn't
my area of expertise. Things are going to get pretty bad here soon if video
evidence loses its street cred.

~~~
untog
But how is it considered trustworthy? That's basically the central struggle
every media organisation has right now and I don't see an easy answer.

Yours is an interesting idea, though. I dread to think how much it would have
to exclude though - probably have no Android client because I'm sure a rooted
phone could easily provide a fake video source. Maybe even on a jailbroken iOS
device too. And once that becomes possible the entire platform is ruined.

~~~
isaaclyman
Yeah, it might need to be a custom-made camera, made for news organizations
and wealthy vloggers. And each device would need to be audited regularly. It
would be a nightmare but not as much of a nightmare as not knowing whether
anything you see on video is real or computer-generated.

------
agentgt
Now we just need some way to digital sign speeches (and hopefully eventually
with quantum cryptography).

I'm not sure how but perhaps this could actually improve the issue of fake
news with or least the assimilation if there as a broader realization how
easily things can be faked... probably not though (I'm eternally an optimist).

~~~
Retr0spectrum
Signatures only solve half of the problem.

What if someone is filmed doing something wrong?

They wouldn't sign it, and anyone could just say the footage was faked.

------
transitionnel
Well...this kind of thing is here to stay, and at least some good may come of
it. We can all be reminded that even "true" words are still just words.

An example from all of history: (This is only semi-serious, but felt like a
good thought experiment).

> Oligarch to politician - "Make this economic change." > Politician to team -
> "Give me post-facto justification for this change I am making." aka "Spin
> this" > Team - _Applies economics to numbers_ > Team to politician - "Here
> you go." > Politician to people - "Economics does not lie." > Vaunted
> economics publications - "Sold. And thanks for the like." > _Economic
> failure_ > Future politician - "Well, we just didn't know then what we know
> now." > Historian of the future - "Their economic calculations lacked the
> full set of economic forces and incentives. The economists of the time were
> in effect hand-waving because they ignored a fundamental economic force--the
> oligarch. Given the size of the oversight, I'd say they were complicit."

Thoughts: \--Economic policy sold without disclosure (or even acknowledgement)
of these massive forces is knowingly flawed, and a willing lie to handle
people. \--Economic theory is rooted in psychology. When an economic decision
is spun to cover hidden motives, the psychological motive basis of that
instance of economics is, by definition, false. \--Data can still fool good
economists when it is cherry-picked; any data produced by a non-omniscient
process is going to be flawed to some extent.

* This is not to say anything good or bad about oligarchs. Merely that they are a tremendous force, and economics, political policy, and civil discussion could greatly improve with a more accurate model of their effect on global systems.

* I'd love to see an economic modeling tool able to place "black boxes" where market distortions are occurring due to probable hidden forces. Captive markets are a real nuisance.

------
theelfismike
I hope they call it Fauxbama.

------
stillsut
There's been a growing fear of how much dirt is going to get spilled when the
current generation with omnipresent camera-phones goes to run for office. But
this signals that we could be nearing the end of the age of embarrassing
leaked video or audio!

From now on, it could be plausible to deny a video and say someone built a
neural net and faked it - 'at least I have no recollection of those events.'
(The excuse will work for supporters)

The Mitt Romney's '47% are takers' could go down historically as the last
great leak where leaks could be believed.

------
dkarapetyan
Time to learn how digital signatures work.

------
skc
If you've ever gone through the pain of trying to convince someone why some
picture or video is an obvious fake then you know just how terrifying this
technology is.

------
LeoNatan25
Like with other, more easily forgeable conteby, such as text quotation, we
will find ways to provide a chain of trust in order to authenticate what has
classically appeared much more difficult to spoof, such as voice or video. The
only challenge that will remain is explaining to people that the nonsense they
watch on Facebook is not factual just because it is on Facebook.

“Use the force, Harry” – Gandalf

------
e12e
Is the first section of the video badly out of sync for anyone else? Both the
real and fake initial segments appear way off to me?

~~~
ipunchghosts
Came here to say the same thing.

------
nocoder
Isn't this similar to Adobe project Voco? If not how are they different. There
is also another tool called Face 2 face. I think overtime people will assume
everything to be fake unless it agrees with their pre-conceived notions of
truth.

~~~
jessaustin
_...their pre-conceived notions of truth._

This is the wrong way to think about this, I think. Think of media consumers,
instead, as Bayesians. We have prior beliefs, we consume media, and our
posterior beliefs are a function of _both_ of those. As the process iterates
over time, our priors are updated, but more slowly over time.

This is why children believe everything they hear from authority figures,
while those with memories e.g. tend to doubt the offered justifications for
the new war when we consider how similar they are to the previously-offered-
but-eventually-disproved justifications for the previous wars.

~~~
nocoder
Perhaps...but this has been my observation recently. A persons ideological
position is always used as a gauge to judge the quality/value of his opinion
on a particular topic. If a person is of/ perceived to be of an ideological
disposition different then mine and then whatever he or she says gets
discounted. Your argument about people being Bayesian assume that they are
rational. In practice, you find people sticking to their prior beliefs even
more strongly even when the evidence to contrary is stronge, so called
backfire effect. In fact, a lot of people reject the evidence simply based on
the Ideological disposition of the source that it comes from and terming it as
XX wing conspiracy.

------
kristiandupont
I guess that we will start to rely on multiple recordings of an event to
consider it proven. Until, of course, 3D recreations start to appear rendering
that useless as well.

~~~
honestoHeminway
I dread the narcisim this allows to unleash, as people will just select
whatever they believed in the first place to be true.

The only solution to this is a education towards knowledge masochism. Destroy
what you believe in, only in failure there is truth.

~~~
24gttghh
> I dread the narcisim[sic] this allows to unleash, as people will just select
> whatever they believed in the first place to be true.

I think we've gone past this point already.

------
aaron695
This also may allow high quality educational videos to be created. (Think of
all those smart people who are bad presenters)

And high quality entertainment.

Ain't all bad.

------
imranq
What's the benefit with a technology like this? Seems like it can only cause
harm

~~~
evan_
If you were able to make it work in real-time you could do some really cool
stuff with characters in multi-user VR spaces. Your avatar could be shown to
move its mouth in a realistic way as you spoke to other users, which would go
a long way towards building a feeling of "realness".

------
_pmf_
"And make no mistake."

------
wubbfindel
I read a book last month that featured a faked president video created by an
AI.

Kill Process by William Hertling Link:
[http://amzn.eu/c1ZJNcv](http://amzn.eu/c1ZJNcv)

Seems it was closer to non-fiction than I expected.

~~~
wubbfindel
Why the downvote? Am I not allow to link to a book that shows fact has caught
up with fiction?

------
signa11
and then you have this as well: [https://lyrebird.ai/](https://lyrebird.ai/)

------
cmurf
Proof of provenance.

We're going to need signed videos...

------
jbrl
Well it's better than what we have now...

