
A Nixon deepfake, a 'moon disaster' speech and an information ecosystem at risk - mrkn1
https://www.scientificamerican.com/article/a-nixon-deepfake-a-moon-disaster-speech-and-an-information-ecosystem-at-risk1/
======
akiselev
This Youtube channel has lots of other great examples of whats possible with
voice synthesis today: [https://www.youtube.com/channel/UCRt-
fquxnij9wDnFJnpPS2Q](https://www.youtube.com/channel/UCRt-fquxnij9wDnFJnpPS2Q)

Some of my favorites include Sinatra singing ABBA's Dancing Queen [1], six
presidents rapping the NWA classic [2], and Milton Friedman rapping 50's
P.I.M.P. [3]

[1]
[https://www.youtube.com/watch?v=zo_w4KGifug](https://www.youtube.com/watch?v=zo_w4KGifug)

[2]
[https://www.youtube.com/watch?v=mAZVp-n-5TM](https://www.youtube.com/watch?v=mAZVp-n-5TM)

[3]
[https://www.youtube.com/watch?v=4mUYMvuNIas](https://www.youtube.com/watch?v=4mUYMvuNIas)

~~~
adventured
I'm struggling to understand how they're great examples. They sound absolutely
atrocious and nowhere near believable. They sound really, really bad, like
something an amateur with zero editing experience could put together with
cheap equipment. They're so bad they're difficult to listen to, especially the
Sinatra one.

~~~
norswap
Agreed those are not the best examples.

This one (JFK reading the Rick & Morty copypasta) strikes me as much more
convincing however:

[https://www.youtube.com/watch?v=BkH2zEHJ2Ng](https://www.youtube.com/watch?v=BkH2zEHJ2Ng)

~~~
fokinsean
The Trump/Darth Plagueis copy pasta got a chuckle out of me.

[https://www.youtube.com/watch?v=LEzIAixNkFI](https://www.youtube.com/watch?v=LEzIAixNkFI)

------
mirimir
This is a fake, sure. Maybe it's not perfect, but the software is improving.
So within a few years, I'm guessing that fakes will be indistinguishable from
true recordings.

But this was based on the text of an actual speech written for President
Nixen, to be delivered if the mission failed. So isn't it likely that he
practiced it, just in case? And if he did, it might have been recorded, more
or less accidentally. As Reagan's quip about nuking Russia was.

And so as others note, it becomes crucial to look at the historical context.

~~~
mastazi
> As Reagan's quip about nuking Russia was.

Oh this is interesting, do you have any sources where I could read more about
it?

~~~
paulsutter
While testing an audio setup in 1984, Reagan joked that Russia had been
outlawed and “we begin bombing in 5 minutes”.

[https://en.m.wikipedia.org/wiki/We_begin_bombing_in_five_min...](https://en.m.wikipedia.org/wiki/We_begin_bombing_in_five_minutes)

------
Nasrudith
It is one thing to make apocalyptic predictions - they could be reasonable
conclusions of possibilities, unreasonable, or anywhere in between. They can
be discussed on their own merit. Heck even why they personally conclude
something may be discussed and considered, and something to try to understand
even if disagreed with or considered wrong. But the "here is how you should
feel" leading bullshit is a clear run-arround for critical facilities that
takes far too many people.

Nothing makes me write a source off as a bad actor more than panic mongering
that /tells you/ how to feel as opposed to leaving it to you to decide. It is
not only a sure sign of a manipulator because of scared people being easier to
manipulate. It may be unduly harsh, caustic, or sharp but I have absolutely
lost patience for this tactic.

~~~
082349872349872
The Onion's take on apocalyptic predictions:
[https://politics.theonion.com/conservatives-warn-radical-
kam...](https://politics.theonion.com/conservatives-warn-radical-kamala-
harris-will-impose-he-1844703306)

~~~
casefields
Did everyone forget about Prop 8 out here?

------
ggggtez
I was kind of expecting that this was going to demo some step forward, but
this has all the same tells has the last several high profile deepfake demos.

> Voice is clipped

> Chin

> head bobbing

~~~
frabbit
Knowing the speech was a fake and paying attention to it "skeptically" I
noticed the double head-bob as an odd thing which seemed suspiciously
convenient as a transition between cut-n-pastes.

Now I'm going to have to go and watch archival Nixon footage to see if the
same thing is there or not :)

------
jariel
What made it 'real' was the actual speech itself. It would have been hard to
'fake' that.

The 3 minutes of 'intro blast off' (that could have been 10 seconds) and the
Flash-like landing page are really problematic.

~~~
ggggtez
An AI can't fake a speech about a subject they never spoke about... yet.
Though, to be honest, I think it wouldn't be hard.

1) Collect a corpus of speeches.

2) Use style-synthesis techniques, just like in the art-style synthesis demos.

3) Input a speech about a spaceship disaster and run it in the "style" of
Nixon.

~~~
nullc
> An AI can't fake a speech

Nonsense.

"It is with a heavy heart that we must inform you that Apollo 11 has failed to
return from the moon. We have lost contact with our crew, Neil Armstrong and
Edwin Aldrin. When they left earth seven hours ago their fate was unknown but
now it is certain. They died as men should, ready to die for what they
believed. This loss is a tremendous loss for our nation, for their families,
and for mankind but their sacrifice was not made in vain: It is a testimony to
courage, determination, and human achievement that will be remembered by all
who witnessed it today and for all those who come for all time so long as man
walks this world or any other. For this tragedy will not mark the end of space
exploration; it will strengthen our resolve to continue advancing space
technology and human knowledge. We have enjoyed the fruits of their labor and
shared the pride of their achievement just as we now share their isolation and
grief. We will honor their memory by continuing that work and by taking that
faith and that dream into a future they will not live to see but helped to
build. We will remember them and we will remember what they stood for: The
greatness of man and the hope for a better tomorrow."

This isn't a first shot output, I had it retry a bit and guided it a little,
mostly to get it to write a longer speech instead of a short quote. I think
it's much better than I could have written on my own in a minute or two.

~~~
coolness
Wow, did you make this using GPT-3? Is the first sentence yours and the rest
generated?

~~~
nullc
All the text in quotes was GPT-3 generated-- with a little help from me (e.g.
when it went in the wrong direction-- e.g. ending the speech too early-- I
made it go back and try again or clipped out the dumb part and had it
continue).

I prompted it with some mission description and said that a speech was
prepared for president Nixon that in case the astronauts were stranded.

GPT3 doesn't yet do consistently GREAT output without some guidance, partially
as an artefact of the generation procedure. But with a little help it does
very well.

The issue is that if you just take the most likely symbol it'll rapidly go
into a loop of just copying text or other degenerate behaviour. So instead,
everything uses the model by sampling it-- taking less likely choices by
chance weighed by the model output. Unfortunately, that means that an unlucky
draw will occasional paint it into a corner. If you see that happening you can
just go back and try again and you get much better output.

If that is a fair comparison depends on what your application is... if you
need to to run unsupervised, it isn't consistently great. If you just need a
first draft out of it or some raw ideas to turn an hour writing task into a 5
minute one, it's great for that.

I don't think this kind of manual assistance is much of a cheat either-- a
real speech writer also gets exactly this sort of help from others.

[And FWIW, I did this via the GPT3 based mode in the ai dungeon video game.
... I don't have access to the GPT3 API.]

------
kristopolous
I was expecting the Walter Cronkite part to be the fake thing here in a clever
twist.

They'd pass through the Nixon questions and then say "Alright, but what about
Walter Cronkite?" My true advocacy would be to use another contemporary big
three anchor from the era as the deep fake, then show the real Cronkite
footage during the quiz at the end

That's the real lesson IMHO, not if you can tell when you're prompted to
listen for it, but when you aren't.

------
brownbat
People have been doing this since the cannonballs photo in the Crimean War.

If you think a fully fabricated video is the key to lying to people I have a
poker game you can join.

Here's a discussion of how far you can distort video today, in a way that is
way more subtle and harder to rebut than a total fabrication:

[https://youtu.be/tEGqepsFTbI](https://youtu.be/tEGqepsFTbI)

------
hyko
If deepfakes are so good, why do they always look and sound so crap?

~~~
SamuelAdams
They don't need to be perfect. They just need to be convincing.

Consider MP3 files. Any audiophile will tell you that a compressed MP3 file is
a piss-poor representation of the true sound. The best experience is to listen
to a band live, then vinyl, then FLAC.

And yet the majority of people listen to MP3 files. They strike the right
balance of file size and sound clarity for an overwhelming amount of systems,
doubly so since online streaming took off. So now that people have become
accustomed to the sound of MP3, they are not used to anything else. They are
convinced that an MP3 file is the "true sound" of the song.

Right now deepfakes is in its infancy. Personally I think it is improving at
an alarming rate. If I shared the moon video on Facebook I am willing to bet
most people would think it is real. They don't notice the clipped speech, the
subtle double head nods. Their minds are not as critical of video as you and I
are. So to them, they are convinced it is real.

What happens when only machines can detect if something is authentic or a
fake? What happens when not all site administrators scan for videos and fail
to mark them as illegitimate? What happens when courts use deepfakes
unknowingly as evidence to convict someone, or impeach a president, or fire a
worker for sexual harassment? We have already seen what happens when
"verified" twitter accounts are compromised - what if a CEO puts out a video
announcing some controversial new endeavor, or admission to fraud?

There are very real concerns about this technology and I believe it will very
soon become a weapon that takes misinformation and turns it into very real
consequences.

~~~
cookienapper
Misinformation is already a huge issue; deepfakes can only exacerbate and
compound the issue in ways we can't even imagine at this time.

Imagine deepfake used to create false alibis, 911 calls, etc... That's
probably not even the tip of the iceberg in coming years.

The fact we went in KNOWING it's a deepfake gave us an unmeasurable
advantage/bias. For uninformed individuals that "stumble" upon this on the
web, one can only imagine how it'll playout. This thought actually terrifies
me... imagine a hypothetical (and conservative) 10% overall improvement AND
production price reduction to this video every 6-12 months.

@SamuelAdams: Wish this site had a messaging system. Would be interesting to
have a 1 on 1 discussion on this topic with you.

~~~
SamuelAdams
You mention phone calls. Right now robocalls are very common. But what if
those random calls legitimately sounded like your parents, or your spouse, or
your kids. Furthermore, what if they sounded like they were in real trouble
and really needed money fast? This technology opens up whole new arenas to
more legitimate scams.

~~~
kube-system
This is already being done to scam high profile targets.

Example:
[https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice...](https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-
deepfake-was-used-to-scam-a-ceo-out-of-243000)

------
leahey
This is absolutely remarkable.

~~~
d2v
Definitely felt a little off, but if I didn't know that this was a deepfake, I
would probably accept it as real. As usual, I don't know if that fits more in
to the "remarkable" or "disturbing" category.

~~~
Izkata
Eh, I'm not entirely sure I would have. The effect is near identical to when
you have one person behind another, pretending to be the front person's arms
and hands - the motions just don't match up right.

Maybe people who use faces more than full body language are more affected by
this particular one?

~~~
d2v
I kind of agree, but it's hard to separate knowing that it's fake from my
reaction to it. If this were presented without that context, I might feel like
it's off, but not necessarily question it, especially if it came from a source
I believed to be legitimate.

~~~
m-watson
I also think, on top of all your points (which I agree with), keeping it
shorter would make it more convincing. I think with the current usage of
social media, short clips created through some synthetic means will be harder
and harder to identify.

------
00N8
I remember watching a TV ad where Michael Jordan around the age he was when he
retired played 1-on-1 against himself as a college player. This was 15-20
years ago & I wrote an essay for high school English class on how that ad
shows we can't believe our eyes when it comes to what we see on TV. It was
visually much more convincing than the examples I see here, so I can't help
but think deepfakes haven't quite caught up with professional editing, at
least not yet.

------
rmason
For the majority of those on HN who didn't witness the live event this is a
pretty good summary. It really took me back to that day as our entire family
was glued to our black and white TV watching the man known to all as Uncle
Walter. Watching President Nixon twitch a bit was the only tell that this
wasn't real.

Got me to imagining the video where Eisenhower confesses D-Day was a failure,
the actual speech he wrote does exist.

------
Bayart
It works as a PoC, and it could fool me if I saw it at the edge of my vision,
on my kitchen's TV. But otherwise it _looks_ odd, and most importantly it
sounds like the weirdest vocoder in existence.

We still can't truly fake instruments, never mind the human voice.

------
jolmg
> Beyond this illegal and harmful use of face swapping

Is face-swapping illegal? What law is it breaking?

------
systemBuilder
I really hate the fake static in this video - as a 7-yr old I watched the moon
landing in 1969 and nobody's TV was that bad! If I had a TV that bad I would
either repair it or throw it away!

------
echelon
My website has 40+ celebrity and cartoon voices:

[https://vo.codes](https://vo.codes)

I'm working on voice conversion (voice to voice for TeamSpeak/Discord) now.

~~~
woodson
Any chance you would be willing to share the (finetuned) models? For science?
I’m interested in detection and want to look at artifacts common to speech
audio generated by different models all based on the same overall
implementation.

------
mellosouls
Previous discussion:

[https://news.ycombinator.com/item?id=23896996](https://news.ycombinator.com/item?id=23896996)

------
qwerty456127
Deepfake stuff is not a threat, it's a vaccine. There finally is a chance
people will realize they can believe nothing and nobody, news lie and mislead
and they absolutely have to triple-check everything before they take that in
consideration.

~~~
pjc50
This destroys democracy. Because you can't check everything yourself, or even
a tiny amount of things, so you have no meaningful basis on which to tick that
box on the ballot.

~~~
qwerty456127
You never had it. It has always been an illusion. Now this just got harder to
ignore.

~~~
pjc50
Great. Now what?

~~~
dnate
Trust in media with a reputation. Trust in journalism to unearth attempts of
deceit. Quality journalism and reputation will become more important and
powerful than ever

------
frabbit
3m06s to 3m15s seemed a bit cheesy and stagey

------
TheButlerian
Has something positive came out from the world of IT in the last 20 years?

------
paulie_a
It happened

------
m0zg
Bullshit. Easily 80% of broadcast news today is fake, either overtly, or by
omission. It only exists to advance a narrative, not to inform or support
debate. As Denzel Washington quipped: "If you don't watch the news, you're
uninformed. If you do watch the news, you're misinformed."

The ability to produce fake news artificially as well won't tip the scales
even slightly, because nobody with a brain trusts the "news" anyway.

