
Creating a deepfake took two weeks and cost $552 - panarky
https://arstechnica.com/science/2019/12/how-i-created-a-deepfake-of-mark-zuckerberg-and-star-treks-data/
======
simias
A few weeks ago my girlfriend made me discover Instagram filters. Obviously
they're not in "deep fake" territory but I was still massively impressed by
the quality of the face recognition and tracking coupled with some rather
advanced 3D reconstructions on top.

15 years ago that would've been science fiction, today it's a free gimmicky
feature in an app running on commodity hardware. There are also apps that will
smooth your skin or change the shape of your eyes in real time.

All these technologies are evolving at a ridiculously fast pace, I'm sure that
a decade from now you'll be able to do a much more convincing fake in a
fraction of the time. We're getting deeper and deeper into a post-truth world.

~~~
ryanwaggoner
I don't really understand this argument at all. We've been in this "post
truth" world for a long time already for every form of media other than video.
Text quotes, photos, and audio can all be easily faked. If I post a ridiculous
quote here and say it's from Obama, you won't believe me. But if the NYTimes
does the same thing, it carries a lot more weight. We've been here for a long
time already, now those standards will just apply to video as well. Doesn't
seem that crazy.

Also, I'm fascinated by the idea of using AI to detect the use of AI. How good
will your deep fake have to be in order to fool a deep fake detection AI?

~~~
ceejayoz
> But if the NYTimes does the same thing, it carries a lot more weight.

Sure, and that's part of the danger. Convincing deep fakes will be used to
delegitimize mainstream news organizations, as with what happened with Dan
Rather - a trusted source presented faked documents.

~~~
ryanwaggoner
But why is this the danger now in a way that it hasn't been previously? You
could have presented fake documents for hundreds of years now, but it's not
like we don't have any credible news organizations as a result.

~~~
adrianN
It gets increasingly cheap to manufacture convincing fakes, but it doesn't get
any cheaper to do quality journalism. The risk as I see it is that there is
such an overwhelming tsunami of manufactured reality that it becomes
impossible to tell truth from fiction.

~~~
setr
It was never that expensive to manufacture fake text. Minorly expensive to
manufacture fake images.

And we've been hit by tabloids, spam and so on before. None of that is new.
Overwhelmed by content is/was/will be a problem that is solved very simply:
people limit the distribution lanes they consume, and naturally establish
"trust networks" and hierarchies.

Nyt didn't become "trusted" by accident. People aren't as dumb as you seem to
think.

Anyways the problem is of distribution, not that a lie can be told. If we
can't trust our distribution lanes to actually reflect the institutions we're
trusting, then we can't establish our trust networks.

And this is a real problem: our distribution lanes are divorced now from our
actual information distrubuters; medium, Facebook, Twitter, etc fuck around
with our trust networks, randomly injecting their own bullshit into our feeds
and messing around with feed-order based on non-trust metrics (eg money paid)
such that they've become fairly unreliable.

And our classical trustable institutions have become less trustworthy, as they
flounder about trying to make sense of the "digital age", and have so far done
so in a pathetic fashion

~~~
adrianN
In the past you had to pay a person to write that fake text. Wait a few more
years and you'll see convincing text coming out of some algorithm at the cost
of an EC2 instance.

~~~
setr
That's fine and well, but again, I don't think the existence of it is new; if
they generate 1 lie a day, or 1000, or a million, it doesn't change much. It
gets resolved by the trust distribution network.

When that network breaks down, you only need a single good lie to do the
damage.

------
gdubs
The central technique used in DeepFakes is fascinating. Initially I assumed
they were using Generative Adversarial Networks (GANs). But — and someone can
correct if I’m wrong — Deep Fakes use two Autoencoders.

So essentially you’re training a network how to compress an image down to a
very tiny representation, and then uncompress it as accurately to the original
as possible.

You train two of these: one for the original face, and one for the target
face. Then, you compress with the “source” autoencoder, and then _uncompress_
with the “target” autoencoder. And, voila, ‘source‘ face becomes ‘target’
face.

~~~
tralarpa
How do you guarantee that the two trained autoencoders speak the "same
language", i.e. the code learned by the autoencoder of the original face gives
a reasonable output when uncompressed by the target autoencoder?

~~~
orange3xchicken
This is a good question. I'm not sure which paper the parent comment is
referencing - and in general, DeepFakes is a very general term for a variety
of techniques including GANs. The problem you describe is called the alignment
problem and it's studied in several areas of applied ml: translation, multi-
modal learning, etc.

For example, the canonical way to train deep machine translation systems is to
make use of large parallel corpora - large set of translation pairs. In the
case where this data may not exist, for structurally similar languages, one
approach is to learn unsupervised word embeddings independently (i.e. using
word2vec) which only require monolingual data and to find a transformation
that aligns the two spaces. In many cases, neural networks still struggle with
this task. However, some work in the last few years has demonstrated that a
few seed word-translations is all you need to apply simple algorithms that can
learn a linear or simple nonlinear transformation that work pretty well!

[1] Here is a method that proposes locally linear transformation maps for
alignment: [http://nakashole.com/papers/2018-emnlp-
norm.pdf](http://nakashole.com/papers/2018-emnlp-norm.pdf)

------
xedeon
That's a terrible Deepfake to be honest. This is a much better example.

[https://www.youtube.com/watch?v=bPhUhypV27w](https://www.youtube.com/watch?v=bPhUhypV27w)

~~~
SmallDeadGuy
More convincing for the face, sure, but last I checked Schwarzenegger had
much, much more body mass than in that video. Does anyone know if the
technology for faking body shape (and accessories too maybe) is anywhere near
as good as for faces yet?

~~~
xedeon
> More convincing for the face, sure

Uh.. That's what a DeepFake is? Look up the definition.

~~~
SmallDeadGuy
From
[https://en.wikipedia.org/wiki/Deepfake](https://en.wikipedia.org/wiki/Deepfake):

> Deepfakes (a portmanteau of "deep learning" and "fake"[1]) are media that
> take a person in an existing image or video and replace them with someone
> else's likeness using artificial neural networks.

"Likeness" refers to appearance in general, not just the face.

------
bryanrasmussen
Well Zuckerberg and Data do look like they were separated at birth, which I
guess would make Zuckerberg Lore - which might also explain a few things.

~~~
goatlover
So FB is the crystalline entity or the rogue Borg?

~~~
bryanrasmussen
Facebook does seem to have a strategy of assimilating every successful social
platform.

------
DyslexicAtheist
interesting conclusion:

 _> In the long run, the larger risk may be that public attitudes swing too
far in the opposite direction: that the possibility of deepfakes completely
destroys public trust in video evidence. Certain politicians are already in
the habit of dismissing media criticism as "fake news." That tactic will only
become more effective as public awareness of deepfake technology grows._

Deepfakes themselves will not _disrupt_ democracies or cause the much
anticipated chaos. Deepfakes are in fact pointless for agitprop / disinfo
campaigns:

 _" cheapfakes beat deepfakes"_ \--@thegrugq

[https://twitter.com/thegrugq/status/1206404680223358976](https://twitter.com/thegrugq/status/1206404680223358976)

[https://medium.com/@thegrugq/cheap-fakes-beat-deep-
fakes-b1a...](https://medium.com/@thegrugq/cheap-fakes-beat-deep-
fakes-b1ac91e44837)

What is fascinating (and uncommon) is, that the damage isn't delivered with
any specific deepfake itself. But the mere awareness that the technology
exists, and that they could _potentially_ be used, is enough to change our
risk perception. Even if the ratio cheapfakes/deepfakes usage in actual
disinfo campaigns remains tiny percentage, deepfakes make much better
headlines. (in a similar way the infosec industry is obsessed with 0days - but
most of the risks are coming from people clicking on links in email)

Maybe one day we will look at deepfakes like "art". If pictures and videos can
be art, and if "all art is propaganda" (see Orwell), then deepfakes too should
be considered art, no?

------
thathndude
I mean. No.

I understand the point of the article is “look how much I did with so little.”
And it’s really good in terms of how much was done.

But the final deliverable isn’t at all convincing. Clickbait title.

~~~
darkcha0s
Made by a dude who has next to no experience in the field. Now imagine what a
guy who knows what he's doing can do with a bit more money. That's what the
allusion here is; the fact that a Reporter with 500$ even comes close should
be an alarm signal to you.

~~~
yboris
This is also an impressive DeepFake:

[https://www.youtube.com/watch?v=Wm3squcz7Aw](https://www.youtube.com/watch?v=Wm3squcz7Aw)

Voice actor reads a poem in voices of famous people; his face gets swapped
with the actor's face; the combination of voice and face makes it very
believable.

~~~
trianglem
This one is fantastic. It seems like just the most characteristic traits are
carried over and the result is seem less.

------
spullara
You can do these overnight on high end gaming GPU boxes with
[https://github.com/iperov/DeepFaceLab](https://github.com/iperov/DeepFaceLab)

------
kitd
Can content providers not include a digital signature in the video metadata?
Then you can authenticate the video against the original provider, a bit like
the green padlock next to the URL bar in Chrome.

Even better (though I'm not sure this technology exists yet), some some kind
of "rolling signature", available running along side the video in metadata,
that a media player can use to validate the authenticity of the video at that
point in time. Then you don't need the full video to be able to verify the
source.

------
Mc_Big_G
I couldn't tell it was Data from Star Trek until I read the story.

------
Zenst
Thinking ahead - will Hollywood in 50 years time be actor based or digital
rights based when your film is computer rendered and you pick your own actors
that get digitally placed. Effectively reducing actors to DLC!

Remember, technology to fake voices has already be done.
[https://www.youtube.com/watch?v=I3l4XLZ59iw](https://www.youtube.com/watch?v=I3l4XLZ59iw)

Certainly interesting times ahead.

------
Mathnerd314
There's a newer model that can be used pre-trained and doesn't need to be
retrained for each face pair:
[https://nirkin.com/fsgan/](https://nirkin.com/fsgan/)

But for now deepfakes seem like a 2018 meme that died out.

------
Veoich_py
This work is just at the research stage right now and isn’t available as
consumer software, but it probably won’t be long until similar services go
public.

------
sitkack
Brent Spiner is by far the best actor on TNG. Even Zuck wearing a Cmdr Data
skin is an inferior copy.

------
Myrmornis
That just looks like Mark Zuckerberg to me.

