
Fun with Snapchat's Gender Swapping Filter - homarp
https://blog.evjang.com/2019/05/fun-with-snapchats-gender-swapping.html
======
Raphmedia
> I have a friend who does drag. It's a lot of work! I'm excited for
> technology like this, because it will make it easier for makeup & drag
> artists to experiment with new ideas and identities cheaply and quickly.

Similarly, I'm transgender and use those applications to come up with new
ideas on how to make little changes to my appearance to feminize my features.
Sometimes coming up with makeup ideas is hard since I lack in experience and
equipment. I don't want to purchase a bunch products I'll end up never using.
Technologies like this allows me to quickly test makeup ideas. For example, I
have green eyes so they automatically add purple eye-shadow to make my eyes
look brighter. I've since incorporated this color in my daily makeup. I've
noticed that they overdraw the lipstick to make the lips look fuller, so I
also do that when I want a more polished look.

However, this is a double edged sword.

Those filters raise expectation on what you might look like after a gender
transition.

I've seen a lot of angry and disappointed people because of that. Those
applications change a lot about your face, sometimes in subtle ways you won't
notice. The curve of the jaw is slightly different, the eyes are a bit farther
apart, your skull is more round, etc. Changes that are pretty much impossible
even with facial feminization surgeries.

People strive to reach that "perfection" and end up being stuck in limbo,
unable to reach it so unable to free themselves from their gender dysphoria.
It's easy to spiral down from there.

~~~
daenz
>Similarly, I'm transgender and use those applications to come up with new
ideas on how to make little changes to my appearance to feminize my features.

Do you ever feel like these applications are re-enforcing physical gender
stereotypes? Like "you selected female, here is what females should look
like." If gender is a social construct, tech like this is clearly maintaining
that construct, yet nobody seems to have a problem with it.

~~~
derefr
There's nothing inherently wrong with maintaining the construct of gender, if
it has been defanged to do no harm to anybody.

If "gender" just means that you put on a dress and makeup and now you're "a
girl", or you put on a suit and cut your hair and now you're "a dude", then
"gender" isn't really anything to be afraid of, because _anyone_ can put on or
take off a gender any time they like, and it's no longer an inherent property
of a person. It's just a word describing voluntary roles that anyone can
assume at any time—archetypal character profiles that one can choose to "wear"
to indicate that people should identify them with that archetype (like wearing
a mask in an opera; or "disneybounding.")

We're not there yet, mind you, but that's the direction the LGBTQ movement has
been trying to push things for decades now: making it less about "am I
female?" and more about e.g. "am I a dainty princess?" where anyone who wants
to play the role of "a dainty princess" can do so; and anyone who wants to opt
out of that role can do that, too, and nobody in particular is expected to
take that role upon themselves.

But if-and-when we have all that, then there would be nothing wrong with
wanting to conform to the stereotype of "a dainty princess", or to any other
gender-based stereotype; and there wouldn't even be anything wrong with
gatekeeping such roles, insisting that "a dainty princess" has a particular
look and behavior, being picky about who does or doesn't fit the mold of "a
dainty princess", etc. If nobody is already slotted into any gender by
default, then "gender" becomes a matter of finding the gender that perfectly
suits _you_ ; rather than something where you're stuck with a gender whether
it suits you or not, and so have to fight for the mind-share required to
deform your gender's definition to ensure it includes you.

~~~
md224
If you remove the link to sex, then what's the difference between gender and
personality? My understanding is that gender is a social construct that maps
behavior onto biology... if you sever that link, then you're left with
behavior. Why do we need to maintain these categories of masculinity and
femininity? Why are we reifying archetypes instead of moving past them?

~~~
derefr
The thing gender "actually is", when you pare everything else away, is a set
of role scripts. Culture is made of rituals, and these rituals have labelled
slots for people to fit into. The labels are genders (and also ages, social
classes, and familial relationships, but that's less relevant here.)

Think of it this way: a wedding has a "bride" and a "groom." One way to think
of it—the beach-head gay rights have established for us—is that either a man
or a woman can be "a bride"; and either a man or a woman can be "a groom."
Therefore, you could have a wedding with two "brides" who are both men, if you
wanted.

Another way to think of it, though, is that being "a groom" (and all other
such similar roles) is _what it means_ to be "a man." In this model, a wedding
is one of many _rituals_ —a stage-play with a script—and one character in this
stageplay is called "the bride", while another character is "the groom." You
can't have two brides; that's not how the play of "The Wedding" is written.
But anyone can _play_ "the groom." And whoever plays "the groom" in the
wedding, is also known as "a man"—at least for the purposes of the wedding
script. If you happen to always play the set of roles in role-scripts that say
that "a man" plays them, then you _are_ "a man." Your gender is something you
_deduce_ from the set of roles you favor.

The work that goes into being transgender—the hormones, makeup, clothes, all
that? Other than for the few people with true Body Dysmorphic Disorder (who
would get reassignment surgery even if there were no other humans left on
Earth to observe the results), the whole goal of the transitioning process is
to get other people to "cast" you as the role you prefer to be identified with
in these cultural role-scripts. A transman is someone who wants other people
to intuitively place them as "the groom" at the wedding. That's the essential
quality of it.

And, if you don't go for any of the available gender role-scripts that our
culture defines? Then you don't have a (simple, obvious) gender, according to
said culture. You would be genderqueer, nonbinary—whatever the word is today.
Even if you're accidentally a completely central example of masculinity or
femininity—if you aren't _playing the role_ associated with that presentation,
then it ain't your gender.

------
the_watcher
I don't usually pay attention to things like this, but I have a younger sister
who uses Snapchat. She used this filter and the output was essentially a
perfect photo of our brother, but with longer hair. I couldn't help but be
impressed.

~~~
ljcn
What about the brother -> sister mapping?

~~~
davemp
Judging from my family's group chat that also works really well.

------
joshypants
You can follow artists like Mario Klingemann to see people experimenting on
the cutting edge of this stuff:

[https://twitter.com/quasimondo/status/1127906469108756480](https://twitter.com/quasimondo/status/1127906469108756480)

------
wtdata
It's fantastic to see how far this tech came in such a short time.

It's also scary, because soon it will be very dificult to trust anything you
see in video. What now demands a big CGI budget, will be possible to do with a
simple App.

~~~
chipperyman573
I would argue it's a good thing. This was already possible for people with a
lot of money, but nobody knew about it because it was so rarely done. When
anyone can do it, these fake videos will become common and people will realize
you can't trust it just because it's in a video. The only drawback is that
there will be an awkward period where smart people can do it but most people
can't, so nobody realizes what's going on.

------
wjn0
From a machine learning standpoint, I would guess that this is achieved with a
partially supervised approach. The idea would be to embed an entire face
dataset from a number of different poses to a latent space by some method
(could be as simple as PCA, or as complex as an arbitrary neural network). An
arbitrary classification method (using a partial labelling of faces by gender)
yields a hyperplane which optimally separates the labelled points. Then,
something like a simple reflection across this hyperplane yields another point
in the latent space, and the generator yields a corresponding face of the
opposite gender.

Anyone know if this has been implemented for a task like this?

------
sirsuki
Are there other online Machine Learning apps out there for this? Or is Snap-
Chat the only one?

I remember back in the VRML/SecondLife heyday where you could pick an avatar
that was another gender. I still do that. But the emphasis for voice chat in
modern games and MMORPGs make it far more difficult. A low gruff voice does
not bode well coming from a feminine avatar. Catches too many off guard.

~~~
whuffman
My company, Modulate, is building "voice skins" for exactly this purpose:
customizing your voice in chat for online videogames!

It's a really interesting technical challenge. On the one hand, changing your
timbre to another human voice is much more complicated than basic pitch
shifting, so we ended up using deep neural networks for a kind of simultaneous
speech-recognition and speech-synthesis approach (though training this system
to preserve e.g. the emotional complexity of your input speech while still
changing out the timbre convincingly is difficult - we use adversarial
training on the raw audio waveform, which is powerful but also pretty much
unknown territory compared to images).

On the other hand, it's important to run with very low latency on your device
while you're playing, which means that we can't simply "throw the biggest
network we can" at the problem. So we have a tradeoff between model latency,
which is easy to characterize, and audio quality / voice skin plausibility,
which is pretty ambiguous and subjective.

Finally, as this kind of tech improves the potential for misuse becomes an
important problem, so we need to build in protections (like watermarking the
audio) that can help prevent fraud while not compromising the speed of the
algorithm or the quality of the output audio.

~~~
Wowfunhappy
> Finally, as this kind of tech improves the potential for misuse becomes an
> important problem, so we need to build in protections (like watermarking the
> audio) that can help prevent fraud while not compromising the speed of the
> algorithm or the quality of the output audio.

Legitamate question: (absolutely not rhetorical, I don't know the answer!)

What the potential misuse of a gender voice changer? If someone wants to
change their perceived gender in a video game (presumably to avoid
harassment), should that be detectable? If both genders should ideally receive
identical treatment anyway, is there any harm in swapping?

~~~
whuffman
I should definitely expand a bit: the voice skins that we're building give you
a _specific_ person's timbre (or you can do some cool "voice space" vector
manipulations to combine timbres). The cool application here is being able to
sound like a character or celebrity in the game, but the risk of misuse for
having a specific person's vocal cords is much greater than that for just
swapping your gender.

That said, there _are_ some interesting things to be careful around, even for
changing your gender, or age, or other basic variables around your voice.
We're mostly worried about what impact this would have for communities built
around these kinds of commonalities: for example, is it okay for a child to
masquerade as an adult in an adults-only social group? I don't think there's a
clear answer to all of those situations - but until we see more use of
realistic voice skins in the real world, we're playing it safe and building in
these kinds of tools!

------
ve55
I appreciate seeing posts like this as someone who doesn't use any of these
apps themselves. It's intriguing what we're already able to accomplish in this
area with sufficient funding and data, it makes the future seem very
interesting.

------
korethr
For shits and giggles, I installed Snapchat just to play with this, after
seeing some friends having fun with it.

For background, I am guy, but with long hair and a beard. I wouldn't look out
of place in a mosh pit at a heavy metal show. I've also been told I look like
Jesus.

To my amusement, my beard and hair seem to break the female face filter. I
take picture with it, and nothing changes. Conversely, if I use the male face
filter there's quite a difference. My hair gets trimmed, my beard is darkened
and my jawline made more square than it already was. The male filter still
gets confused by my hair and fails to remove all of it much below my jawline;
I can still see my natural hair transitioning back into existence about my
neck and shoulders.

This makes me think that the filters are mainly working on hair length, or
perhaps me already having hair short-circuits the filter and it forgets to
address my cheekbones, jawline, lips, nose and brow.

Scratch all of that. It just worked properly. Compared to the male filter, it
did soften my jawline, blur away my beard, and kept my natural lighter hair
color (though it took away my natural curls). It could have stood to reduced
the size of my nose. Still in the uncanny valley for me, but much closer to
crossing that threshold in than some other filters I've seen in the past.

------
codedokode
What an awesome thing. This is probably a rare example of using technologies
like machine learning and face recognition for good.

Also, it means that we cannot trust photos anymore.

~~~
politician
> What an awesome thing. This is probably a rare example of using technologies
> like machine learning and face recognition for good. Also, it means that we
> cannot trust photos anymore.

You can't trust any social media anymore.

Between Lyrebird, GPT-2, Deep Fakes, Sybil attacks on social media sites, and
abusing Facebook/Google targeted advertising to leak targeting information on
anyone, any sufficiently motivated individual, group or nation state has
enough power to locate audiences and reorient them through tailor-made content
to the attacker's objectives.

Today's cat-picture-viewing joke-seeking social media user faces adversaries
who can deploy coordinated attacks from multiple points-of-origin (via Sybil
clones) using multiple mediums (via audio, video, text generation) against
specific targets without collateral damage to non-targeted audiences (target
discovery via surveillance platforms like Facebook, Google). The information
weapons deployed can change minds, destroy relationships with family members,
undermine friendships, cause confusion, addict, and waste time that could be
spent organizing against the attacker's policies.

In your profile, you state that you're a Russian developer. Consider that it's
known that the Russian government interfered with American elections using
these divide-and-conquer techniques. Consider that we have to question whether
or not your profile is a Sybil account designed to slightly shift the
perspective of HN readers towards the idea that Russians are not undermining
elections by presenting the idea that educated Russians are naive to the
threat. Likewise, consider that this response might be a cointel operation.
One can _hope_ that you're genuine, but that's about it.

Sadly, the untrustworthiness goes far beyond photos.

[https://www.usatoday.com/story/tech/2019/05/13/deepfakes-
why...](https://www.usatoday.com/story/tech/2019/05/13/deepfakes-why-your-
instagram-photos-video-could-be-vulnerable/3344536002/)

------
aiCeivi9
I looks like what you could see in "A Scanner Darkly"
[https://www.imdb.com/title/tt0405296/](https://www.imdb.com/title/tt0405296/)

~~~
Bobbleoxs
You read my mind! I was just going to post this -

So here's a real application than Snap and kiddie jokes (not saying they are
not fun!) - a scramble suit to protect witness/whistle-blower identity

------
stedaniels
Mirror:
[https://web.archive.org/web/20190513170015/https://blog.evja...](https://web.archive.org/web/20190513170015/https://blog.evjang.com/2019/05/fun-
with-snapchats-gender-swapping.html)

~~~
ahofmann
Strange, I can still call this up at the original address at the moment. Just
in case: [http://archive.is/lTgGj](http://archive.is/lTgGj)

------
Insanity
That is just crazy impressive to me. I'm not a fan of those kind of apps
myself but it's pretty cool what they're engineers get to spend time on.

------
ebg13
I'm guessing this filter is only available on certain phones? I installed
Snapchat to play with it, but apparently I don't get access.

~~~
chipperyman573
Did you find the filters? You have to hold down on your face when the selfie
camera is activated. It's on my LG G7, which basically nobody uses, so I doubt
they only targeted certain phones (unless they block generic criteria like
less than 2 GB ram or something).

~~~
jedberg
> Did you find the filters? You have to hold down on your face when the selfie
> camera is activated.

This right here is why I hate Snapchat. I've never seen a more unintuitive UI.

~~~
wishinghand
There's theories out there that Snapchat's confounding UI is to make older
people frustrated with it and avoid it, keeping it cool.

------
giggles_giggles
Physical gender stereotypes? The differences in physical appearance, like the
shape of the bones in your face and the fact some people have penises and
other people have vaginas, caused by activation of the SRY gene (the sex
determinism gene), are not social constructs.

~~~
tobr
The makeup and haircut applied by the filter are social constructs, though.

------
throw03172019
Did the post get deleted? Getting a 404.

~~~
redfern314
Looks like it.

[https://web.archive.org/web/20190513170015/https://blog.evja...](https://web.archive.org/web/20190513170015/https://blog.evjang.com/2019/05/fun-
with-snapchats-gender-swapping.html)

~~~
xrd
I wonder if Snap offered him a job to take it down.

~~~
cycrutchfield
If anything, it is free advertising. Not sure why they would be bothered.

