
Facebook Decided Which Users Are Interested in Nazis – and Let Ads Target Them - mnm1
https://www.latimes.com/business/technology/la-fi-tn-facebook-nazi-metal-ads-20190221-story.html
======
fixermark
Worth noting: the article's headline is one of those tricky situations where
the summary isn't wrong, but should probably include more information.

FB allows advertisers to target at specific topics, and they've been
blacklisting objectionable categories. But the blacklisting appears to be
manual, so while "nazi" isn't a micro-targeting category, things like Josef
Mengele and a white supremacist punk band are.

Manually keeping up with and out-thinking objectionable content keywords is a
perpetual arms race. If FB wants to win it that way, they'll have to invest
pretty hard in that space if they don't want a story like this every quarter.

~~~
WhompingWindows
Why not just ask people for $1 per year to use FB, and then have non-targeted
ads to cover the rest of the loss in revenue?

~~~
p49k
Because an insanely high percentage of people will refuse to pay $1/year.
Also, because $1 does not come close to covering the cost difference; targeted
ads typically bring in 2x or more revenue per impression and the average
revenue per user per year in the US is around $25.

~~~
llamataboot
I don't know about that. The latest estimates I heard was that you would have
to pay the average American $1k not to use facebook for a year.

That doesn't mean they would or can spend $1000 on FB, but with the
proliferation of subscriptions on app stores that are $10+ a month, I bet FB
could get $10 a month from a large chunk of users and start to move towards
other revenue models.

~~~
llamataboot
This isn't to say they could make as much money with such a model, but if they
were more concerned with the downsides of surveillance ad targeting capitalism
as their revenue stream, they could certainly explore shifting to be a
different sort of company - one with different and maybe even less revenue
streams.

Funny thing that, why do we never hear about a company that wants to stay the
same size or shrink? What is facebook decided they wanted to be half the size
they are now expense-wise, get out of the ads business, and encourage people
to use their platform for less things?

~~~
gattilorenz
> Funny thing that, why do we never hear about a company that wants to stay
> the same size or shrink?

I guess it's a rhetorical question, but if you went public it would scare away
all the investors, right? Who would invest in a company that does not plan to
grow, and might even shrink?

Private companies sometimes do that, tho.

------
1123581321
Can Facebook tell the difference between interest in studying history and
interest in repeating it? Interest in a skinhead band seems to only have one
interpretation, but couldn’t Goebbels and Himler search queries just be to
learn and see who is discussing them? I know I’ve spent a lot of time reading
about WW2. I’m not trying to be obtuse, just ignorant to what degree Facebook
can target different intents behind a query or interest.

~~~
whiddershins
Not all skinhead bands are nazis. It might be seen as a minor point but I am
worried how quickly we make sweeping appraisals based on partial information.

Early skinhead culture and music overlapped heavily with Jamacain rude boy
culture.

~~~
rhcom2
There's even a group who is (was?) dedicated to skinheads fighting racism.
[https://en.wikipedia.org/wiki/Skinheads_Against_Racial_Preju...](https://en.wikipedia.org/wiki/Skinheads_Against_Racial_Prejudice)

------
RickS
What's the expected fix here?

I was unfamiliar with the bands and most of the people in those targeting
lists.

The way I see it, these are the ways you can handle this:

1) Facebook builds this data the hard way. They staff a team of experts on
"undesirables", who research and implement custom blocklists at facebook's
scale. Insanely cash and time intensive, to say nothing of the "who decides
what's undesirable" problem.

2) Spread cost and effort by amassing a central repository of known baddies,
and all the orgs contribute and share access. The government does something
like this with hashes of sex trafficking imagery, so that eng teams can filter
against a blacklist. I think this topic FAR more nuanced and less binary than
"does this picture contain illegal pornography or nah". Who maintains this
list of undesirables? You're at "social credit score" in a hurry.

3) Algos. You let software extrapolate commonalities from known-bad actors –
school shooters, confirmed russian propaganda branches, etc. And let the
machine learn their language and flag accordingly. This is going to be coarse
and stupid in the way ML always is, and local business owners with names like
Heinrich are gonna get their livelihoods smashed accidentally here and there.
Not great.

4) What Simulacra said – you just turn the whole targeting infra off. Facebook
stops making money. This is great, I'd love to see it as regulation, but it's
a big stretch, and very lofty when phrased like this.

5) Some kind of adtech equivalent of finance's KYC (Know Your Customer)
regulation. Tie ad buys to confirmable, prosecutable identities, and rather
than filtering before launch, aggressively follow up after launch. You run an
ad campaign for nazis? Cool, your LLC and its primary stakeholders are
permabanned. Facebook has already tried light versions of this, but it was lip
service.

IMO 4 and 5 are the places to spend effort. I think we nee to start having
conversations that do away with the idea that humans are autonomous and
impervious to influence, and start having the discussion in a new context:
When and how are you allowed to manipulate the minds of citizens at scale, and
what kind of paper trail does it leave?

~~~
pbreit
I don't understand why we don't let such platforms adopt a more laissez faire
approach to such situations. There's an inordinate amount of pressure to curb
free speech these days which seems very un-American.

~~~
munk-a
I was in this camp for a while but the current political reality has shifted
me towards viewing the normalization and acceptance of such hate speech as a
negative to society. When I was growing up nazis and white supremacy were
always framed in a negative light to make it clear they were wrong (this is
mostly relevant in the developing years, less than 14 or so where children
don't have a developed moral compass) now a days the extreme right is being
treated as just another opinion you can have, normal adults recognize the
danger associated with unrestrained nationalism and hate speech but young
people are unexposed or unfamiliar with what it can lead to.

This stuff is dangerous, it can warp the way you view the world, any
information glorifying it should always be accompanied with explanations and
warnings as to the hate it is imbued with... It is un-American to hate another
person because of their origin or their religion, it's also un-American to
squash an open discussion on that topic but... that discussion needs to happen
between mature adults in a setting that makes it clear how unacceptable it is
to lean on racist tropes.

~~~
eponeponepon
Can we be clear on this, please? It's in _human_ to hate another for their
origin or religion - not just un-American. This problem isn't about whether or
not you're a good American, it's about whether or not you're a good _human_.

~~~
dmurray
Hating people from outside your tribe is as human as any behaviour gets. It's
a behaviour we have to work so hard to suppress precisely because it's so
natural.

~~~
eponeponepon
It's a throwback to our feral ancestry. I'd contend that the urge to overcome
our animal nature is the bedrock of what makes us human.

------
Simulacra
Can we just have a social network that doesn't target anyone, for anything,
and just lets us communicate, and build relationships without trying to
manipulate us or violate our privacy?

~~~
spinach
That would at the very least, mean that users would pay a fee to use their
product. As of now, as a free product that is the only way to monetize. The
internet loves it's free, open products but at the same time, ironically,
loves to build products whose founders are billionaires and even raises them
to near hero status. It culminates in the users having a pretty terrible
experience in the way of manipulation and privacy.

~~~
kevin_thibedeau
Commercial radio and broadcast television worked without fine grained metrics
on the users. There's nothing special about the internet that makes invasive
tracking a necessity.

~~~
floatrock
"Worked."

How are they doing now?

It's a cats-outta-the-bag situation. What DOES makes invasive tracking a
necessity is it works so much better than the free models that came before it.
If you're not doing it, your competition is, and they're getting one up on
you.

I don't think we have a free model that monetizes more efficiently than
invasive tracking, so the only option you're left with is a not-free model.
And for consumer-social, "not free" is a small niche (eharmony comes to mind
as making it work, but even they bought all the free dating companies because
the free ones were eating the market...)

------
ahallock
Am I the only one who doesn't mind the targeted ads on FB? Even from a site I
visited recently, it doesn't really bother me. I actually like seeing products
I'm interested in as opposed to generic targeting like you have on TV. I've
discovered some pretty compelling products this way.

~~~
Klathmon
I feel the same, but at the same time I understand that not everyone feels
that way, and that there should be easy ways for those people to not have to
be subject to it.

Sadly any time this kind of thing gets discussed, the nuance gets thrown out
and the discussion becomes either "ban all targeting" or "don't regulate
anything", both of which are horrible ideas IMO.

------
catilac
Just stop using Facebook. Stop putting the onus on Facebook to fix all of
this. We just need to stop using it. There are alternatives out there; start
using them.

These articles are always going to come up. We're going to act surprised for 5
minutes, and continue to feed the machine.

~~~
throwaway_9168
After seeing these comments, I think we have it all completely wrong (that is,
the mental model of those of us who wish FB stopped existing). Don't ask
people to stop using Facebook. Encourage them to use it more.

We should instead call out the people who fund Facebook as sponsoring child
abuse. [1]

And those who work at Facebook as inciting pogroms. [2]

And finally, those who defend Facebook as "dumb fucks" because, well, that's
what they are anyway according to Mark Z.

At the same time, don't ask anyone to stop _using_ Facebook. In fact, they
should use Facebook so much that they bring down Facebook's servers. [3]
Encourage the low ARPU "deadbeats" to keep using Facebook and its network as
much as possible.

Just call out everyone who is giving them money [4].

Lets see how long FB operates after that.

[1] [https://www.npr.org/2019/02/21/696430478/advocates-ask-
ftc-t...](https://www.npr.org/2019/02/21/696430478/advocates-ask-ftc-to-
investigate-facebook-deception-over-kids-in-game-purchases)

[2] [https://newrepublic.com/article/147486/facebook-genocide-
pro...](https://newrepublic.com/article/147486/facebook-genocide-problem)

[3] [https://www.jbaynews.com/whatsapp-crashes-almost-
worldwide-o...](https://www.jbaynews.com/whatsapp-crashes-almost-worldwide-on-
new-years-eve/)

[4] [https://m.signalvnoise.com/become-a-facebook-free-
business/](https://m.signalvnoise.com/become-a-facebook-free-business/)

------
angott
I just don't understand why Facebook doesn't provide a subscription option. I
would be willing to pay $4.99/month or perhaps a bit more in exchange for no
ads and no tracking.

What is so wrong about being able to pay for things?

~~~
RickS
It blocks you from seeing the future. If they get money and no data, they're
getting paid for who they are today with less insight into who they need to be
tomorrow to keep getting paid.

The idea that data is "just sold to businesses" and can be substituted for its
sale-value equivalent in cash is wrong, IMO. Serious insight, product
directions, election swinging power, really big shit – are all emergent
properties of data in aggregate like this that are somewhat unknowable until
you actually get the data and see things unfold.

So what they're actually keeping, by choosing data over cash, is priceless
long-term optionality.

------
duxup
The problem with algorithms and such that show you what you're interested in
is that sometimes they work.

The math and computers don't care, we have to care unless we want to
facilitate just about anything.

~~~
hn_throwaway_99
Couldn't agree more, and the real problem is that Facebook (would be the same
for any other company with a similar ad targeting model) is now in the
position of arbiter of what is acceptable and what is not. Perhaps a punk
white-supremacist band is off limits, but what about joining "maga" with
"Infowars" with "Insane Clown Posse"? You'll likely be targeting many of the
same people anyway.

------
MrZongle2
So let's step away from the "Nazis" part for a second, and rewrite the
headline: "Facebook Decided Which Users Are Interested in ____________ -- and
Let Ads Target Them Directly".

Isn't this how it is _supposed_ to work?

So let's go back to the "Nazis" part of this. Yes, Nazis, Neo-Nazis, white
supremacists and racists of all stripes are repellent to most of us. Are we
agreed on that? Good.

How is Facebook supposed to differentiate between somebody who is an _admirer_
of Nazi icons, and somebody simply doing research on them?

Is somebody supposed to be curating a list of what's "acceptable" for people
to like and not like? Who's in charge of that list? What happens when those
people leave and are replaced by different people?

What happens when something deeply embarrassing to Mark Zuckerberg or Facebook
takes place and starts getting attention? Should _that_ be on the list? And if
somebody places it on the list, what recourse does anybody outside of that
company have to _remove_ it from that list?

Popular, happy speech isn't the speech that needs to be protected. Everybody
nods when this point is raised, then we end up having this same stupid
conversation again in a couple of weeks and people act like _this time_
something is new and different and in _just this case_ perhaps some light,
smiley-faced censorship is necessary... but of course it surely _won 't_ get
out of hand.

Sheesh.

~~~
anigbrowl
_How is Facebook supposed to differentiate between somebody who is an admirer
of Nazi icons, and somebody simply doing research on them?_

The degree, frequency, and enthusiasm with which they share content are good
clues, among other factors.

------
hirundo
There are a lot of people interested in these keywords because they're
horrified by the Nazi phenomena, and so wish to understand it, to help ensure
that it doesn't happen again. I recently made several such searches after
rereading Winds of War and War and Remembrance. Would if be better it I had
found nothing?

George Santayana had a better take: "Those who cannot remember the past are
condemned to repeat it." When someone asks "what could go wrong" it's better
to have an answer.

Let's say they perfect a method of only targeting actual Nazi sympathizers.
Personally I have a few things that I'd like to show those people, without
insult or invective. Like photos of dead family members. It seems to me at
least as important to be able to send a targeted message to Nazis as to cat
fanciers.

~~~
MrZongle2
Amen. Sunlight is the best disinfectant.

Yet we are repeatedly bombarded by stories of this nature and voices telling
us that well in _this_ case, maybe we shouldn't have any sunlight because the
subject is just too horrible to behold.

A censored public is an ignorant public.

------
cmurf
Google (youtube) is having a similar problem with pedophiles taking advantage
of comment keywording to make it possible for people to find things that
ostensibly should be banned. [https://techcrunch.com/2019/02/18/youtube-under-
fire-for-rec...](https://techcrunch.com/2019/02/18/youtube-under-fire-for-
recommending-videos-of-kids-with-inappropriate-comments/)

------
buboard
This sounds like a non-issue.

> Facebook allowed The Times to target ads to users Facebook has determined
> are interested in Goebbels, the Third Reich’s chief propagandist, Himmler,
> the architect of the Holocaust and leader of the SS, and Mengele, the
> infamous concentration camp doctor who performed human experiments on
> prisoners. Each category included hundreds of thousands of users.

If i make a documentary about goebbels' life i shouldn't be able to advertise
it? How about even selling it? I think this is crossing the line of fake
outrage.

~~~
whoisjuan
This. I think I have searched for Nazi-related stuff a lot in the past when
watching documentaries or reading WWII books. I think this goes without
saying, but I'm NOT a sympathizer of Nazi ideology. I just enjoy learning.

By this rationale, I should be in a hate crime watchlist already.

------
minimaxir
Facebook/YouTube's approach to fixing these issues is reactive instead of
proactive. Would an open-sourced list of unequivocally objectionable topics
work to help companies QA their algos? I'm up for building one.

~~~
chillacy
I imagine if your list is missing anything at all (and it will) then you’ll
get slammed in the press for missing it. Skinhead bands are by no means
obvious to me, but even if they would have been in your list, you’ll miss
something inevitably.

------
xkcd-sucks
"Deradicalization" is a form of "Advertising" which doesn't seem to cause so
much mainstream consternation

------
sickcodebruh
I encounter and consider this when creating ads for my black metal band.
Avoiding supporters of nazi metal is such a common concern that I didn't even
think of it as odd, just business as usual. I actually appreciate that I'm
able to explicitly exclude audiences, as it limits the likelihood that I'll
deal with a deluge of hate mail.

Incidentally, Facebook seems to have already reacted to this article by
removing "national socialist black metal" from its interest targeting options.

~~~
eponeponepon
NSBM's a difficult problem. For one thing, where do you draw the line? For
every band that are out-and-proud unequivocal neo-nazis preaching race-war,
there's another who insist they're apolitical as musicians, another who just
want to "explore themes of our national history" (yeah, pull the other one...)
and probably half a dozen who just flirt with the imagery because it sells
records.

For another thing, this stuff is _interesting_ \- there's no two ways around
that. Musically, it's almost entirely straight-up bad (when Varg Vikernes is
the best a movement has to offer, you know there's a quality problem), but the
cultural mechanisms that made it and the social history that feeds it is,
speaking with cold clinicism, really very interesting.

~~~
craigsmansion
> Musically, it's almost entirely straight-up bad (when Varg Vikernes is the
> best a movement has to offer, you know there's a quality problem)

That's a very precarious judgement call, unless you mean the severely limited
production value, which has become a hallmark of black metal by itself.

~~~
eponeponepon
No, I mean half of them flat-out don't know how to handle their instruments
properly, and even the ones who do are still using them to make art that is
lazy, adolescent and derivative.

The production-value stuff I totally understand and wholly dig, and that's not
why I lack respect for their music.

~~~
craigsmansion
> half of them flat-out don't know how to handle their instruments properly

I'd venture to say that's in the eye of the beholder. A highly skilled, say,
progressive rock guitarist could reasonably claim all of them don't know how
to handle their instruments.

> lazy, adolescent and derivative.

I wouldn't discount any argument that would claim this is true for _all_
metal. In a way, that's part of its appeal.

~~~
eponeponepon

        > In a way, that's part 
        > of its appeal. 
    

You're not wrong - and this is no small part of why it's a tough problem.

But I think those three are a bit of a "pick any two" situation.

------
redm
Nazi imagery and information is prevalent still, such as History Channel's
many Nazi shows, or Nazi mega weapons on PBS, or Science/Discovery which also
sell many ads on Nazi-related shows. They cover the atrocities but also the
many successes of the regime. I'm not surprised this programmatically
translates into Facebook as well, i.e. Nazi information is not equivalent to
skinheads. It's a tough problem to solve programatically.

------
some1else
How come ads are allowed to evade public scrutiny when they're shown online?

------
RickJWagner
It's amazing to me how attitudes have shifted so critically.

In the 70s, 'Hogans Heros' was a hit tv show. It portrayed the Nazis as
bumbling idiots, but still it was a topic that was featured prominently in the
show. I bet today people would be fearful to say they watched such a show.

The same thing is true of the 'Dukes of Hazzard'. Imagine it: A tv show where
two wild young men drove a car that had a Confederate flag on the hood, and
the horn played 'Dixie'. (Even though the show famously portrayed African
Americans only in a positive light.) Today, people are ashamed to admit they
watched the show, had t-shirts, etc. Yet the show was wildly popular back
then. (And race relations seemed like they were better at the time, TBH.)

It's good to make progress in calling out evil, but things feel a little odd
in this area.

------
randyrand
Lots of historians are interested in Nazis. As are many people that follow
politics. Who cares?

------
fixermark
This story immediately becomes more interesting if an advertiser figures out a
way to exfiltrate Facebook's graph of who it thnks Nazis are.

... but in general, the ad networks are architected to make that kind of
exfiltration as difficult as possible, since it violates the privacy
constraints users assume.

------
fixermark
The libertarian answer to this would seem to then be "Excellent; now anti-
fascist organizations have an optimized channel to get their message to those
people."

~~~
TuringNYC
The libertarian answer is "to let the market decide" and walk with their
pocketbook / talk with their wallet. That is all fine in theory but the theory
doesn't address what happens when there is only a single dominant player. For
example, if I "walk away with my wallet" on my cable tv provider, i'm done --
there is only one provider.

~~~
krapp
>That is all fine in theory but the theory doesn't address what happens when
there is only a single dominant player.

The market can decide that only one player is sufficient to its needs.

~~~
heurist
A single player that is strong enough can decide that the market will not have
more than one player.

------
not2b
They'll take anyone's money. I got a sponsored ad (meaning Facebook got paid)
from a notorious antivax organization, these people:
[https://www.skepticalraptor.com/skepticalraptorblog.php/phys...](https://www.skepticalraptor.com/skepticalraptorblog.php/physicians-
for-informed-consent-another-radical-anti-vaccine-group/)

------
wbxrs
...and? Don't they have categories for every political leaning?

~~~
anigbrowl
People generally have a problem with political leanings that are explicitly
predicated on abuse of others.

