
We're building a dystopia just to make people click on ads (2017) [video] - walrus01
https://www.ted.com/talks/zeynep_tufekci_we_re_building_a_dystopia_just_to_make_people_click_on_ads/up-next
======
csdreamer7
I remember reading an article (somewhere) years ago that criticized Silicon
Valley for spending so much money to hire bright people to design a better way
to click on ads. I thought that was a real dystopian future.

Now Facebook is finally dying... I haven't been on it in ~6 months. People are
paying for more and more subscriptions. Nextcloud is becoming a better and
better replacement for Gapps. DuckDuckGo is good enough I can 'google' for 95%
of my stuff on it. Firefox is now competitive with Chrome. I think we are
better off now then we were a few years ago.

~~~
Freak_NL
> Now Facebook is finally dying…

Doubtful. Facebook is not just facebook.com, it's the whole package of FB
Messenger, WhatsApp, Instagram, etc. People may uninstall the mobile app and
limit their Facebook access to their laptops or desktop computers (that's a
popular response among my colleagues), but I don't even know anyone who has
actually deleted their account — aside from the folk who never had one, like
me.

I also get the impression that usage of Facebook and related services differs
greatly from country to country, and from demographic to demographic. This
article¹ from 2014 on WhatsApp usage isn't joking; In countries like mine (the
Netherlands) not using Facebook-owned WhatsApp (like me) is rather
exceptional. If I had kids they would miss out on basically everything
(playdates, birthday parties, changes in schedule for sports/scouts/etc.; not
to mention becoming social recluses because all their peers (or their parents
when they are younger) are on Facebook/Instagram/WhatsApp) if I kept refusing
to use Facebook's offerings.

Facebook is not going anywhere soon (although I wish it was).

1: [https://www.wired.com/2014/02/whatsapp-rules-rest-
world/](https://www.wired.com/2014/02/whatsapp-rules-rest-world/)

~~~
machinehermit
Facebook is "dieing" as much horny guys acting inappropriate towards women has
been "solved".

Basically, not at all unless you live in some strange activist bubble that has
nothing to do with the real world.

All this really did was create a nice buying opportunity yesterday in FB
stock.

~~~
galdosdi
You're probably right, though I'm sad about that. For a lot of people FB use
seems almost like cigarettes. I mean, not that order of magnitude, but same
idea. Doesn't really matter what bad news comes out because it's so habit
forming.

The bad news has to be truly existential to make a real difference.

------
wruza
Late to the party as usual, I think this speech underestimates the will of
companies to do what’s described. Most cynical actions are powered by money
and harm anonymized/faceless groups that “hey, they were suicidal anyway”. The
simple fact is: you cannot dislike, remove or train networks to get what you
want – you eat what is served instead, liked or disliked. Your opinion is not
an argument of targeting function, only attention is. My youtube feed was full
of crap until I removed all suggestions. (Did you know you have to remove it
3-5 times for it to not show up again?) And still, it presents that toxic
whatnext thing, which my inner self often treats no less than disgusting. I
CANNOT find the content I really like with all the searches and smart
ordering. I only get it from random articles by pure chance.

The only way to take control of it is to make modern tech available to wide
public as in next-next-done, not as a github-link. The internet and all its
power is still here for us from the start, but heck most turn into companyname
zombies because everyone already is and _that’s the easiest way to
communicate_. Instead of spending time on ultra-asynchronous scalable tech,
could we stop and think of small businesses and just people who want to
publish and exchange content in their small volumes?

The Web Suite consists of a simple but overcomplicated browser app, instead of
a full platform to create, publish and manage the content with few design
hints to not turn it into myspace. Until that changes, we all, no matter
#deleted or not, will swim in crap that is pushed by so-called media giants
(media jerks in my view). Removing yourself from everywhere doesn’t help, as
it doesn’t help to be healthy in mental asylum.

------
HenryBemis
The moment when I dropped facebook, was the time they start manipulating my
timeline. One day I realized that they don't show me things in their posting
order, but they were moving them around. I remember using a bookmark ending
with .com/?sk=h_chr to beat them, but I gave up when I realized that this is
cat and mouse game, and they will continue these shady practices.

[1]:
[https://www.theguardian.com/technology/2014/jul/02/facebook-...](https://www.theguardian.com/technology/2014/jul/02/facebook-
apologises-psychological-experiments-on-users)

[2]: [http://time.com/3951337/facebook-chronological-
order/](http://time.com/3951337/facebook-chronological-order/)

~~~
HenryBemis
15m 34s, Zeynep says: "as a public and as citizens" [1]

And HERE is the problem.

Facebook sees us ONLY as Product (our data) that can manipulate/buy/sell - and
treat us accordingly.

Advertisers see us ONLY as Consumers they can sell to - and treat us
accordingly.

Politicians see us ONLY as Votes (not voteRs) they can snatch - and treat us
accordingly.

And we let them.

[1]: [https://youtu.be/iFTWM7HV2UI?t=934](https://youtu.be/iFTWM7HV2UI?t=934)

~~~
passivepinetree
Okay, so what is to be done about it?

~~~
ModernMech
Start with the politicians. Vote out anyone who doesn't support campaign
finance reform. Then we can make our government a little less sociopathic.

~~~
etherealG
Start with your attention. The impact will be even bigger than your vote.
Avoid services altogether that abuse you, even if there’s some benefit you
gain out of it.

------
johnsonjo
This was a great speech she wasn’t trying to say all algorithms are evil or
that machine learning is evil, but despite it having good consequences it can
just as easily bring about a great evil. Not only do we have to be aware of
hidden biases in machine learning that could be doing unethical things, but we
need to be aware of the great power and effects these algorithms have on those
they are being used on and how they can be used to manipulate.

Near the end she states, “Many of these ad finance platforms [e.g. Facebook,
Google] boast that they are free. That means that we are the product that’s
being sold. We need a digital economy where our data and our attention is not
for sale to the highest bidding authoritian or demagogue.”

~~~
FranzFerdiNaN
Shame that CS people seem among the least interested in ethics, as it's just
such a bummer when you have to consider anything else but technical solutions.
It's why the field really needs the humanities.

~~~
uoaei
I'm doing my Masters right now and taking a look at the job market /
discussions about it. I'm stunned to see people talk about Palantir etc. as if
it's just a thing you need to do to get enough money to live comfortably and
not the enabling force that is bringing about next-gen warfare with more
capacity to kill than all previous technology combined. People go into
"defense" because planes flying themselves is cool but that's where their
consideration ends. I really wish more people thought about what their efforts
enable, directly or otherwise. Like it or not, we are all connected, and there
are causal relations between your actions and the harm (or benefit) of others
every day. Try to minimize that harm.

~~~
infimum
I have a good friend who just finished his Master's in CS and considered
taking an internship at Palantir. I was very apalled at first how he could
even consider to work for a company that are obviously the 'bad guys'. He
comes from one of those countries your president considers a sh*thole and
scoring that internship polishes his CV enough so that he can get a stable and
high paying job which will allow him to stay and not have to go back there.
Can't really bame him....

~~~
zAy0LfpBZLC8mAC
Yeah, how could you blame someone for hurting people if he benefits from it? I
don't see it either ...

~~~
infimum
You know, I'm inclined to agree with your sentiment. I just think it might be
more nuanced. At least to me my friend's suffering is more immediate. Whatever
work he would have done there is multiple layers removed from me. I'm not
saying that this is ok, but I feel it's just human. Luckily the story ends
with him getting a nice internship at another company that's less problematic.

~~~
zAy0LfpBZLC8mAC
Well, yes, rationalizing individual behaviour that in aggregate leads to
authoritarianism, totalitarianism, genocide, war, ... is "just human". Just as
building power structures that add sufficient indirection that allows
individuals to use those rationalizations is "just human". And tribalism ...
yes, definitely "just human".

But is any of that really a justification for anything? Yes, things are more
nuanced, some things are worse than others, sure, but that doesn't really make
"but he is my tribe, and he benefited from it" a good justification, does it?

~~~
infimum
I have to agree again. What you point out is correct and I hope to be able to
act accordingly. My personal take-away is still that these kinds of ethical
ponderings are a bit like 'first-world problems'. I have the luxury that I can
accept or deny any job based on my personal moral values. If the alternative
was to have to go to some place where my life is significantly worse, I'd
probably think twice too.

~~~
zAy0LfpBZLC8mAC
Well, yes, it is understandable what motivates the individual, and depending
on what exactly you are weighing against what, it might even be justifiable--
but then, the reason why some (actual) shithole countries are shithole
countries is essentially this sort of attitude, as that is essentially the
rationalization for corruption. I guess the point is that evil institutions do
not usually come about due to evil people, and when they do, they are not
sustained merely by evil people, so, if we don't want to have them,
concentrating on evil people won't do shit, we have to concentrate on the
actual cause, and the actual cause are those perfectly understandable, human
decisions.

------
gt_
If anyone wonders how we got to this point, I think that is understandable.
Here are a few thoughts:

It’s easy to overlook how much _advertising works_. We each have a limited
supply of bandwidth for perceiving the world we live in. The result is a high
demand for undermining that perception and using it to manipulative ends.

Authoritarian dystopias from books and films usually illustrate _force_
because _force_ is more immediately graphic. It also reflects the dismal state
of a dystopian existence. _Manipulation_ would look a lot different. Fear of
_manipulation_ is a fear of knowing someone else is _trying_ to manipulate
you.

We don’t regularly notice evidence of brands and companies _trying_ to
manipulate us, but most of us understand they are incentivized to.

I have worked in advertising for almost a decade and the truth is much worse.
Although individuals who work at companies may be on your side, they get paid
to spend their days doin _what the company tells them to_. Even managers and
CEOS may be great people but when they go work, they are an employee. They
have a duty to the company. And, the more efficient and skilled we get at
manipulating members of society into behaviors which direct money our way, the
employees will remain _working for the company_.

Are the employees immoral? Not from the perspective of the family they
support. And whose opinion is more important? The family’s. It’s pretty simple
that we cannot expect individual decisions to compete with those of the
profit-motivated companies they work for.

~~~
TeMPOraL
> _Are the employees immoral? Not from the perspective of the family they
> support. And whose opinion is more important? The family’s. It’s pretty
> simple that we cannot expect individual decisions to compete with those of
> the profit-motivated companies they work for._

We could expect more on a social level, though.

I still don't fully understand how did we get to the point when marketing
became a _respectable occupation_. Something young people dream about being
part of, when they learn to be a graphic designer, or a programmer, or a media
person. I guess it's the consequence of how much money the industry makes.

But looking at it from the top, we're literally celebrating the profession
that's meant to exploit and hurt other people. If I came to you, as a friend,
and did to you what half of the adtech companies do to people at large, I'd
get - rightfully - punched in the face. And yet we tolerate that happening at
scale.

~~~
dennisgorelik
Marketing has an important function that improves people's lives: it helps to
find products and services that match people's needs.

~~~
aninhumer
Do you think this function is impossible to achieve without marketing?

Do you think marketing is the most effective way of achieving it?

And do you think the trillion dollar marketing industry that exists is really
creating trillions of dollars of value for consumers?

~~~
dennisgorelik
1) It is possible to find products and services without marketing.

2) Whether marketing is the most effective way -- depends on the
product/service and on the audience.

3) Yes, generally trillion dollar industry - really creates over trillion
dollars value for consumers.

If it was not the case - consumers would not use it.

~~~
aninhumer
>If it was not the case - consumers would not use it.

Consumers don't use it, producers do, and consumers have no choice about that.

What value does the consumer get out of it that justifies the producer
spending billions on advertising?

------
cameldrv
The average American has maybe 3-4 hours of discretionary time per day. They
choose to spend an hour of it on Facebook. The newsfeed algorithm could be
optimized to do anything -- Facebook's own experiments show that they can make
people happy or sad. They choose to maximize their own revenue. There is
immense potential to improve the lives of their users, and they choose to sell
an unimaginable amount of human potential for what amounts to 30 cents per
hour.

------
bwang29
Just imagine one day, some algorithms completely take control of/help us
decide what we eat, who we date, where to travel, who we talk to or make
friends with, what job offer to take, what subjects to study in school, what
startup ideas to invent.

Then imagine that 100 year later, the original developers of these intricate
algorithms retired, passed away, complex system got maintained by automated
tools and programs, and the outcome of these algorithms become more and more
obscure.

Eventually, the complex machines and A.I. algorithms will be looked as
mysteries to our grand grand grand children and they will study them as we're
studying nature today..

~~~
clumsysmurf
> what job offer to take

... or be offered.

Part of Google Cloud Job Discovery is letting seekers find jobs ... the other
is getting the "jobs in front of the right candidates." Does this sound
familiar? It seems like another opportunity for an individual seeker to be put
in an algorithmic jail, with opaqueness and little recourse we usually expect
from Google.

~~~
ikeyany
> algorithmic jail

Not sure if that's succinctly prescient or absurdly reductionist. But it would
certainly describe the age we have entered.

~~~
ryandrake
Literal algorithmic jail is coming, too, as in: prison sentences handed out
with humans completely out of the loop. This will so obviously and inevitably
happen it doesn’t even seem worth typing.

------
fragsworth
A friend of mine had this idea that there's a path towards Facebook's AI
learning to actually kill people. If they take a certain approach to machine
learning (which they may or may not), the following could happen:

A machine learning algorithm might notice that when someone commits suicide,
there is an uptick in activity on Facebook that follows the event. It may
determine, then, that a "suicide" event is "good".

If they try to use feeds to drive more long-term site activity, the algorithm
could learn a correlation between certain types of feed posts and suicide. In
other words, if a user's feed starts to look a certain way, they will be more
likely to commit suicide. The algorithm could then see this as "good" because
it is also related to the spike in overall activity, and promote these kinds
of feeds.

It then becomes a feedback loop where the algorithm is triggering people to
commit suicide.

~~~
scrollaway
Yeah. It's even simpler than that. I saw a similar post on HN a few months ago
theorizing that Facebook's machine learning algorithms could very plausibly
pick up that controversy generates activity, and promote controversial topics
more and more to the point of optimizing for divisiveness and extremism.

And most people here are slowly realizing that it's not just a theory... it's
likely already happening. We're on the early years of acknowledging that it's
happening. 5-10 years from now, some article will come out about how Facebook
has been manipulating our world view for decades and people will link back to
these posts and say "Duh, we've known about this for ages".

Hey, to the people who work at Facebook, Twitter, Google et al, it's your
responsibility to make sure this stuff isn't happening. The company you work
for has no path to fix these problems on its own, this stuff needs human
supervision and intervention, and hyperawareness of what kind of damage is
caused by manipulating people en masse.

~~~
sametmax
The humans in regular medias already do that.

You always see more bad news than good on CNN, but I'm pretty sure there are
more good things than bad to report.

~~~
evan_
CNN can’t do it on a personalized basis.

If Facebook’s AI knows that you hate nothing more than anti-vaxxers, it could
choose to show you seemingly-randomly that your great aunt has liked an anti-
vax page.

I’m not sure if that would ultimately help drive engagement but I’m sure it’s
something they’ve studied.

~~~
sametmax
That's a good point. And it avoids general outrage against big entities or
concepts, instead people just fight each other more.

~~~
ryandrake
Which is ultimately the goal of the folks in charge. The 1% win when they can
get 49.5% to think the other 49.5% are ruining everything, and vice versa.

------
mizay7
I really like her commentary and glad she is getting more attention.

I think the issue is the magic of free and the irrationality of humans. Free
creates irrational responses. People will use a lot of a thing if its free
even if its bad rather than dealing with the perceived loss of paying 2 cents
for something better.

So as long as no one figures out how to make micro-transactions cognitively
easier than free stuff then free tech will prevail. And payment will be
harvested via information and attention.

The dystopia we are building is not because we want free stuff, but because we
are too lazy to make the choice about spending 2 cents.

I am trying to change that with oalrus.com but we shall see if i mange to
build a large enough of a list of alpha users to give the network a chance.

~~~
invalidOrTaken
I don't think people are averse to 2-cent payments. I can imagine a scenario
where I regularly spend money on the Internet besides Amazon. I don't know
_how_ much I've spent on Steam, for instance.

What people are guarding against is optimization. I don't mind paying 2 cents.
I _do_ mind paying 2 cents and then becoming marked for upsell and becoming
subject to a full-fledged marketing campaign, or having some recurring 2-cent
payment be changed to $20. Cognitively, I know that I am no match for the
kilonerds out there trying to extract money from my wallet. So the only move
is not to play.

~~~
mizay7
Fear of uncertainty makes sense and something I haven't thought about too much
in this project. Sad that the modern net has created such a 'gotchya'
impression. I would think that this comes from everyone promising free but
actually desperately looking for revenue somewhere.

I feel like this is solvable with smart policies and good institutional
culture. One thought that I had was that a user commits to depositing say $2
to use for micro payments for the month, but doesn't actually get charged
until the end of the month. This delays the pain point of spending money
(thereby reducing it) but also may minimize the sensation of risks.

------
taurath
So funny to see these posts here and getting votes - wasn’t too long ago
(literally months) that practically anything against the ad ecosystem was
impugned. Granted, it may not be the same people speaking or talking, but what
of you, dear lurker? Have you finally changed your mind on the value of these
platforms?

------
agumonkey
The previous system of TV + commercials wasn't perfect. It's often mocked and
criticized for over romanticizing products (from daily health to polluting
cars to cigarettes). But these era seems worse because it's disguising it's
aim behind "real" ads (because they don't coat the product in glamour, because
they use "more data") but in the end it's as bad if not worse as before; just
a web based reincarnation of the same needs to make money.

------
chx
The problem is much deeper: in the roughly 25 years of WWW we have not really
figured out better ways than ads.

In the fourth quarter of 2017, Facebook had 2.2 billion monthly active users,
ad rev was 12.779B thus if active users on average paid somehow a little under
two dollars a month, it'd be the same. I am not saying, by far, that a flat
fee is realistic, I am just showing there's a lot of room for improvement.

~~~
cyphunk
you point out something important. for-pay models don't properly deal with
income disparities. you can be sure that for a portion of current 2.2b users
$2 a month for access would be a significant chuck of their income.

~~~
deckard1
What's the benefit to Facebook for catering to such people? Presumably,
advertisers can only sell cars to people that can afford cars. Or is this
where Cambridge Analytica comes in? Then the equation stops being about
selling _things_ and becomes selling consensus. Maybe that was the plan all
along.

~~~
lmm
The cool "starving artist" types putting up their events/content on Facebook
is what makes it valuable to the rich people they want to advertise to. So
it's not as simple as just keeping the people who can afford to pay.

------
squarefoot
It's not just about viral videos. People in general is being educated to trade
being with possessing in the pursuit of an immediate short time satisfaction
(which is sad because someone could take away what you have but not what you
are). So it makes perfectly sense to me that driving people into sadness,
paranoia etc. in many ways (news, music, movies etc.) could also be not just
exploited but willingly planned for business reasons. This goes beyond the
obvious "someone could invade the country -> buy more weapons" or "the big one
quake/tsunami/plague etc. is coming soon -> amass food and stuff"; if people
is being driven into thinking their life is miserable or they're in danger,
most of them will seek immediate gratification by buying more of this or that.

------
wanda
*to make bots click ads

------
aurelien
And because United Nation don't get the flag on the point that it goes against
the evolution of Humanity.

------
soundpuppy
This is the only reason why I would support dropping net neutrality. If the
ISPs charged more for content, we could potentially see an ad free future!

~~~
wvenable
Net Neutrality is about taxing content, not charging for it. Right now
everyone (you, your neighbor, Netflix, Facebook, Google, ISP themselves)
already pay to move bits between each other. ISPs want the ability to extort
money from end-nodes of the Internet who they don't currently have any direct
business relationship or physical connection with.

Right now it's all just bits come across their network from peers _but_ if
they can start blocking traffic from specific sites, like Netflix, they can
also make deals to ensure that doesn't happen. Maybe even make those bits move
a little bit faster.

In no way does this lead to an ad free future. It would probably lead to more
ads as content providers would have additional costs to recoup that they don't
currently have.

