
The Real Danger to Civilization Isn't AI. It's Runaway Capitalism - wallflower
https://www.buzzfeed.com/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway
======
Animats
"Runaway capitalism" means one big winner in each sector. This is the result
of the "frictionless economy" \- distance doesn't affect cost much, and
organizations can scale to planetary size. It started with shipping containers
and faxes, and continued to cloud-based services and the attention economy.

So we have Google, Facebook, and Amazon. (China has Baidu, Tencent, and
Alibaba, but they need the Great Firewall to compete.) The US is down to four
big trillion-dollar banks. (Bank #5, the Bank of New York, is about a third
the size of #4.) Telecom is consolidating to AT&T and Comcast.

These companies act like monopolies. They stop competing on price and
influence government and culture instead. While organizations, rather than
AIs, they optimize for size and shareholder value. That's runaway capitalism.

------
Barrin92
Ted Chiang as always is a joy to read.

I hadn't made the 'mirror' connection before and it's a great image, but
pretty much the first time I heard Bostrom, Musk et al talk about this I
really wondered how absent any issues of political or social tension was that
they are fuelling.

Andrew Ng talked about this too. Instead of talking about how technological
change is impacting our social, cultural and political life the tech industry
lets their machinations lose on the world and everybody else can deal with the
fallout.

------
iamcasen
The author touches on a salient point: capitalism has no market for social
good. Social good does not equate to profit, in fact, it is usually an
exorbitant cost.

Furthermore, capitalism has obvious logical limitations that most people who
are in love with it seem to ignore. It runs on resource and energy extraction
from the earth. All economic operations run on energy and materials. There
will come a point where all resources are extracted from the earth, so no new
wealth will be generated in that way.

At that point (likely much sooner), capitalism as we use it today will not
function what so ever.

------
Fenrisulfr
I wish the author dipped into accelerationist philosophy and its adherents.
Most accelerationists nowadays seem to be into Bostrom-esque
singularitarianism and the idea of marching forward with unabashed capitalism
as an 'optimum resource allocation process' to allocate capital into AI
technology efforts. Essentially: Bring the singularity faster, through more
capitalism. They are pretty explicit about it, and I would bet the author
would have a good time taking their ideas head-on instead of avoiding them
entirely. See: Nick Land, Robin Mackay, Benjamin Noys.

------
strangeloops85
This elegantly captures something I think I've always observed, but been
unable to articulate, in all of the superintelligence literature and the AGI /
AI safety line of thinking: There is an implicit set of unarticulated
underlying assumptions, and biases, that form the foundation from which the
rest of the argument is "logically" progressed or extrapolated/ inferred.

~~~
laretluval
Could you elaborate what you think those underlying assumptions are?

~~~
ythn
That artificial super intelligences are not science fiction

------
contextfree
I've seen similar arguments before (for example here ->
[https://thoughtinfection.com/2014/04/19/capitalism-is-a-
pape...](https://thoughtinfection.com/2014/04/19/capitalism-is-a-paperclip-
maximizer/) ) and I think it's a good analogy.

What I don't understand is how this gets presented as an argument _against_ AI
risk. If the "paperclip maximizers" we construct now out of humans, machines,
institutional processes and legal codes already exist, are successful, exhibit
runaway growth, are a danger and lack insight in Chiang's sense, isn't that
actually evidence _in favor of_ the viability of such entities? If they are
already dangerous even with humans in the loop and relying on cumbersome
institutional processes, isn't that all the more reason to think about how
much more dangerous they can get as they become more autonomous and agile?

~~~
jonny_eh
> What I don't understand is how this gets presented as an argument against AI
> risk

I took it that the author wasn't doing that. He was using the "AI Threat"
argument and pointing it back at the tech entrepreneur's current creations:
their companies.

------
justinpombrio
> This scenario sounds absurd to most people, yet there are a surprising
> number of technologists who think it illustrates a real danger. Why? Perhaps
> it’s because they’re already accustomed to entities that operate this way:
> Silicon Valley tech companies.

This argument is not from technologists! The major proponents I'm aware of are
Nick Bostrom, a philosopher, and Eliezer Yudkowski. Musk is simply repeating
their argument, because he found it convincing. It strikes me as extremely
dishonest to psychoanalyze an argument based on the celebrity who repeated it.

~~~
moyix
Stuart Russell is also a proponent of the idea. You'd be hard-pressed to say
he's not technical.

------
maxthegeek1
He totally misunderstands the danger of runaway AI's. Even perfectly
intentioned AI researchers living in his imagined utopia would be at risk of
accidentally creating one. Aligning AI's with human values is a non-trivial
technical problem, and there's no guarantee we'll find a solution before
someone creates a super intelligent AI, regardless of how well we structure
society.

------
eref
Why not both?

~~~
wernercd
Why not neither?

The greatest danger to Civilization, in my not-so-humble opinion is tribalism.
My country is better. My religion. My political philosophy. My color. My sex.
My gender.

And I'll riot and kill if you don't agree.

Capitalism? Socialism? AI? Those won't overcome the stupidity of large
populations of humans.

~~~
laretluval
Tribalism has been around for the entire course of civilization, and has
decreased over time since we got the idea that it's a bad thing. So if
civilization seems like it's in trouble now then it seems unlikely that
tribalism is the cause.

~~~
Id15eba6u
Tribalism is not a bad thing, neither is nationalism, it is only a problem
when it turns sour. Just like everything else. It's horribly limiting to think
in only black and white.

------
swasheck
Extremism of any sort is the actual threat. People as a whole seem repulsed by
moderation and centrism, happily sailing the seas of entropy to the extremes.

------
civilian
Runaway capitalism, oh shit. So that does that mean:

\- Global poverty will continue to decrease

\- We'll get a variety of options to choose from whenever we want to buy
something

\- We'll get to choose how to spend our own money

\- We'll get quick and diligent service

\- If we don't like something, we can choose to stop buying it or working for
them?

The horror!!!!!

~~~
VonGuard
We'll also get highly optimized algorithms that sell us things we don't need
through the use of psychology and the exploitation of human nature.

We'll also get rampant misspending of scientific research funds, which will go
to boner pills and hair replacement techniques, rather than curing malaria or
exploring space.

We'll also have the end-game for all capitalist nations be leadership that is
completely bought and paid for.

We'll also have workers treated as useless and interchangeable cogs, while
public markets sway back and forth, removing jobs, pushing mergers, cutting
pay.

We'll also get things like McDonald's and Coca-Cola, two of the largest
companies in the world, both of which are literally, 100% dedicated to selling
addictive foods that are diabolically unhealthy.

Capitalism ensures people get what they want most. The problem is, people are
fucking horrible, and what they all want most is cheeseburgers, soda, angry
Facebook rants, fake news, and bigger dicks.

Sure, society wants much more than this, but at the individual level, these
are the second-to-second motivations, and capitalism's atavistic psychic
mining project always ends up down on this dopamine level of reaction in all
things.

Humans can be wonderful and do good things. The problem is capitalism is a
race to exploit the worst in human nature, effectively heading off those
better instincts with porn, fast cars, and lap band surgery.

~~~
civilian
And what would your solution be? Because I imagine it would be government
control on what we're allowed to spend money on. You are a totalitarian, and
while it might be a "good" thing in some situations, that kind of power is
quickly and easily corrupted. How long do you think you can last before
regulatory capture happens? How long can you last before you're persecuting
people for thoughtcrimes?

I'll take the "people can spend money on what they want" dystopia before I
take yours. Because the capitalism dystopia to presented can be fixed by
people changing their minds and changing their spending habits. Your dystopia
can't change without deaths.

~~~
dgabriel
The government controls a LOT of what we can buy already. Sex work is highly
regulated, and you can't buy _people_. You also can't buy endangered animals,
ivory, or nuclear weapons.

In terms of people spending money on what they want, people spend lots of
money on opioids and it ruins their lives, or kills them. Should doctors and
pharmaceutical companies not be bound by any regulatory body? Should people
just be allowed to OD on fentanil because they want to spend their money on
it?

~~~
civilian
Lol, you're using the opiod argument because you think it's a slam dunk.

People who use drugs are not harming anyone with force or fraud. And if drugs
were decriminalized, they'd be far cheaper, and drug addicts wouldn't have to
turn to crime to pay for their drugs. The drug war is a massive failure and a
huge civil rights violation.

I would also hope that, when drugs are decriminalized, the drug addict would
choose a safer opiate, like simple morphine or heroin. Yes, they are very
addictive, and they can cause overdoses, but if people could buy their drugs
in a legal market they could actually get good dosage information and it
wouldn't be cut with shit like fentanil.

------
ASalazarMX
Maybe the Corporatocracy is inevitable, only with stupid names instead of
Weyland-Yutani or OmniCorp.

------
Id15eba6u
It's runaway capitalism It's runaway socialism It's runaway what-ever-ism

As always it's hard to take anything from buzzfeed serious when they are so
obviously biased in everything they post. What ever thing can 'runaway' is a
threat to civilization, from pollution to propaganda in the news.

~~~
mercer
While generally I'd agree, this is a pretty respected (guest?) writer.

------
zeep
capitalism is easier to fix than AI gone rogue, probably

------
aaron-lebo
Nicely done Buzzfeed.

 _Billionaires like Bill Gates and Elon Musk assume that a superintelligent AI
will stop at nothing to achieves its goals because that’s the attitude they
adopted._

Kind of reminds one of _Ex Machina_. If amoral profit maximizers design AI,
it's very possibly gonna be an amoral profit maximizer.

It still irrationally bothers me after 30 years of shitty MS products and
Gates literally preventing technological progress so he could make money, he's
one of the good guys. It's like if you donate enough to the church you can
earn penance. The new versions of Gates don't seem much different.

~~~
civilian
But he's not donating to the church for "penance". He's helping save lives
from malaria, and working on other projects to help the developing world! You
can't dismiss that, and he's giving away so much of his money that it can't be
considered a "fee". He's gonna 90% of it away by the time he's done.

~~~
aaron-lebo
Of course and you can't knock him for doing that. It's good, good for him. We
could at least acknowledge that others didn't accumulate that wealth because
their standards for business were higher than his.

~~~
VonGuard
Gates is doing this because in a few short years he went from America's
darling super rich success story, to the monopolistic Satan most of us tech
people remember. He was not prepared for being vilified so quickly and
completely during the anti-trust years.

Same thing as Andrew Carnegie: After the violent putting down of steel worker
strikes, Carnegie was seen as the devil himself. He was so upset that this
would be his legacy, Carnegie went about doing charitable work to change his
image. You probably have a library he built within 50 miles of you, no matter
where you live in the US.

Gates is doing the same thing. He does not want his legacy to be that of a
businessman, but rather, that of a philanthropist.

~~~
arcbyte
The Carnegie library I grew up near is the reason I am where I am today in
life. If I hadn't had access to all the books I spent my childhood and teenage
years reading from that library, I'd be a lesser person.

~~~
rusk
There’s a few Carnegie libraries around Ireland too [0] and as a child I fully
benefitted from my local one. In fact I obtained my first Linux distro from
there all the way back in 1997 when I was supposed to be studying for my high
school finals .. it was a pivotal moment in my career

[0] [http://www.historyireland.com/20th-century-contemporary-
hist...](http://www.historyireland.com/20th-century-contemporary-
history/carnegie-libraries-in-ireland/)

------
hal9000xp
Capitalism is danger for civilization? Writes who? A buzzfeed. A very, very
leftist media site.

How we end up in situation when huge portion (if not majority) of hacker
community turned out to be _die hard socialists_? (nowadays they call
themselves social-democrats)

I remember when I lived in Uzbekistan and I didn't have access to internet, I
got floppy disk with saved copies of Eric Raymond articles. Once I've read
"How To Become A Hacker", it influenced me a lot. I would say it was the most
influential article I've ever read in my life.

I imagined "a hacker", an individualist person, who likes to find non-standard
solutions to problems, who thinks out of box, who don't blindly follow
authority, who likes decentralisation as opposed to centralisation. "A hacker"
is a person who almost by definition right-libertarian.

Latter I found this article of Eric Raymond:

[http://www.catb.org/esr/writings/libgates.html](http://www.catb.org/esr/writings/libgates.html)

Years later, I found articles of Paul Graham. It was amazing to read him. And
he also turned out to be on free-market side:

[http://paulgraham.com/wealth.html](http://paulgraham.com/wealth.html)

I like his quote:

> This is why so many of the best programmers are libertarians.

Now, what I've see is disappointing, I don't like this and it shouldn't be
like this at least on Hacker News. We have more than enough people in the
world who wants more government, more authority, more religion.

~~~
tstactplsignore
Your worldview is common, but doesn't really understand where left-wing, pro-
democratic socialists are coming from. Like you, we are driven by a deep
mistrust of power. We hate absolute authority, like decentralization, and are
especially interested in out of the box thinking.

Right now, the box we find ourselves in is a capitalist society where
capitalist ideology is completely pervasive. The most powerful and dangerous
authorities are corporations and excessively rich individuals. The best hopes
for decentralization lie in genuinely decentralized solutions that give power
back to workers, like worker cooperatives, democratic labor unions, and
political protest.

What we reject is the idea that capitalism has any of the right answers here.
To the left, capitalism is the primary agent of centralization of wealth and
power and authority. I don't expect you to agree, but I hope you better
understand where we are coming from.

~~~
civilian
Socialist governments killed tens of millions of people in the 20th century.
They acquired power far more completely than any company has.

Big companies come and go. They may have power, but it's temporary, and
there's nothing violent when they start losing their footing.

I'd rather have Amazon delivery my groceries than live in Venezuela.

~~~
TheAdamAndChe
> I'd rather have Amazon delivery my groceries than live in Venezuela.

This is an either-or logical fallacy, acting like the only two options are
either pure capitalism or pure socialism.

Many countries in the world manage to have some basics like socialized
healthcare and education, yet they manage to not be hell-holes. Germany
managed to enforce anti-trust legislation in their country to such an extent
that Walmart left the country, yet "tens of millions of people" haven't been
killed in the process.

Any ideology taken to a violent extreme is bad. That includes both communism
and capitalism. Market failures are real, and the increase in socialist
tendencies in the US is because most people recognize that those market
failures need to be corrected.

------
fortythirteen
Not surprising from a publication who has editors that openly support "full
communism"[0].

Buzzfeed "news" is the last place I look for an objective article.

[0] - [http://www.sfgate.com/entertainment/the-
wrap/article/BuzzFee...](http://www.sfgate.com/entertainment/the-
wrap/article/BuzzFeed-Editor-All-I-Want-For-Christmas-Is-12438485.php)

~~~
dcre
It's not an article. It's an essay by a science fiction writer.

~~~
fortythirteen
I question the biases driving which essays they choose to publish.

