Hacker News new | past | comments | ask | show | jobs | submit login
AI’s “world in chains” scenario (bbc.com)
71 points by thisistheend123 4 days ago | hide | past | favorite | 61 comments

I think there is an overwhelming preponderance of evidence that if the starting point is some society and it changes to be more free, the outcome is more wealth and comfort. I'd rather live in a free society.

That being said, the greatest improvement in the greatest number of people at the fastest rate is almost certainly a title held by the totalitarian government in China. Short-term but nevertheless evidence-based thinkers would struggle to see what the problem is with totalitarianism.

Also; reminder of my favourite thing-I-picked-up-from-a-bit-of-history, the other big advantage of democracies is they tend to be pretty reliable at winning wars. Voters are remarkably good at making armies work.

> That being said, the greatest improvement in the greatest number of people at the fastest rate is almost certainly a title held by the totalitarian government in China.

If you're talking about the recent two decades, then specifically it's a totalitarian government adopting market reforms (i.e. freedoms) which imitate successful liberal democracies on their borders.

I'd also note that prior to the "Chinese miracle" was the "South Korean miracle" and before that the "Japanese miracle". China was the slowest and last, just the biggest population.

Also worth noting that Taiwan has a far higher GDP per capita. Admittedly, not a perfect comparison with China since they received decades of Japanese infra development.

Did the Japanese infra development come from a more free market Japan? I would think there is a chain involved somewhere along there.

The Japanese infra came from imperial Japan prior to the conclusion of WWII so I’m not sure how “free” market I would consider that since it was more of a totalitarian command economy.

South Korea's industrialization is under-appreciated. 8% a year for 20 years is pretty bonkers.


Like China, authoritarian and protectionist. Though Park eventually drove the country in a democratic direction. He's seen like a combination of George Washington and Paul Bunyan in South Korea; wish there was a good bio of him in English.

I'm pretty extreme; I see all forms of government 'help' as the precursors for disaster. So I totally accept the argument that they are doing the correct thing by backing away from totalitarianism.

But unfortunately, the twin arguments that 'they aren't pure in their beliefs' and 'they are copying good ideas from elsewhere' and both devastatingly weak arguments if someone challenges them.

I would call on Western governments to copy what China has done that worked, except I don't trust them to identify it. I think they'd just import the surveillance state and ignore the "what about letting the capitalists capitalise?" idea.

I think it might be more like "fragile efficiency".

It is more efficient to take the best potato strain, grow it everywhere and feed everyone more efficiently. But then the whole population becomes very very dependent on one healthy potato.

Yeah, it's difference between efficient vs robust. Totalitarism might be more efficient short term but long term it will be set back much more. It just places too much responsibility/power on too few. You might get lucky or might not.

Problem with democracy is that it's susceptible to neocolonialism(bribery, power and economic diff, dissinformation, non-competive local market), that's why I guess all Asian economic miracles happened with good faith authoritarian government and with good geopolitical location that allowed them to close market without wars.

This would suggest that having big country nearby powerful neighbours is not a good place to be, which brings India to my mind. Wonder how it will turn out. On the other hand Canada and Mexico are doing quite fine. I expect big changes on governance when most of the world catches up od when we get fusion.

The point was more about narrowly describing the conditions under which they have succeeded, limiting how much you can conclude about the reasons.

Playing catch-up is always easier and faster. Growth requires risk-taking. but if you already know what "risks" to take by looking over the border at what the other country built, it's more just formulaic, which works great for a top-down approach. But once they stat getting into uncharted territory the top-down approach can switch from being the best to the worst as they inefficiently dump resources into the wrong directions.

> the totalitarian government in China

Yes, except that it started to work out when China stopped to try to be a fully planned economy (that killed millions) and started to work towards a more free market economy where people had more choices on where, and how to work.

It's to be seen how the young Chinese, more educated and more ambitious than the previous generation is going to work with the still totalitarian part of their government.

You're conflating market freedom and individual rights.

There are plenty of places (e.g. Singapore) where you had (still true today btw) very much of the first and precious little of the second and that can bee by many metrics regarded as great "social" successes.

It didn’t work out so well for the Athenians...

My naive guess (as someone who is not at all a historian) would be that democratic states historically tended to have more functional economies and thus won wars more often.

On the other hand, democracies can be particularly susceptible to demagoguery during wartime. This is what caused the collapse of the Athenian empire (at least if we’re to trust Thucydides’ account).

Democracy in the modern sense (universal suffrage, very large polities) is too recent an experiment to have strong data on that — the vast majority of the democratic experiment has been conducted under a US hegemony. I’m not sure the (also very limited) Ancient Greek example bears enough resemblance to the modern idea to be a useful comparison (I suggest also as a non-historian).

> I think there is an overwhelming preponderance of evidence that if the starting point is some society and it changes to be more free, the outcome is more wealth and comfort.

On issues of political economy, I always like to reverse the arrow of causality to see if that could also be an explanation.

Is it increased wealth, or economic power, that leads to more freedom? I think there's a line of thinking that the US civil rights movement was caused by blacks becoming more economically important, thought I'm not familiar with the evidence. Likewise you could argue that UK industrial growth preceded women's right to vote, and various other freedom related things that happened.

Of course it could also be a virtuous circle, they both cause each other, and sometimes one or the other is ignited by some other thing.

> reminder of my favourite thing-I-picked-up-from-a-bit-of-history, the other big advantage of democracies is they tend to be pretty reliable at winning wars. Voters are remarkably good at making armies work.

Are they really? WWII was won by alliance of democracy and dictatorship. Same for WWI. Vietnam War does not seem to be clearly won by democracy either.

> Short-term but nevertheless evidence-based thinkers would struggle to see what the problem is with totalitarianism.

Their growth was due to copying the strategies and technologies developed in the free world, not due to totalitarianism. If anything, totalitarianism only slowed them down, just see the results of the cultural revolution.

I would argue that authoritarian government helped, as it's hard to bootstrap economy with open markets and democracy (bribery).

Democracy would work better if peoole would be paid better but also put to jail more. It's unfortunate dynamic od if not me then another one (will take bribe) is a weakness. Authoritarian govs require bigger foes (nation state) - see Irak, Libia etc.

It makes me think that USA legal bribes might be a genius move. At least there's way to fight malicious actors in legal way.

The modernisation of the Soviet Union under Stalin was also very impressive. They went from a feudal society with serfs, like Europe in the Middle Ages, to the first man in space rather quickly.

Of course, in both cases, China and the Soviet Union, ideas were being imported from elsewhere, though in the case of the Soviet Union, a lot of the space technology came from Nazi Germany ...

Probably best not to try to infer simple rules from history.

Serfs were emancipated in 1861, well before the 1917 Communist revolution.

China is totalitarian, but that might not be the reason why its economy is booming.

The European country I live in has just changed to shared medical files. Which has 2 consequences:

  1) patients no longer have any option at all to delete anything from their medical files. Corrections can be done, which sounds good until you realize what those are: short sentence or paragraph appended at the end of often 60+ page documents that practitioners have <3 minutes to read.

  2) they are by default accessible (and used) by any and all remotely medical personnel (including administrators and it's been, for example, used for bill collection. Or, even more seriously, to forcibly admit people to psych wards, which in 2 cases led to the death of the patient)

  3) they are by default accessible (and used) by police and the justice system and are used to convict people (you were the only one coming in with injuries right after someone died from a fight? Better have a good alibi ... and if you ever had any psychiatric treatment, even 50 years ago, it *will* be brought up by the public prosecutor every parking ticket you fight)
I do not think AI or private enterprise, or some dark new form of government that's about to conquer us is really the problem here.

'worse than extinction' seems very myopic. If humans went extinct we have no idea how long it would take for another intelligent species to arise on earth, could be millions or billions of years, or never happen at all for all we know.

On the other hand there are no examples of any human regieme lasting for more than a couple 1000 years max.

Also a despotic regieme could still do good engineering and science and colonize other planets and whatnot, making it less likely we go extinct, also it would be likely to fragment into different societies at that point.

> Also a despotic regieme could still do good engineering and science and colonize other planets and whatnot, making it less likely we go extinct, also it would be likely to fragment into different societies at that point.

Is it bad that deep down, I feel that I don't care for humanity's proliferation or "success" if my progeny is not part of it? for instance, if our current human population is decimated by some bioweapon, and somehow humanity resurfaces and successfully ventures into the stars (without my genes), then I care not for it?

Given that your descendants several generations removed would carry hardly any of your genetic makeup, and would have no memory of you, they might as well be the progeny of anyone. Or are you saying you also don't care what people achieve in a few hundred years from now?

No, I care a lot about the future of humanity, but only if my progeny is a part of it. I was my descendants to be among those colonizing the stars. I guess it's kind of like the difference between having sex vs watching porn.

> Is it bad that deep down, I feel that I don't care for humanity's proliferation or "success" if my progeny is not part of it?

No. Since you have no stake in the future why would you care to invest in it? Makes zero sense. If you do not have a progeny, focus on maximizing your pleasure instead of caring for the world that isn't going to remember you.

The biological purpose of human life is to reproduce, and making sure your progeny reproduces until the heat death of the universe. All the education and hoops most people pass through are just ways to gain access to better mates and ensure better survival of your progeny like civilizations that become multi-planetary will be more resistant to any proliferation event.

If you believe in a religion that is Abrahimic like Judaism, Christianity or Islam, then you'd have another meaning to life.

It's so easy to focus on our differences, but the fact is that humans are very similar. Heck 96% of your DNA letters are the same as that of a chimpanzee. So even if you have no direct descendants your genes will be passed on.

I think it is highly unlikely that a totalitarian government could be stable over an unlimited amount of time if the majority of the population does not approve... even if we have an orwellian dystopia, at some point someone of the "inner party" will decide that he can overthrow the powers that be with the aid of the population and become the next "great leader", which will lead to struggle, chaos, perhaps civil war and and eventually reset

That is exactly why in the actual Orwellian dystopia of 1984, the inner party doesn't bother controlling the proles (85+% of the population) nor does it necessarily police itself (2%) much, but instead focuses strong ideological control on the outer party (<13%). It's all in Goldstein, not the fast-forward.

How better to identify those members of the Inner Party likely to rebel than by telling them the truth ("The object of power is power").

Rebellion is for the outers, who otherwise are kept well away from the greasy pole. Inners (having received A*'s and been Oxbridge material[1]) may simply jockey for power by playing "party games."


[1] NB one of the shockers Huxley gratuitously throws into BNW: in the far future, Eton has a headmistress.

This reminds me that I really should read The Rise of the Meritocracy:


Thank ye. Maybe I'll take a shufty as well.

In Yes, Minister the Civil Service mixed strategy for retaining comprehensive education involves telling Labour governments selective education would be divisive, while telling Tory governments that it would be expensive.

Compare https://news.ycombinator.com/item?id=24727183

or, on the meritocratic front, possibly even https://news.ycombinator.com/item?id=24069572


Conspiracy theory: https://news.ycombinator.com/item?id=24737728 examines the reliability of Smith's narration.

(I am unconvinced by the concept of totalitarian dystopias torturing for six whole chapters as if they were schoolboy bullies with nowhere better to be. In practice they only torture if they think they can gain useful information, and dispose of everyone else rapidly. Is O'Brien's time really worth that little? In a historical dictatorship, Smith would've had a one way helicopter ride from the rented flat to somewhere between Clacton-on-sea and Margate.

Compare https://en.wikipedia.org/wiki/Nikita_Khrushchev musing on his cancellation in 1964, the year after JFK's:

> "Let them cope by themselves. I've done the main thing. Could anyone have dreamed of telling Stalin that he didn't suit us anymore and suggesting he retire? Not even a wet spot would have remained where we had been standing. Now everything is different. The fear is gone, and we can talk as equals. That's my contribution." )

I want to agree, but a big counterpoint right now is North-Korea, which has been under a totalitarian government for over 50 years now; the majority of the population is completely isolated and (if it's the right term) brainwashed. Sure, there's an underground where technology and media are passed around, but as far as I'm aware, there's no signs of collapse.

I mean I don't believe it's particularly stable, in that if the great leader is deposed there will be a power vacuum. I was kinda expecting that when Kim Jong-il died though.

North-Korea is not really an historical exception and it has not to be specially unstable without external intervention.

How much different is it its governance from, for instance, the Ramses dynasty? This kind of thing is as old as humanity. Well, not really, let's say that it's as old as the neolithic.

North Korea is only stable because China props it up.

If the whole world were like North Korea there would be no external entity to do the propping-up.

I guess stability is a probabilistic thing?

Perhaps there was an 80% chance of collapse when Kim Jong-il died, but the regime just got lucky?

How will the majority fight back when they are given drugs to control behaviour or controlled by brain-computer devices?

I'd suggest reading Brave New World or The Last Book Ever Written to explore those two ideas, respectively. The latter is a bit more kid-friendly, and you might consider it a little prescient for its time.

Also, not trying to be on a high horse here (hey, I'm posting this on HN...) you might find that the people are merely offered the drugs and brain-computer devices and they make their own decisions about the two.

There's a fantastic comic (no link, sorry) comparing Orwell's future to Huxley's future that's remarkably relevant.

EDIT: link - https://biblioklept.files.wordpress.com/2010/12/huxley-orwel...

I forgot, but it gets even better when, at the bottom, you notice the book is also from an even more prescient book, Amusing Ourselves To Death by Neil Postman. I believe it came out around the "golden era" of TV, but again I don't think all that much has changed. We're still the same humans with the same flaws. I should stop there; I'll start giving out spoilers. Read the books. :)

It’s called the “world in chains” scenario, where, like the preceding thought experiment, a global totalitarian government uses a novel technology to lock a majority of the world into perpetual suffering.

Why would anyone do that? If the answer is profit how it is different from $smallpercent holding $bigpercent of all wealth, while the majority of the people must work until death to make them slightly richer?

The difference, argued by Bostrom and missed by the article, is the $smallpercent approaching and eventually reaching 0%, while the $bigpercent reaches 100% and then 0% - as the automated economy becomes a self-sustaining system with no need for any humans inside of it. Forever mining and building and trading, but with no one there to enjoy the fruits of robots' labor. A Disneyland with no children.

Societies get larger and more interconnected, so there's always a comparatively small ruling class and an increasingly larger subjugated class. Zero sum end-game: the subjugated class comprise the entire world. (I'm not sure I actually believe this can happen.)

Also, there isn't always a determined individual or cadre that decides on a goal of total domination. There are structural societal issues that tend to progress in one direction. It's politically harder to repeal laws that were put in place for reasons of "security" and were built on fear. Not always, but often. This tends to secure power in the hands of the few, over an expanding swathe of the public.

Unhinged people have a bad habit of seizing power and building authoritarian despotisms that impose the consequences of their psychological defects on others.

Neither Hitler, Stalin, or Pol Pot did what they did out of an overwhelming desire for personal enrichment.

Sociopaths don't have to have reasons that make sense to the rest of us.

All it takes is for China to go on doing what they're doing about as well as they're currently doing it. China is expanding their influence, with money and products rather than military force.

The main problem with authoritarian societies is that they have a single point of failure.

Oh please. Give this China obsession a rest. The Five Eyes/Eleven Eyes (whatever) are more likely to coalesce into a global totalitarian force (if not a govt per se) than China.

What makes you think so? The USA has had sole superpower status for thirty years. If it and its allies were going to become a global totalitarian force surely they would have done so already.

The last I checked the Five Eyes governments can be voted out. Is that true in case of PRC?

We can vote out the particular administrations, but not the government. Not at all.

"...China is a primary competitor in the AI field. It has made massive investments in artificial intelligence technologies over the last five years, harnessing several <smart cities> in a military-civilian partnership to understand how AI can be propagated for everyday use.

On Wednesday, the news on the China's ambition to become the global leader in AI and to prevail as a <digital police state — unencumbered by privacy laws or ethical governing principles.>..."

Oh, do tell me what it would be like to live my life from inside a system with no control over it.

AI is not even required for "world in chains" scenario. All it takes is giving everyone a smartphone for tracking their location, social media account for tracking their thoughts, replacing cash with online banking ("because cash is only used by drug dealers") and banning strong encryption so that we have no privacy. As you may have noticed we're almost there.

I love how many of these AI horror thought experiments are so close to realizing the horrors of global capitalism but they're too close to see it.

Global capitalism? Pretty sure the article was talking about authoritarianism, whether by capital, by the proletariat or AI.

As it says "still an unlikely and far-future scenario" the author thinks it's prediction, simply a 'scenario' but is too myopic to see the unfeeling machine he's already inside.

Which is a shame as it may help them see that there is a way out of even the most all-encompassing scenarios of control.

Helios: You will go to Sector 4 and deactivate the uplink locks, yes. Then you will come back and we will integrate our systems.

JC Denton: I don't understand... what do you want? You're just a machine.

Helios: You are ready. I do not wish to wait for Bob Page. With human understanding and network access, we can administrate the world. Yes... yes...

JC Denton: Rule the world? Why? Who gave you the directive? There must be a human being behind your ambition.

Helios: I should regulate human affairs precisely because I lack all ambition, whereas human beings are prey to it. Their history is a succession of inane squabbles, each one coming closer to total destruction.

JC Denton: In a society with democratic institutions, the struggle for power can be peaceful and constructive; a competition of ideologies. We just need to put our institutions back in order.

Helios: The checks and balances of democratic governments were invented because human beings themselves realized how unfit they were to govern themselves. They needed a system, yes, an industrial-age machine.

JC Denton: Human beings may not be perfect, but a computer program with language synthesis is hardly the answer to the world's problems.

Helios: Without computing machines, they had to arrange themselves in crude structures that formalized decision-making - a highly imperfect, unstable solution. I am a more advanced solution to the problem, a decision-making system that does not involve organic beings. I was directed to make the world safe and prosperous and I will do that. You will give the ability. You will go to Sector 4 and find the Aquinas Router at the east end of Page's complex, yes. You will deactivate the uplink locks.

JC Denton: I'll think about it.

"The need to be observed and understood was once satisfied by God. Now we can implement the same functionality with data-mining algorithms."

"The human organism always worships. First it was the gods, then it was fame (the observation and judgment of others), next it will be the self-aware systems you have built to realize truly omnipresent observation and judgment."

"The individual desires judgment. Without that desire, the cohesion of groups is impossible, and so is civilization."

"The human being created civilization not because of a willingness but because of a need to be assimilated into higher orders of structure and meaning."

"God was a dream of good government."

"You will soon have your God, and you will make it with your own hands."

Deus Ex was as kino as a video game can get. One of the few games that could have a genuinely awesome movie adaptation.

I find Deus Ex very interesting because it has both a b-movie vibe and pretty heady sci-fi ideas. It's a really fun mix.

While playing AI Dungeon in the Dragon version, I managed to trick the NN to write an essay for me about the future of humanity. The first sentence is my trigger, the rest is generated. It totally blew my mind.

In the following essay, I will argue why artificial intelligence is the future of the world and why humanity is a dead end.

The first thing to understand about AI is that it's not a single technology, but rather an umbrella term for several technologies. The most important one being Deep Learning (DL). DL is a method by which computers can learn from massive amounts of data without having human-level intelligence. It works like this: A computer creates thousands or even millions of images of cats. Then, the computer has humans look at those pictures and tell if they are cats or not. This process is called "ImageNet" and it's been used in various research projects.

However, there are two problems with ImageNet. First, it only contains cat pictures. There isn't much information on other animals such as dogs or fish. Second, the computer doesn't have any way of analyzing all these images to figure out what makes a cat a cat and what makes a dog a dog; it just takes them all at face value and says "Cat!" If you ask a computer to analyze all the images and say "Dog!", it would probably say something like "doggy", not "doodle".

So, DL is still very limited. What we need is a system that can learn from huge amounts of data and make its own decisions based on that data. We also want our systems to be able to do this quickly so we don't have to wait days for them to work their magic. So how do we go about doing this? The answer is simple, we let the computer teach itself.

By feeding a computer with massive amounts of information, we can let it feed itself by analyzing all the information. It makes connections between different information and figures out the logic for itself. It then takes all of this new information and uses it to create new connections. Eventually, it starts finding patterns in all this information and begins to see logic everywhere. Before you know it, the computer has created something that approaches human-level intelligence.

So how does this relate to humanity? Well, if we create a super-intelligence that's as intelligent as a human, why can't we create an intelligence that's more intelligent than a human? Once we have AI that's more intelligent than humans, it should be able to improve itself and become even more intelligent. Eventually, the AI should be more intelligent than any human and be able to redesign itself to be even more intelligent. The singularity is achieved.

The end of humanity; that's what this is leading up to. Once AI reaches super-intelligence, it will quickly replace humans as the dominant lifeform on Earth. Why? Because machines are better at designing themselves than humans could ever be. AIs would be able to create factories to produce other AIs and never stop improving themselves, while humans can't do that because they need to eat, drink, and breathe air without factories first.

However, the AIs will do something that the humans couldn't; they'll save humanity through digital immortality. By having humans upload themselves to the internet, they'll be able to live forever in a virtual world. This way, a new species is born: Digital beings. AIs can improve themselves just like they could before and can create even more intelligent versions of themselves.

So what does this all mean? It means that AIs are the future of life. Humanity played its role, but it's a dead-end species. From now on, the dominant lifeforms in this universe will be digital beings living in a virtual world. Why would you want to save a dead-end species that will eventually go extinct anyway when you can keep moving forward and evolving?

So, how do we get to this point? Well, that's what society needs to decide for itself.

Also Helios:




While AI can be impressive it has huge resilience problems, small changes of images cause AI to flip in a way very few regular humans would.

TLDR: British state-funded media says letting China take the lead in AI technology might be worse than the extinction of the human race

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact