
The Panopticon Is Already Here - phsource
https://www.theatlantic.com/magazine/archive/2020/09/china-ai-surveillance/614197/
======
eivarv
Here's hoping for a Butlerian Jihad... or, you know, just introducing privacy
legislation that make specific outcomes, not methods, illegal – in the same
way that murder is illegal, regardless of method, tools, etc.

While Tethics and AI are buzzwords du jour, the problem is more general: The
weakening of liberal democratic values.

We need laws and regulation that guarantee privacy more concretely as a
foundational right.

~~~
baybal2
Spot on, America, and West such, wouldn't have a reason to worry in the first
place if it managed to maintain the integrity of the Western bloc after the
cold war.

By letting its foreign policy wonder into grey area by playing overtures with
regimes of all kind without any reservations, it allowed for norms, and
boundaries to be blurred, and resolve of its camp blunted.

What is happening now, is that the whole Western camp have descended into
whataboutism with relation to what rogue regime each bloc member is playing
ball with, without no authority left in the bloc "clean enough" to police it.

What rogue regimes made to the West in the last 20 years, is nothing, but a
gang initiation in reverse.

First they lured the West into playing their dirty games, and when Western
countries thoroughly dirtied their hands, then come and say "your hands are as
dirty as mine, what's the point fighting now?"

~~~
canjobear
What is the evidence that US foreign policy decreased in integrity after the
Cold War? There was plenty of dirty stuff during the Cold War too.

~~~
baybal2
The critical difference was that when the West was engaging with rogue
elements, they stayed rogue elements, not "strategic partners," and trade deal
members.

~~~
lavrov
What do you think happened in El Salvador, Chile, Argentina, Iran, Indonesia,
Nicaragua, Honduras, etc

~~~
baybal2
USA has since disposed of their pawns in Chile, Argentina, and did not oppose
them being taken out in Indonesia, and Salvador.

~~~
blaser-waffle
That's because they stopped being useful in pushing back against global
communism. Chile, for example, ditched Pinochet but kept the liberal US style
economy.

And Argentina is kept in line via WHO loans and regular scourging through
economic means.

------
mathieubordere
It’s a good, chilling read but this sentence right at the end bothered me.

“Until they secure their personal liberty, at some unimaginable cost, free
people everywhere will have to hope against hope that the world’s most
intelligent machines are made elsewhere.”

I don’t agree, with the “right” people in charge, the intelligent machines
pose a risk to humankind everywhere.

~~~
thecreamedcorn
I agree with that intelligent machines pose a risk regardless of who has the
best intentions, but its also true that certain countries (namely china) have
demonstrated that they are more willing and motivated to use it for devious
surveillance purposes.

------
dccoolgai
There is a lot to read here, but it is all important and all possible. I don't
know that it can even be stopped at this point, it's just an index to check on
periodically to confirm "ok, that's how far down the path we are right now".

~~~
mellow2020
"Predictions" are pointless. It's wrong, and just about everything that is
good and decent about human life and the human mind is endangered by it. So
I'm against it, regardless of the "chances" of whatever outcome. I know what's
required for human dignity, I will rather die than betray that, and if
everybody thought that way, it would be over in one week. That they don't is
their problem and something for them to find peace with. I don't worry about
that nor about saving the world, my sole responsibility is being _worthy_ of
having been born into a better one, or into the same one among better peers.
That is within my agency, and only I can let it slip.

~~~
pc86
This seems like pseudo-self help nonsense.

What exactly are you suggesting, other than "be a good person?"

~~~
mellow2020
> What exactly are you suggesting, other than "be a good person?"

I didn't say "be a good person", so what exactly are you taking issue with,
other than what it "seems like" to you? I even stated that I'm not primarily
concerned with outcome or others, this isn't advice to anyone, it's a
statement of fact.

~~~
arcticbull
It doesn't seem like facts, it seems like opinions, but that's ok too. For
instance...

> "I know what's required for human dignity..."

This is a personal value statement, different people have different ideas of
what dignity means. There are likely some universals but there's a huge amount
of variance too.

~~~
mellow2020
> This is a personal value statement

Of course it is. What else could it be?

> different people have different ideas of what dignity means

Then they can come forward and state that theirs is compatible with being
subject to totalitarian rule.

Even without trying, I find so many wise words in support to what I know to be
true both in my heart of hearts and from decades of lived experience.

> _I am well acquainted with all the arguments against freedom of thought and
> speech — the arguments which claim that it cannot exist, and the arguments
> which claim that it ought not to. I answer simply that they don’t convince
> me and that our civilisation over a period of four hundred years has been
> founded on the opposite notice._

and

> _The fallacy is to believe that under a dictatorial government you can be
> free inside._

\-- George Orwell

> _the concept of “freedom and the pursuit of non-material goals” is
> incredibly important, but also incredibly fragile. Not only does it allow us
> to pursue our own lives, it also prevents us from becoming tools of crime.
> It is humanity’s first line of defense, or we should say the last. Actually,
> it’s the sole line of defense. [..] Freedom is not a handout, we need to
> earn it with our efforts. You can lock up my body but you can never imprison
> my will._

\-- Qu Weiguo

> _Suppose that humans happen to be so constructed that they desire the
> opportunity for freely undertaken productive work. Suppose that they want to
> be free from the meddling of technocrats and commissars, bankers and
> tycoons, mad bombers who engage in psychological tests of will with peasants
> defending their homes, behavioral scientists who can 't tell a pigeon from a
> poet, or anyone else who tries to wish freedom and dignity out of existence
> or beat them into oblivion._

\-- Noam Chomsky

In the meantime, the _criticism_ is that "other people think differently",
with not even a hint of what arguments they might have.

And the kicker is, if the arguments are for totalitarian rule, their content
wouldn't even matter, I could simply kill or ignore or slander the people who
bring them forward to win the argument -- while that would violate my own
principles, it would not violate theirs. If being ruled by and deciding facts
by force is okay, my force is as well as any -- but if that path is suggested
in earnest, the hypocrisy instantly becomes visible. And preaching water and
drinking wine isn't _even_ a different set of values, it's nothing.

~~~
chishaku
+10

------
mcshicks
Stuck at home with more time on my hands I've been rereading a lot of old sf.
I just finished the last of John Twelve Hawk's 4th realm trilogy which has a
modern technological Panopticon society as it's central theme. While some of
the technological speculation is strictly fiction, it's surprising how much
the series (last volume published 2009) foreshadows modern surveillance
trends. The trends mentioned in the article could have dropped right into the
story.

[https://en.wikipedia.org/wiki/Fourth_Realm_Trilogy](https://en.wikipedia.org/wiki/Fourth_Realm_Trilogy)

~~~
cheschire
Ive considered making decisions off my RSA token values before. I’ve come to
realize I’m not one for the chaos of harlequin life though.

~~~
tetris11
The Tiktok man rewards you with one more minute of life then

------
erikerikson
Does the technology exist? Can we put the genie back in the bottle? So then
what?

This article and a lot of the literature assumes the same tools are not also
turned upon the managers of society (i.e. that they get to use the times in
secret). Many fear that knowledge of our lives will support interference in
our lives. Perhaps any such interference must be just as observable and
prohibited by law in a free society.

Who watches the watchers? We all could. When there is a conflict or question
we have records to review and public opinion to adjudicate.

Of course we would have to rethink a number of assumable cultural
expectations.

------
ALittleLight
Very sad and horrifying to read about. Worse because I don't know what
constructive things I can do in this context.

AI is a race that absolutely must be won by good actors rather than
totalitarian states.

~~~
canjobear
It looks like AI will be simple and not prohibitively expensive to develop. So
it doesn't matter who "wins the race"\---once it exists, it will be available
to all.

------
ilaksh
My personal belief is that the problems are being oversimplified and that is
very counterproductive.

I'm sure people will misinterpret what I am trying to say. But, it is framed
as a "democracy or tyranny" question. I believe that although the
authoritarianism is quite horrific in some ways, in some respects there are
actually advantages. Which, if you are still reading, is not to suggest in any
way that it is the correct path, but maybe is a hint that our current
"democratic system" may not be quite what it is cracked up to be either.

Again, in no way suggesting we should get closer to a closed system, but I
feel like honest evaluation will see very significant deficiencies with
western governments such as the United States. For example, looking at the
extreme political divide in the country sometimes makes government seem like a
joke.

I personally believe that the best and maybe the only way to move forward
constructively is to be realistic about the flaws in both extremely divergent
views (east and west) and think of a totally new shared philosophy and way for
government to operate..

But most likely that will not happen, and I also personally believe that
another world war may be stimulated by poor technical adjustment to global
accounting collapse (along with the complete failure of cultural and political
integration). I think if this occurs then it will prove that humans are not
fit to control the planet, and hope that we will soon have competent and (one
can hope benign) but much more sophisticated AIs that we can pass the torch of
evolution to.

------
chrisweekly
"A crude version of such a system is already in operation in China’s
northwestern territory of Xinjiang, where more than 1 million Muslim Uighurs
have been imprisoned, the largest internment of an ethnic-religious minority
since the fall of the Third Reich."

There's so much in this article that merits comment / discussion, but this
sentence really jumped out at me. I'd been vaguely aware that this kind of
thing was happening in the PRC, but not at this historic scale....

Western values matter more than ever. Here's hoping for a sea change this
fall.

~~~
HenryKissinger
Pro tip: If you ever find yourself in a semi-professional discussion, I would
avoid referring to democracy, human rights, and individual freedoms as
"Western values." Attaching a specific cultural and anthropological carte to
what I view more as universal values that have yet to take roots universally,
is exclusive and counter-productive. For one, China went through a period of
Western domination starting in the 19th century, and Chinese people haven't
forgotten that. Asking China to follow Western values in this context is about
the worst thing you can say. Just because these concepts originated in the
Western world should not preclude one from putting them in the context of the
larger human enterprise. Do not ask, "Why shall China not walk in the
footsteps of the West in giving its people the right to shape their destiny?"
Ask, "Why shall the Chinese people not be at liberty of shaping their own
destiny?"

~~~
runawaybottle
Oddly enough, the West conceded this battle against the Muslim world to some
degree. We are exhausted from that debate. The Muslim world mostly believes in
their own bullshit, thankfully we bowed out from that fight.

We need to bow out in the same manner with the debate against the East.
There’s a billion people, if they don’t want free uncensored Internet, who are
we to say anything? If they wanted it, they’d have it. They know the ins and
outs of the idiosyncrasies of their society, the same way the west does (how
the west reconciles virtual slave labor to maintain it’s lifestyle of
affordable luxuries (everything from food to clothes to electronics)).

But, there is a new issue. Slavery in America was considered the ‘peculiar’
institution. Surveillance state is something we have to ignore, similar to, I
don’t know, making your women wear varieties of face coverings (Islam), we
have to accept they are acclimated to the mental gymnastics required to
sustain their pride. Whatever. We shouldn’t waste our energy on that, it’s a
losing battle.

But, the ‘peculiar’ institutions are the things we need to probe. A repeat of
pre-extermination Holocaust in China is a no go. I hope the examination keeps
up, and I hope we retroactively apply this new spirit to some of the
trespasses that occurred in Palestine as well.

~~~
Animats
_Oddly enough, the West conceded this battle against the Muslim world to some
degree. We are exhausted from that debate. The Muslim world mostly believes in
their own bullshit, thankfully we bowed out from that fight._

China, though, owns an Islamic region. So they can't run away from it. They
could build a wall, as Israel did, but that just contained the problem. Israel
was never able to get large numbers of Arabs to abandon Islam. So Israel has
an ongoing headache. China is trying to avoid getting stuck in that deadlock.

It's not the surveillance system that's the big problem. It's that they went
all the way to concentration and re-education camps. They're trying to break
the hold of Islam. In too brutal a way.

------
daxfohl
Sounds like we're heading to a new era of feudalism.

~~~
Havoc
If not worse. The current system of capitalism already feels quite feudalism
like in that the "peasants" have very little chance of escaping.

Developments like these will just turn that up to 11

------
WealthVsSurvive
Let's try a thought experiment. Suppose that human intelligence and
consciousness are not ends unto themselves: I think therefore I am is false.
Suppose that human intelligence is secondary to the will to power, to life,
which is merely a chemical that repeats or does not: I am that I am, just an
infinite paperclip factory. Suppose what is meant by consciousness and feeling
is merely a death/not-death projection machine that attempts to conjure
scenarios of fear and joy, dreams and nightmares, through the dimension of
time, via memory stored in gray matter. Now this human AI creates a tool which
can read the nuances of the human AI and bend them to its will.

What occurs after?

------
airstrike
[https://outline.com/ApPUdb](https://outline.com/ApPUdb)

------
api
We are creating a concentration camp for ourselves to get people to click ads.

~~~
Nasrudith
There is a major logical flaw in that rhetoric. What good would displaying ads
to concentration camp prisoners do? Advertising is fundamentally about getting
something that they want from the capabilities of the audience be it votes or
money. Concentration camp victims are rather lacking in them - even if you
include "ability to resist" which in itself is redudant given the extreme
measures of control.

That is like saying "In the dark future of unrestrained capitalism will sell
ads to sex slaves chained in the basement - where they have no money or
freedom." It might tug at heart strings but it makes so little sense it sounds
like an outright parody of such sentiments.

~~~
aarpmcgee
I took "concentration camp" as a metaphor fwiw.

------
drummer
A good read. Again, imagine such a government in charge of universal basic
income distribution and the majority of 'citizens' dependent on that. Not a
bright future.

~~~
dantondwa
Sorry, what does this have to do with UBI?

With such a government, any future wouldn’t be desirable. Imagine such
government in charge of unemployment benefits, health services, the military,
anything really: it’s grim in any case.

Moreover, universal basic income is universal, by definition. If a government
denies it to some particular citizens according to some criteria, it is not
universal anymore. It becomes an income provided to anyone who is in favor of
the government, which is a very different thing. So, if universal basic income
existed, even the Chinese government would have to give it to everyone... or
otherwise give it another name.

~~~
twblalock
> Sorry, what does this have to do with UBI?

UBI gives government a lever to control people by threatening to reduce or
stop payments as a form of punishment.

~~~
rorykoehler
We have the technology to not build that lever into the logistics design of
UBI

~~~
twblalock
This isn't a technology problem. Congress could put whatever restrictions on
UBI it wants to. The technology would be required to implement whatever kind
of UBI system Congress made into law. Congress could also change the law any
time in the future -- and every time party control over congress changes, you
can bet they will push for tweaks to UBI that they promised to their base.

The most likely outcome, which happens with many other kinds of government
programs, would be for Congress to delegate significant authority over UBI to
some regulatory bureaucracy in the executive branch.

Similar to the way the IRS, EPA, and FDA work, there would be some department
of UBI with broad discretion over how the law is interpreted, the ability
create additional regulations and fine people who do not obey them, etc.

You can't solve this problem with code. It's a political problem.

~~~
rorykoehler
If you had crypto that used biometrics to generate unique secret keys and
automatically mined X amount of coins (based on a set of public economic
inputs) to evenly distribute to all participants, and made it part of the
constitution that the distribution mechanism can never be changed you'd be as
close as you can to solving this issue.

Any group trying to take it away once people have it will find out it won't go
well.

