
China's brightest children are being recruited to weaponize AI - hopefulengineer
https://www.scmp.com/news/china/science/article/2172141/chinas-brightest-children-are-being-recruited-develop-ai-killer
======
dang
All: Regardless of your feelings about China, if you do nationalistic flamewar
on HN, or angry denunciation, or population slurring, we're going to ban you.

These threads have been getting noticeably worse lately and the worst sections
of this one are vile.

------
vslira
Killer bots will be treated just like nukes: world powers will justify their
investment in the technology by saying that the other side is also doing it,
and when it becomes clear that there are only two or three powerful countries
that can wield the technology, the "international treaties" will come barring
everybody _else_ from developing it.

I'm not saying that everyone should have nukes, I'm saying the EU, US, China
and Russia should be making those treaties right now instead of later

~~~
3pt14159
The problem is enforcement, not will.

With nukes we can have inspectors and we can lock down uranium. Killbots are
too diffuse and too multi-use. I'm all for arms control / non-proliferation,
but I really don't see how it works for these things. Also, we already have
weaponized AI. We have for decades. We're now talking about a matter of
degree.

I'm far, far more worried about the mass weaponization of civilian systems
(self-driving car, etc) via cyber attack than I am about killbots, but they're
both symptoms of a different problem:

As technological growth continues the space of potential combinations of
methods accelerates. This ever expanding search space results in unpredictable
threats and increasingly asymmetric attack economies.

I've been meaning to write an article on it, but I'm having trouble pulling
together the math because it's so abstract. But that's the general idea.

~~~
Nasrudith
Sounds a bit like the Fermi Paradox and Drake Equations - while the math
technically works out the missing terms make it useless on a practical level.

The possibility space is limited by sanity checks of the actors - technically
a terror group could start putting live rattlesnakes into snake in a can prank
devices on people's doorsteps but it would be logistically difficult and
incredibly stupid.

Beyond that one interesting thing about attackers are how memetic they are
over what is appealing to their personal image as opposed to effectiveness.
Like the symbolism matters more.

Terrorists in the US could cause lasting damage and disruption by
infrastructure attacks like going around and emptying a few AR-15 clips into
unguarded transformers and substations or train track sabotage around oil
freight trains yet they don't at all. Similarly looking at the historic trends
between bombings vs mass shooting vs vehicular murder plots seems to be more
cultural than availability related.

~~~
3pt14159
I wouldn't say useless, but I agree with some aspects of your general thrust.

I agree that symbolism seemingly matters more. I was actually once on a
podcast years ago arguing that terrorism was essentially a solved or non-
existent problem because armed with nothing more than a rented truck terrorist
could plow through crowds. And in the US it was even easier! They could buy
weapons at gun shows! Why did they need to hijack planes?

Ultimately, though, my arguments didn't hold. The terrorists did exactly what
I thought they would do if they really wanted to cause damage.

Just because X could Y (but doesn't) doesn't mean X won't ever Y.

I'm not worried about ISIL style attacks where a couple hundred people die.
Frankly cutting out one can of cola a week would probably save more lives in
the West.

I'm worried about mass attack. We can skate around it all we like, but we're
one bad Windows / Dropbox / Tesla / Linux / Cisco / QNX update from hundreds
of thousands or millions of people dying. The Windows worm that preceded the
electrical grid blackout in the mid 2000s was the inadvertent cause. If an
accidental worm can take out that much we should be much more worried about
the scale of the threat.

We've also never been able to secure the boarder. With drones this gets even
harder because capture is no longer a real deterrent.

Security is an arms race that favours the offense over time. A theatre that
strongly favours the offense requires deterrence and intelligence for
dominance, but the deranged can't be deterred and I don't think intelligence
will work forever. Ultimately the arms race needs to end.

I don't know the solution, but my intuition points to GATACA and UBI. But even
with an optimistic estimate I still consider that a stop-gap.

------
wnkrshm
If I recall correctly, DARPA regularly hosts challenges where university teams
try to solve problems that clearly have military applications.

I know this can be seen as whataboutism but I feel the Western world might
have gotten desensitized to its own activities in the sector, while looking
warily at possible geopolitical competitors.

~~~
starbeast
Whataboutism only really applies to state actors bickering when both are being
hypocritical.

After all, it was created as a US propaganda term for saying that it is
hypocritical for the USSR to propagandise the hypocrisy of the USA, when both
governments are hypocrites.

Outside of that, it is called making comparisons and it is a completely valid
thing to be doing.

~~~
elefanten
It refers to the practice of changing the conversation (often towards
accusation of hypocrisy) rather than answering a claim. Especially when the
comparison is different in kind or scope, or just plain irrelevant.

Making comparisons is valid, but it doesn't make you incapable of responding
to the original arguments.

~~~
starbeast
Often it is used to shut down a line of argument when the comparison is
entirely valid though.

------
Communitivity
The US does this with the DARPA Grand Challenge
([https://en.wikipedia.org/wiki/DARPA_Grand_Challenge](https://en.wikipedia.org/wiki/DARPA_Grand_Challenge)).
I suspect other countries may be doing this through sponsoring/watching teams
from their nation entering international competitions such as the Tech First
Challenge ([http://business-review.eu/tag/first-tech-challenge-russia-
op...](http://business-review.eu/tag/first-tech-challenge-russia-open-2017)).

~~~
beginningguava
You don't see a difference between giving prize money to university teams vs
recruiting children under the age of 18 and then putting them through a 4 year
program where they are expected to enter AI research for military purposes and
the candidates are explicitly vetted for loyalty to the government before
entrance?

~~~
virmundi
The Chinese are more efficient? Take an 18 year old, toss him or her into a
college that does the challenge, get a job at one of the defense contractors
later. The Chinese employee is able to do the same thing about 4 years
earlier.

~~~
elefanten
Your answer seems given in bad faith, but it's worth answering anyway.

There are many other things you learn and things you can go on to do as
[engineering student] at [university]. Doing a [resume-boosting challenge] is
one small part of your universe.

On the other hand, when you are entering a special program for weapons
development and giving press releases about how long you've been studying
about guns... your intentions and the expectations for you are very different.

------
wycs
China is full of extremely bright, hardworking people. It is not a nation of
brain-washed automatons. I think AI alignment is some of the most important
work in the world, and I think spreading and translating AI alignment research
in China, such as Stuart Russell's work, may well be more important than
direct AI-safety research. Because of their huge population and much higher
average quantitative ability among Han Chinese, China has roughly 20 times the
number of people capable of AI research, at least if you trust their PISA
scores.

Should we get human-level AI, there is a good chance it will be made in China.
As I don't think the Chinese are a nation of heartless automatons, I think
spreading alignment ideas in China is not at all a hopeless task.

~~~
jbob2000
I highly doubt your conclusion. You know what happens to smart Chinese people?
They leave China. The comforts of capitalism and the cultural gravity of the
West is too strong to overcome. They still don't even use toilets in much of
the country and you're claiming they're going to invent AI, come on.

Don't get me wrong. They are not a nation of automatons, I am not calling them
dumb or inferior. Just be real. They are making social progress, but they're
still very far behind, and that pushes a lot of smart people out.

~~~
kwizzt
Can you show me some sources for the claim that 'Chinese people do not using
toilets in much of the country'? For all I know, it's the Indians and some
Middle East countries that prefer water over toilet paper.

~~~
jbob2000
Sources? Have you been to China? It's only in some major cities that places
have toilets, otherwise you use a hole in the ground!

~~~
kwizzt
Yes I have been to China, as I was born and raised in China. I've been to
small villages, towns and large cities. I would say that very few places are
actually not using toilets like the some places in north-western/south-western
part of China. However, the majority of the Chinese people live in other parts
along the coast and major water bodies.

I remember when I was very little, so at least 20 years ago, there were no
toilets in the village I was born in. It has not remained that case since
then. I would say your information is pretty stale.

------
HarryHirsch
In 1925, JBS Haldane, the biochemist, wrote _Callinicus, A Defence of Chemical
Warfare_ : [http://jbshaldane.org/books/1925-Callinicus/haldane-
callinic...](http://jbshaldane.org/books/1925-Callinicus/haldane-
callinicus-1up-ocr.pdf)

He was involved in the Brtitish chemical weapons program. There are some
interesting arguments in that little volume, which apply equally well to AI
killerbots.

------
ryanmercer
And this is why I'm glad Sam and Elon and everyone else started OpenAI. We
need more people working towards benevolent AI/AGI to offer
competition/opposition to state-funded AI/AGI that will almost certainly be
used first and foremost for military applications.

~~~
vezycash
I don't think OPENAI will make much difference. Companies & states will simply
leech off OPENAI and copyright, trademark, patent or hide their own
improvements.

~~~
ryanmercer
From my understanding of OpenAI they plan to be open with their research but
not necessarily share code.

Sure, anything they patent gives a foreign government a starting point and the
U.S. government can use secrecy orders to prevent patents from being public or
even flat out locking them down via things like the Invention Secrecy Act but
I think OpenAI, and similar initiatives, will still considerably benefit
mankind when they actually start to develop 'AI' with practical applications
and certainly if they develop AGI.

For example with OpenAI this work they're doing with learning dexterity with
the robotic hands could drastically assist amputees in the very near future by
learning quickly from input from the user to make considerably more functional
artificial limbs.

------
justicezyx
Bejing institute of technology is roughly comaprable to UCSC.

Calling them brightest seems overly glossy.

~~~
lolptdr
University of California at Santa Cruz?

------
api
I'd like to point out that a lot of "defense" research ends up used primarily
or even exclusively for things other than military applications. The military
is used as a feeder for R&D because calling it defense is how you sell useful
spending (and Keynesian stimulus) to conservatives. The US defense budget is
full of only nominally defense R&D. This is probably as true in China as it is
in the USA.

The US did this quite a bit, especially during the height of the Cold War. In
fact you go could so far as to say that America's elite schools operate (or
did at least then) as intellectual feeders for the defense industry.

[https://www.imdb.com/title/tt0089886/](https://www.imdb.com/title/tt0089886/)

(Ninja edited to add the top paragraph...)

------
natch
China has cultivated in its people a very troubling mental complex centered on
national pride and driven by the Century of Humiliation.
[https://en.wikipedia.org/wiki/Century_of_humiliation](https://en.wikipedia.org/wiki/Century_of_humiliation)

To say they are fervent about righting the wrongs and perceived wrongs of the
past is an understatement. It's pretty much a driving force of life with many
people. It's a powerful dynamic. And it's one that's held close to the chest
by Chinese students studying here in the US but in China it's freely talked
about, written about, and taken for granted as the right thing.

On our (the west) side we also have racism and nationalism. But I think our
more diverse (talking about points of view, not race here) and more activist
anti-racism voices are a tempering force that China lacks.

Relating this to weaponization of AI, I think people like Stuart Russell are
naive to think that watching Slaughterbots would give these students pause. On
the contrary, it would excite them, because it's showing exactly the kind of
thing they would be delighted to develop. Not that I know these students, but
I'm aware how China drills bitterness into its people about their past
humiliations, and uses this to develop racist nationalism. Pretty scary.

~~~
jl6
There must be actual Chinese people here on HN. Can any weigh in on the
reality of this? It’s a heavy charge to make against an entire population.

~~~
baobrain
I've been through 3-8th grade education in Chengdu at a top 100 school. The
rest of my education was in the US, so I may be biased in one way or the
other.

From my perspective, I didn't really see it that way. Like any country, the
history curriculum focuses on China's history, focusing primarily on ancient
history (pre-Qing dynasty) for 3 semesters with one semester of modern Chinese
history (Qing to present). While there certainly was a nationalistic tendency
in many topics I would regard the information as historically accurate (as
compared to courses about east Asian history here in the US). And in reality,
my many friends in China with whom I've maintained contact over the years do
not have this sort of mentality and are against censorship and much of the
nationalistic messaging.

Another interesting note that people here have no knowledge of is professors
in China are often times anti-CCP. Many professors take this to the extreme
and in papers will often just give a bad grade if it mentions any good policy
by the CCP.

On a semi-related note, the comments many make here on HN about China make me
extremely uncomfortable. Not because they touch on sensitive topics but
because they smear an entire population of people as liars, thieves, and
mindless followers. Replace the word "China" and "Chinese" with "Italy" and
"Italian" in many comments and they could have been written back in the Ellis
Island days.

~~~
dang
They make me and the other HN moderators uncomfortable too. Unfortunately, I
don't know what to do about it at scale. We will continue banning the worst
and asking the rest not to do this. But it's a matter of mass trends meeting
mass psychology. I feel like Canute commanding the waves to stop, knowing that
it won't work.

When you notice examples of this, we'd appreciate it if you (or anyone) would
email us at hn@ycombinator.com so we can at least post moderating comments in
the thread. We can't come close to seeing all the comments, and sometimes I
notice wretched stuff days later that it pains me to have missed at the time.

------
yters
You think other nations actually buy into peaceful globalism? Everyone is just
looking for an opportunity to be the next global superpower and rule the
world, and only play nice-nice insofar as it furthers this goal. Global trade
certainly does not guarantee global peace. We had global trade before WW1 and
WW2, and it did nothing to stop the most horrific wars ever seen in history.

~~~
dang
We detached this subthread from
[https://news.ycombinator.com/item?id=18450681](https://news.ycombinator.com/item?id=18450681)
and marked it off-topic.

~~~
yters
Can you explain why it is off topic? It appears China is building up its
military might through the use of Western educational system.

