
Bill Joy: Why the Future Doesn't Need Us (2000) - miobrien
https://www.wired.com/2000/04/joy-2/
======
CPLX
For those of you that are youngsters out there seeing this article now, a
little context is perhaps appropriate. I remember when this article came out,
being a Wired subscriber at the time, as one did if they were in their 20's
working at a "dotcom" and thinking this Internet thing was turning out pretty
cool.

This article was viscerally and emotionally _terrifying_ at the time. For
whatever reason it seemed really real. I remember overhearing people talking
about "grey goo" over cocktails at startup events -- the scenario here seemed
real and imminent.

As usual, of course, as with most doomsday predictions, the actual future
reality turned out to be equally terrifying and completely different.

~~~
ataturk
I've read Kurzweil's books. Reflecting on them over time, I think he's kind of
full of it. It's not that the material was un-interesting but rather that the
claims he was making were a bit wild and overly-optimistic. I see the same
kinds of flaws in the reasoning of many people of his generation, where they
compile this list of truisms and try to get you from A->B but really the
fundamental assumptions from A aren't that solid.

Kurzweil wants to live forever. He wants to become a cybernetic being. That
was his undoing in those books--his desire to outlast death forced him to make
wildly optimistic predictions.

The dangers facing us are not AI but simply mundane databases. Everything is
being tracked these days. Compliance is being shoved down our throats. You are
being watched on the Internet and in daily life. "But I have nothing to hide."
Yes, you do. You just don't realize it. You should read up on how people were
treated in East Germany and the Soviet Union because that is your future.
Technology gives zero shits about humans.

~~~
spitfire
I call these the Bhopal 2.0 problems. Bhopal was a disaster where 32tons of
toxic MIC gas was released into the atmosphere and killed 3,787 injuring half
a million people.

My premise for Bhopal 2.0 is we automate processes so much and have no
accounting for waste and damage that we simply focus on the top number. Ask an
ML to optimize for an output and it will, happily, All the while harming
people around the process - automatically. Right now Bhopol 2.0 processes are
all focused on information systems - abstractions, so it's much harder to
point and say "look, this is dangerous". People do very, very, very badly with
abstractions.

In some way this already happened with the recent elections. But that was
purposely aiming the machines at us.

------
otisfunkmeyer
I was 19 when this came out and this essay single-handedly changed my life and
led me to Ray Kurzweil, Minsky, the Unabomber Manifesto, Naomi Klein, and then
ultimately to spirituality through Hawking-trained-physicist-turned-cave-
dwelling-monk Peter Russell's "Waking Up in Time."

The rest of my life quite literally flows from reading this essay over and
over when it came out. Genuinely happy to see it here.

~~~
bmomb
I never heard any of theses terms or texts, can you provide more background? I
tried to search for the one "the Unabomber Manifesto" and it seems to be
linked to a "mathematician/terrorist", and in the archive.org there is an
article about "bad things of industrialization" and some weird characters,
what that means?

Can you explain what i'm missing? I'm young and non-american, maybe i'm
missing context.

~~~
steaknsteak
Yes, the Unabomber was a mathematician-gone-terrorist who had a manifesto of
sorts published decrying industrialization and technological profress. His
academic career was pretty short. There's a Wikipedia page if you want to read
up on it.

~~~
redthrow
There's also a Netflix series called _Manhunt: Unabomber_ which is pretty
good.

------
andyidsinga
One of the things that keeps creeping into my head about sentient machines,
the robot future etc etc is this: If they become sentient, and have
extraordinary capabilities etc etc ...why on earth (no pun intended) would
they stick around here? They certainly wouldn't be limited by the same things
we biological beings are. They could just leave into the great expanse - poof,
I'm outta here dad!

Furthermore, lets say, we become part of the machines (a la Greg Bear's EON /
Way universe). The line of thinking and questioning starts to move towards
resources. These incredible advances in machines, it seems would be
accompanied by incredible advances in resource utilization. The notion of poor
vs rich entities would be completely different, in relative terms, than what
we think of them today.

Our insignificance relative to the machines might not matter in the slightest.
fwiw - worth reading David Brin's uplift books - in those the machines seem to
generally stay the heck away from bio life forms :)

~~~
eksemplar
I’m not worried about sentient machines. I’ve done a lot of work with AI over
the past few years and we are no where close to anything sentient. It’s really
just advanced search mechanism where the biggest danger with the tech it self
is that dangers we misinterpret results and use them for something horrible.

The real danger I see comes from ownership. ML is good at predicting things.
Nano tech along with genetics will make you immortal and have near magical
abilities, and currently only a handful of people will own these technologies.

We are much closer to any cyberpunk dystopian than a robot revolution. In a
way it’s always been like that with capitalism, if we don’t regulate the free
market, then the free market turns people into slaves - and right now we’re
letting the market regulate both society and tech.

------
ThomPete
This was a really big deal when I read it back then. It led me more seriously
into robotics and ai and the discussion of the singularity and jobs. But Bill
Joy's essay was not just waiting for the AI to become super-human he was aware
that way before it would be dangerous to humans.

One of my favorite quotes from the article which really has informed a lot of
my thinking later:

 _" Clarke continued: "Looking into my often cloudy crystal ball, I suspect
that a total defense might indeed be possible in a century or so. But the
technology involved would produce, as a by-product, weapons so terrible that
no one would bother with anything as primitive as ballistic missiles."_

------
AdmiralAsshat
I'd say the author's own work stands in opposition to his conclusion: vi is
over 40 years old at this point and hasn't yet been rendered obsolete.

~~~
cgh
Vi is an example of the Lindy Effect:
[https://en.wikipedia.org/wiki/Lindy_effect](https://en.wikipedia.org/wiki/Lindy_effect)

------
ChuckMcM
I've worked with Bill and even shared an office with him at Sun (although he
was rarely there, and to this day probably just remembers me as that guy that
drank a lot of Diet Dr. Pepper and argued about capabilities for the Java
language :-) and his greatest strength and greatest weakness is that he can
see too far ahead along a path. He would answer emails that I thought I had
completely thought through with a one liner that would illuminate some fault
in my logic. It was annoying and amazing at the same time.

In 1994 he was convinced that the kinds of "compromises" that James Gosling
were putting into Java guaranteed it would be dead on arrival. He wasn't
wrong, those choices would ultimately limit the language (and they have) but
he completely missed the 20 years between then and now where Java would have a
huge impact.

When this editorial came out I had moved on from Sun and was dealing the
leading edge of what would be the dot.com implosion shockwave and now Bill was
telling us it was all pointless, the world would probably die on its own
desire to create cool new things. Well he wasn't saying it was pointless per
se, he was saying we needed to confront the ethics of what we were doing now
instead of in the middle of the crisis. And there is much to like about that,
but recall that Facebook was created in a dorm room, not a laboratory like
Bell Labs or Sun Labs. So there was no oversight, no 'adult supervision' of
people who would ask, as Bill would have, what happens when ...?

So to understand Bill's essay in context I have to ask, "What would he have
said to Mark Zuckerberg?" I don't doubt for a moment that had Mark confided in
him his vision and his plans, that Bill would have foreseen the size and
extent of its impact. Bill is a guy who made more money on Microsoft Stock
than on Sun Stock because he sold the latter and bought the former,
recognizing that at the end of the day Microsoft would have a larger impact.
So what does he do? Does he convince Mark to throw it away? Does he say "You
will be one of the richest people in the world but you'll have created a tool
that nation states will use to undermine democracies around the world?" And
how does Mark respond to that? Probably, "If not me, someone else will figure
this out. Look at myspace.com, I'll take the money and figure out the rest
after it becomes a problem."

The future doesn't need us, and neither does the present. It is the ultimate
hubris of humans from the beginning of time that they are somehow "more
special" than the rest of the machine that is the universe. When you read
books like "The Vital Question"[1] you might be struck that humans are just a
'step in the path' rather than the starting or ending point of that path. You
can imagine self aware machines arguing over the notion that they evolved from
meat.

The power of Bill Joy for me has always been his willingness to say something
outrageous that was the logical extension of a path through the point of
absurdity. And in that moment stretching the pre-conceptions of the people
hearing him such that they were able to think of something new that previously
they would not allow themselves to think it. I've felt it first hand and seen
it in happen in others. The after the meeting discussion that goes "That was
the craziest thing I think I've ever heard, but something that might not be
crazy is if we did this ..."

[1] [https://www.amazon.com/Vital-Question-Evolution-Origins-
Comp...](https://www.amazon.com/Vital-Question-Evolution-Origins-
Complex/dp/0393352978/)

~~~
ArchTypical
> Bill is a guy who made more money on Microsoft Stock than on Sun Stock
> because he sold the latter and bought the former, recognizing that at the
> end of the day Microsoft would have a larger impact

That's an interesting revision of history to fit a narrative.

~~~
jng
Are you implying that Sun had a larger impact than Microsoft? Of course
"larger" is not well-defined if the dimension isn't specified, but in the
financial context it is used, I would say that is a fair assessment.

~~~
ArchTypical
I don't think that's true at all. Let's set aside all the improvements Sun
made to the ()nix ecosystem, which includes dtrace and filesystems and network
protocols, among others.

Most servers were migrated from Sun to Linux (with minor hiccups). Sun
basically set the standard for security and reliability that led to the
successor instead of OS/2 or MacOS or ReactOS or whatever. Microsoft HAD to
change their OS to meet the needs of the users (ANY security, more
interoperability between ()nix and windows, etc).

The statement is either poorly phrased or wholly inaccurate, depending on your
point of view.

~~~
ChuckMcM
As a founder, Bill had a lot of Sun stock. Sun, Microsoft, and Oracle all went
public in 1986 (which was the same year I joined Sun). Sun came to see
Microsoft as the enemy in many ways, not the least of which that they were
growing faster than Sun was and had more 'seats' in terms of people who used
them to do work. Bill sold a chunk of shares, and I don't recall the exact
amount but it was significant and invested them in Microsoft shares. This was
a "big deal" around Sun because of the "message" it sent.

Between 1988 and 1998 Sun stock increased 10x and Microsoft stock had
increased 100x. By converting 15% of his Sun holdings into Microsoft stock he
outperformed his Sun holdings.

------
zackmorris
I read Ray Kurzweil's The Age of Spiritual Machines when it came out and
although it's low on specifics, the abstract concepts are solid. The main one
IMHO being that once a chip can emulate the inputs and outputs of a neuron,
then a human brain could slowly be replaced by silicon and the person wouldn't
even be aware of when he or she became a machine.

We used to say that artificial general intelligence (AGI) was 20 or 30 years
away. Now it's more like 10. The main thing stopping it is the utter lack of
exploration in computer science of highly parallelized/concurrent processing.
So hardware-wise that means we need a DSP with perhaps a million cores with
local memory (like map reduce), this would be like a video card but with
general-purpose cores that can run any traditional language instead of
OpenCL/CUDA etc. The software side means that we need more experience with
languages like MATLAB/Octave, Elixer, Erlang, Go etc so that we can build
other tools like TensorFlow on top of them from first principles.

Given those two foundations, many of the methods in neural nets, genetic
algorithms, etc become one-liners and kids could play around with them the way
we might have written ray casters and fractal explorers when we were young.
They'll quickly discover novel ways of solving problems, combining intelligent
agents in various layers, automatically assigning hyperparameters, basically
all the things we struggle with today. And not long after that, there will be
a repo on GitHub where you can download a brute force AGI and see how fast it
can do your homework.

This seems completely inevitable to me and I would have loved to have made a
career of it but now there's no time. Any idea you can think of is between 2
weeks and 2 years away from being manifested by someone on the web. Which is
why I think we're looking at 5 or 10 advances over the next decade that will
make AGI all but certain (barring a global recession like the dot-bomb
followed by an embrace of fear like the global war on terror that set tech
advances back 10-15 years, but I digress).

~~~
lioeters
"the utter lack of exploration in computer science of highly
parallelized/concurrent processing"

I'm not so familiar with such efforts myself, but that seems like an overly
general statement to make. Surely there are countless examples of exploring
parallel/concurrent programming and processing?

"I would have loved to have made a career of it but now there's no time"

Who knows, as you pointed out, it's becoming easier than ever for non-experts
to start playing with machine learning, with access to inexpensive and
powerful resources.

"..there will be a repo on GitHub where you can download a brute force AGI and
see how fast it can do your homework."

Haha, yes, I can totally see that!

------
mooneater
This is a good thread for:
[http://marshallbrain.com/manna1.htm](http://marshallbrain.com/manna1.htm)

Important reading for these topics.

"the robotic takeover did not start at MIT, NASA, Microsoft or Ford. It
started at a Burger-G restaurant in Cary, NC"

------
qpooqpoo
Kaczynski has written two books since his incarceration: TECHNOLOGICAL SLAVERY
(2010) and ANTI-TECH REVOLUTION: WHY AND HOW (2016)

His essential points, according to the preface to Technological Slavery, are:

1) Technological progress is carrying us to inevitable disaster. … 2) Only the
collapse of modern technological civilization can avert disaster. … 3) The
political left is technological society’s first line of defense against
revolution. … 4) What is needed is a new revolutionary movement, dedicated to
the elimination of technological society. …

------
th0ma5
I would be interested to hear people's opinions on Jini
[https://en.wikipedia.org/wiki/Jini](https://en.wikipedia.org/wiki/Jini)
(Apache River) and its influence.

~~~
fnord77
It was interesting in that Sun came up with Service Location Protocol in RFCs
and this was an early implementation. Jini was flawed though - it didn't scale
well.

Of course now we have things like Bonjour and Avahi.

------
jancsika
> These engineered human beings may be happy in such a society, but they will
> most certainly not be free. They will have been reduced to the status of
> domestic animals.

Well, if the system is advanced enough that its actions become
incomprehensible to humans, then it's a moot point. After all, nobody seems to
go around questioning free will because of earthquakes.

The dangers seem to me to be from systems that aren't anywhere as
sophisticated. Those little pyscho-dogs from Black Mirror are sufficiently
scary. And they are much less likely to have read Sojourner Truth than a
sentient cloud that keeps us as pets.

------
kraig911
I recall reading this too. It's apt to think now how gracious we've been
afforded that software innovation has not kept up with hardware innovation.
While computers have got relatively that much more powerful as he states, they
are still performing maybe a little more than they were in the 90's. One day I
think one of these dystopian realities will happen but I like to think we as
humans are actually just really stupid, and that an army of super smart
machines if they do come into existence will only happen once we actually stop
killing each other.

------
qpooqpoo
"Anti-Tech Revolution: Why and How is Kaczynski’s well-reasoned, cohesive
composition about how revolutionary groups should approach our mercurial
future….. I recommend that you read this compelling perspective on how we can
frame our struggles in a technological society." \-- The Tech, MIT's oldest
and largest newspaper

------
projectramo
I was interviewing for my first real job when this essay came out and they
took the applicants to lunch. (The firm had a big name in consulting.)

The interviewer brought up this article and how scared he was an I told him he
really should not worry about the AI part though maybe we should worry about
the biological part.

He did not appreciate that.

It's weird that we could run a "Where were you when Bill Joy's article came
out."

------
soperj
It's interesting in that he was right about when Moore's law would end. Maybe
off by a couple years, but pretty dead on.

------
jstewartmobile
" _The Machine Stops_ " E.M. Forster (1909)

[http://archive.ncsa.illinois.edu/prajlich/forster.html](http://archive.ncsa.illinois.edu/prajlich/forster.html)

------
wglb
For some perspective on the other side, check out
[http://idlewords.com/talks/superintelligence.htm](http://idlewords.com/talks/superintelligence.htm).

~~~
tntn
I don't view these two perspectives as from different sides of anything. Joy
seems to talk about unintended consequences in a more general sense than just
superintelligence, and Cegłowski also discusses the serious problems of not-
intelligent machine learning.

I'm not terribly concerned about the paperclip maximizer that may someday
exist. I am concerned about the pay-per-click maximizer that already exists.

------
3pt14159
I agree with the general thrust of much of this piece (and was shocked to read
that he essentially classified these types of things as WMDs, just as I did
for mass hacks of autonomous systems) but I will disagree with one main point.

I see no reason that humanity is worth saving over some other, better form of
life. I'm not saying its ethical to mow down innocents to make way for the
next generation, but some humans are pretty fucking awful. If our descendants
are robots with the intelligence of Hawking and are full of kindness and grace
then why should we insist that humanity exist forever? Even if we extinguished
some of our evil impulses, like genocide and rape, what makes us _better_? My
late aunt had Down syndrome but if I could invent a vaccine that stopped it
from ever happening again I would even if I never harm someone with Down
syndrome.

The best counterargument I can come up with is essentially the robots asking
"What else is there to do?"

In some post-singularity paradise the robots can be heading in all directions
at near the speed of light, but once they've figured out this universe's
physics and ended armed conflict then it stands to reason that they'd end up
creating or simulating places that did not have god-like creatures.

I think we might depend on the resource constraints and conflict to give our
lives meaning. Kinda a riff on Eden's "knowledge of good and evil" a
creature's intelligence and wisdom may depend on a universe of consequence.

~~~
pmoriarty
_" I see no reason that humanity is worth saving over some other, better form
of life. I'm not saying its ethical to mow down innocents to make way for the
next generation, but some humans are pretty fucking awful. If our descendants
are robots with the intelligence of Hawking and are full of kindness and grace
then why should we insist that humanity exist forever?"_

If any beings exterminated humanity or through inaction allowed it to go
extinct, I'd seriously question their kindness.

Truly kind beings would bring humanity along and help us to be less
destructive.

~~~
BLKNSLVR
Depends on where their loyalty of kindness lies. Were its kindness
'programmed' to be the preservation of the most life at the expense of the
least, then human extermination would be the lowest of the low-hanging fruit.

Keep breeding stock, but then you get into adjudication of value, and at that
point all I can think of is Dr. Manhattan saying "the world's smartest human
is no more threat to me than the world's smartest termite".

------
ranie93
For anyone stuck behind the paywall (PDF warning):
[https://www.cc.gatech.edu/computing/nano/documents/Joy%20-%2...](https://www.cc.gatech.edu/computing/nano/documents/Joy%20-%20Why%20the%20Future%20Doesn%27t%20Need%20Us.pdf)

