
We're Underestimating the Risk of Human Extinction - ca98am79
http://www.theatlantic.com/technology/archive/2012/03/were-underestimating-the-risk-of-human-extinction/253821/
======
mseebach
_There are so many people that could come into existence in the future if
humanity survives this critical period of time---we might live for billions of
years, our descendants might colonize billions of solar systems, and there
could be billions and billions times more people than exist currently.
Therefore, even a very small reduction in the probability of realizing this
enormous good will tend to outweigh even immense benefits like eliminating
poverty or curing malaria, which would be tremendous under ordinary
standards._

It's really an interesting moral discussion - it's like an extension of the
"sacrifice one person to save five" classical dilemma, but I really can't
agree with his flippant assertion that our moral obligation to the unborn
future billions eclipses that to our obligation to help our contemporaries.
And he's not even talking about preserving the planet for the future
generations, he's merely concerned with them being born in the first place.

Also, his argument has the same short-coming as the "sacrifice" dilemma: We
cannot know for sure that a certain action will have a certain outcome - or
that any action taken was actually the cause of the outcome.

~~~
rudiger
Just like there's a time value of money, there should be a time value of
people. The unborn future billions should be discounted to the present value
so that we can accurately compare them. What's the discount rate?

~~~
jerf
It _has_ to be very high, because our uncertainty about the future is
absolutely enormous. I'd put the discount rate even higher than the monetary
discount rate, which with fairly standard numbers is already effectively 0 in
20 years.

~~~
onemoreact
It takes a ridiculous discount rate to effectively become 0 in 20 years.
5%/year * 20 years = (1-.05)^20 = 35.8%. 10%/year * 20 years = (1-.1)^20 =
12.6%.

Still, 100 years is often considered a reasonable limit on such things as 5% *
100 years = 0.6%.

------
cletus
It's worth bringing up _The Most Important Video You'll Ever Watch_ [1], which
the lecturer characterizes as humanity's biggest problem is our inability to
understand the exponential function. Watch all 8 parts.

I have come to the conclusion that there simply are too many of us. We can
probably sustain our current levels for a century, maybe two, but at some
point scarce resources (and their subsequent cost) will have a devastating
effect.

Basically, we need to correct our population before nature does.

As much as people point to space being our future, I simply (sadly) do not
agree. While there might be plentiful resources in the asteroid belt (and on
other bodies) nothing compares to how cheaply we can pull things out of the
ground here on Earth. Our society is predicated on cheap, plentiful resources
such that it can't survive them being several (or even one?) order of
magnitude more expensive.

As far as interstellar space goes, even if we solve the reaction mass problem
and have perfect (100% efficient) conversion of matter to energy, it will
still be prohibitive to go to even the nearest stars.

Perhaps the simplest explanation of the Fermi Paradox is that potential growth
for a starfaring civilization is geometric (being a sphere ultimately limited
by the speed of light) while growth rates are exponential. And exponential
will ultimately "win".

[1]: <http://www.youtube.com/watch?v=F-QA2rkpBSY>

~~~
astrofinch
That video is a joke for anyone who has more than a superficial understanding
of math. Just because a process has fit an exponential curve does not mean
exponential growth is going to continue forever. Past growth patterns are
fairly weak evidence of future growth patterns. ("Housing prices always go
up!")

That's not to say we shouldn't fear a process that is inherently exponential
in nature, like the reproduction of bacteria or a nuclear reaction going
supercritical. But if the process only appears from the outside to have been
growing exponentially, that's only a weak indicator that it will start
behaving in an insane fashion.

In any case, it seems that as nations become more developed people stop having
kids:

[http://www.overcomingbias.com/2010/11/fertility-the-big-
prob...](http://www.overcomingbias.com/2010/11/fertility-the-big-problem.html)

~~~
wazoox
> _Just because a process has fit an exponential curve does not mean
> exponential growth is going to continue forever._

Did you watch the video? This is precisely what it's talking about. Though the
economy and energy usage have grown up exponentially for the past two
centuries, we cannot hope that it will go on forever. Population growth isn't
a concern _per se_ , resource overconsumption and the impossibility of future
economic growth are.

~~~
crististm
Resource consumption is directly related to population. So if the latter is
not a concern then neither the first.

------
growt
"... we humans will destroy ourselves. ... Most worrying ... human technology"

no shit! breaking news everybody!

I hate to break it to him but there are 10 million science-fiction books out
there dealing with every possible way humans could wipe themselves off the
earth. (and maybe just as much real science)

~~~
angersock
I don't know why you are getting downvoted...

Maybe people don't read as much speculative fiction as they used to (or books
in general, for that matter, but I digress...) but it seems like lately I've
run into more than a few people acting surprised about concepts that, frankly,
have been explored to death in books as recently as half a century ago.

------
rquantz
_1) Almost all civilizations like ours go extinct before reaching
technological maturity.

2) Almost all technologically mature civilizations lose interest in creating
ancestor simulations: computer simulations detailed enough that the simulated
minds within them would be conscious.

3) We're almost certainly living in a computer simulation._

It seems like there's a fourth possibility here, which is that a full
simulation of the universe is not possible. Anyone care to comment on the
computability of something like that?

~~~
Cieplak
Also, it could be that the 'real' civilization is not simulating the entire
universe, but rather just the solar system and simulating the light entering
the solar system from outside the solar system, substantially reducing the
computational complexity of the simulation.

Still, modeling every living being as a sort of cellular automaton and running
the simulation would take a lot of quantum computers. What would be the
computational complexity of modelling the consciousness of a human being?

~~~
pbhjpbhj
> _What would be the computational complexity of modelling the consciousness
> of a human being?_ //

You only need to model one though.

------
astrofinch
If you buy Bostrom's arguments, you could make a donation to his research
group:

<http://www.fhi.ox.ac.uk/donate>

I still can't believe that "trying to save the human race from destruction" is
being allocated such a tiny fraction of the world GDP.

~~~
rwallace
The scare quotes are well placed. The problem with Bostrom et al. is not that
they are spending money. It's not even the fact that they are wasting their
time on imaginary risks. It's that, by trying to bend policy around those
imaginary risks, they are increasing our vulnerability to real-life risks.

~~~
MBlume
Interesting assertion -- explain?

~~~
rwallace
For example, there has been a lot of talk for the last few decades about
bioterrorism. In 2001, it finally happened; a bioterrorist attack occurred. It
killed five people.

In the meantime, natural infectious diseases kill several million people every
year, and far worse is possible. In 1918-1919, a flu epidemic killed two or
three times as many people in a single year as the First World War had killed
in four years. In 2002-2003, a SARS outbreak killed a thousand people before
it was stopped by quarantine measures. We got off _very_ lightly; SARS is at
least as infectious as the 1918 flu, is no more curable, and has nominally
roughly twice the lethality rate. The real lethality difference is higher, for
the 2002-2003 outbreak was small enough that most of the victims could receive
oxygen treatment, which significantly improves survival; in a pandemic, of
course, there would not be enough such treatment to go around. And there have
been historical plagues deadlier by far than SARS. Terrorists have political
goals relative to which the weapon that killed their own communities and
families would be counterproductive. The forces of mutation and natural
selection working on disease organisms have no such reason for restraint.

Now what exactly do the bioterrorism criers think is going to happen if they
start being taken seriously enough to influence policy? It's only too obvious
what's going to happen, because it's the same thing that always does: more
regulation on biotech research, more red tape, more of the sort of field day
for the paranoid and bureaucracy gone mad that we already see in the security
theater at airports. It's hard enough to fly to Disneyland in those
conditions, let alone do cutting-edge research.

And so when - not if - something deadlier and more insidious than SARS does
come along, whatever chance we might have had of being prepared for it may end
up being thrown away.

It's not difficult to understand why our instinctive assessment of threats is
so wildly irrational. In the ancestral environment there was plenty of
disease, to be sure, but almost no medicine, so little selection pressure to
be sensitive to that threat. Violence was the main cause of death _that you
could do something about_. The human brain is wired to assume our fellow man
is the primary threat, regardless of the facts.

But we can sometimes override even hardwired assumptions, once we understand
that we live in a world in which they are no longer true. We had better learn
to override this one, and quickly.

~~~
astrofinch
You might as well say there was a lot of talk about flight in the centuries
before it was actually invented. Historical data can only take you so far.

>Now what exactly do the bioterrorism criers think is going to happen if they
start being taken seriously enough to influence policy? It's only too obvious
what's going to happen, because it's the same thing that always does: more
regulation on biotech research, more red tape, more of the sort of field day
for the paranoid and bureaucracy gone mad that we already see in the security
theater at airports. It's hard enough to fly to Disneyland in those
conditions, let alone do cutting-edge research.

Are you in favor of government regulation for people researching new designs
for nuclear bombs?

It's all about cost benefit analysis. The TSA is a waste of time not only
because hijacking is rare, but because the average hijacking kills fewer than
1000 people. If there was a strong theoretical argument for how a hijacked
plane could permanently end the human race, it would not be a waste of time.

I'm in favor of unregulated biotech research if the expected benefit from
additional disease cures exceeds the expected risk from potential engineered
viruses. Frankly, I'm more concerned about the engineered viruses because it
seems like an engineered virus has a better chance of killing off everyone (as
opposed to not everyone, which is an extremely important distinction in my
view; I'm playing for team humanity) just because engineered things tend to
work better than things that assemble by chance.

~~~
rwallace
> Are you in favor of government regulation for people researching new designs
> for nuclear bombs?

The cases are not particularly similar. It doesn't matter that we have
crippled nuclear weapons R&D with regulation. It will matter a great deal if
we similarly cripple biotech R&D.

> it seems like an engineered virus has a better chance of killing off
> everyone (as opposed to not everyone, which is an extremely important
> distinction in my view; I'm playing for team humanity)

I am not at all as optimistic as you are about that. It is the way of
extinction that what kills the last individual may have nothing to do with the
underlying factors that doomed the species. The last passenger pigeon died of
old age. In my view, far more likely than a single super-disaster that kills
everyone at the same time is a sequence of events where one factor delays
progress enough to make us vulnerable to another that kills enough people or
causes enough disruption to crash civilization, at which point we can't reboot
because all the easily accessible fossil fuel deposits are long gone and
there's no way to jump directly from wood-burning stoves to solar panels,
leaving the survivors to struggle along until ordinary geological processes
finish us off. That's the kind of scenario I'm concerned about.

~~~
astrofinch
It's definitely plausible that you are right and that deregulating biotech
research is the way to minimize existential risk.

I'm not sure I've seen any statement from the Future of Humanity Institute on
this issue, except for this blog post by research associate Robin Hanson which
would appear to support your point of view:

[http://www.overcomingbias.com/2009/12/tiptoe-or-dash-to-
futu...](http://www.overcomingbias.com/2009/12/tiptoe-or-dash-to-future.html)

------
PaulHoule
uh, the risk of human extinction is 100%

the main question is do we have 8 months left or 8 years, 80,000 or 800,000.

~~~
Symmetry
I'm hoping we'll make it to 80,000,000,000 but your general point is correct.
There's only so many computational cycles that can be extracted from the
universe before you run out of places to sink entropy.

~~~
uvdiv
That's a bit pessimistic. If you can do 10^10 years, you can go out to a red
dwarf and stick around 10^13-10^14 more. And with a few small mods you can go
further. For example, should you want a "small, slow" world, consider what
what would if you "turned off" a star (e.g. tore it apart until it could no
longer sustain fusion) and burned its fuel more slowly, in engineered fusion
reactors. The sun has a natural lifespan of a few billion years; but consumed
at a relative trickle of 10^17 W (the part that strikes the earth), there's
enough fuel for about 10^21 years. (Conversely, you could grow your world
until it matches the power output of a star, like a Dyson sphere. Urban
planning decision.) The part that humans actually metabolize is barely 10^12 W
(10^26 years in bubble chambers), and non-organic human simulations could do
better. And there's a lot more hydrogen where that came from... and human
knowledge of this universe's physics is clearly incomplete (what the hell is
dark energy?)

------
tianshuo
I'm bringing up an equation that needs consideration.
PotentialDamage=NxP1xP2xP3xMaximumDamage

Where N=Population of World P1=Percentage of people with knowledge and access
to Dangerous Technologies P2= Percentage of Personalities who are destructive
P3=Percentage of People who actually act MaximumDamage=The Number of
Casualties that can be caused by a single person

While P1,P2,P3 is relatively stable, and N is almost linear, MaximumDamage is
exponential. You can imagine weapons accessible to normal evolved from sticks,
to axes, to guns and explosives. At the present moment the main reason that
nuclear weapons are not in exploded by terrorists is not because of technology
but because of scarce resources. But for bio-warfare, technology cost will
exponentially shrink and impact will exponentially grow.

This means if this goes on, PotentialDamage will one day be larger than our
population, and extinction will occur. By estimating these parameters we can
even predict a data.

------
koningrobot
The one thing that's always skipped over in this kind of article is... Why?
The universe doesn't care whether we're around or not. It sure as hell doesn't
affect _us_ whether people are around even as nearby as a thousand years from
now. If you're worried about future generations suffering, then don't make
them. That's a lot cheaper than spending fortunes on a technological solution.
The money saved can be used to help those suffering now.

------
bdunbar
You don't have to be a Velikovsky to appreciate that the solar system is a
giant game of billiards, humanity is in a fragile aquarium perched on the
eight-ball.

To put it another way: I have fire insurance, but I've never had a fire. I
have a gun (or four) but I've never had to shoot anyone.

Prudent folk recognize that life is risk and prepare accordingly.

------
Tangurena
We're underestimating the risk because we're using Net Present Value to
determine the value of future events. While it can be useful in business to
determine profitability of future endeavors, it isn't useful when we are
unable to correctly calculate all the other things that get wrapped up into
the word "externality".

------
uvdiv
_"Even with nuclear weapons, if you rewind the tape you notice that it turned
out that in order to make a nuclear weapon you had to have these very rare raw
materials like highly enriched uranium or plutonium, which are very difficult
to get [a]. But suppose it had turned out that there was some technological
technique that allowed you to make a nuclear weapon by baking sand in a
microwave oven or something like that. If it had turned out that way then
where would we be now? Presumably once that discovery had been made
civilization would have been doomed."_

It works both ways; technology could find simpler ways to build superweapons,
but it could also lower the entry barrier to complex, resource-intensive
projects. Look at the consequences of even mild singularity predictions for
robotics and AI -- things that lower industrial "costs" by orders of
magnitude, things like 3D-printers, self-replicating machines, cheap and
ubiquitous fab robots. Anyone could build incredibly sophisticated machines in
their own homes -- including sports cars, jet engines, and giant TV screens,
but equally, compact laser enrichment cascades [b] and nuclear weapons.

Perhaps it's lack of knowledge on my part, but I don't see what will stop
atomic bombs from being as common as handguns in 30-100 years. Fissile
material is ubiquitous [c]; there's no barrier beyond economics and
engineering, of exactly the sort near-term AI could unpredictably disrupt.

[a] _(Tangential silliness: terrestrial uranium WAS weapons-grade material a
few billion years ago -- U-235 decays with a 0.7 billion year half life,
compared to 4.5 billion years for U-238, so in geologic history the fissile
fraction used to be extremely high. Another one for the "anthropic principle"
bin: earth's intelligence must have evolved now, and not earlier, because if
it had it would have trivially nuked itself...)_

[b] _Check out [NYT][APS] -- it's actually a mainstream position among arms
control experts to consider shutting down US research in laser enrichment, to
prevent the knowledge from being developed at all. To me this plan sounds
about as airtight as have suggested to close the NSA, to keep mathematical
secrets like RSA from being discovered._

[c] _Here's a blogger who goes hiking and brings back uranium ore by the
bucket [Willis], for his own hackery experimenting. Uranium ore occurs
everywhere [Cameco]; even common granite is a low-grade ore containing 5-50
ppm (parts per million) U [AZGS]; and research suggests it's even feasible to
extract it by bulk from seawater [NBF]._

[NYT]
[http://www.nytimes.com/2011/08/21/science/earth/21laser.html...](http://www.nytimes.com/2011/08/21/science/earth/21laser.html?pagewanted=all)

[APS] <http://aps.org/units/fps/newsletters/201007/slakey.cfm>

[Willis] [http://carlwillis.wordpress.com/2008/02/20/uranium-
chemistry...](http://carlwillis.wordpress.com/2008/02/20/uranium-chemistry/)

[Cameco] <http://www.cameco.com/uranium_101/uranium_science/uranium/>

[AZGS] <http://repository.azgs.az.gov/uri_gin/azgs/dlio/414>

[NBF] [http://nextbigfuture.com/2009/09/uranium-from-seawater-on-
la...](http://nextbigfuture.com/2009/09/uranium-from-seawater-on-large-
scale.html)

~~~
angersock
_"Perhaps it's lack of knowledge on my part, but I don't see what will stop
atomic bombs from being as common as handguns in 30-100 years. Fissile
material is ubiquitous; there's no barrier but economics and engineering, of
exactly the sort near-term AI could unpredictably disrupt."_

You guess right--it's your lack of knowledge.

 _Fissile material_ is not useful for making nuclear weapons (of any more than
the dirty bomb variety). So far, we seem to have converged on uranium and
plutonium, and while at least uranium is relatively common in the Earth the
processing and dredging required to get a useful amount of it, and then the
refining to get the useful isotopes from _that_ , is nontrivial.

Engineering and fast computers are great and all, and will get you arbitrarily
close to the physical limits--but we're there right now, and physics says you
aren't getting a centrifuge with meaningful output in your garage.

More troubling is the idea that frankly we've had the technology to build
sports cars in our homes for decades and decades now. It's called buying a
lathe, an arc welder, a mill, a forge. Despite the availability--cheap!--of
these things to the general populace, not only is it not widespread among
those who can--and there are damned few of those even with this magical
Internet thing telling you how to do all of it--it will get less likely as
people focus on cat pictures and tweeting.

We aren't worthy of a singularity as a culture.

EDIT:

Thank you for attaching some interesting facts. Allow me to do the same:

The useful fissile uranium isotope (235) accounts for bout .72% of naturally
occurring uranium (<http://web.ead.anl.gov/uranium/guide/facts/>).

Your hiking source claims uranium metal, but points out that it is completely
locked up in slag and that only at large-scales does the approach seem
tractable.

Even allowing for _that_ , the amount produced is negligible.

That's just for metal--we aren't even talking about a usable isotope yet.

Take 7/1000 of that result from the blogger, and wave a wand to make it pure
enough to use.

Now collect the (at least) several pounds needed of that to make a functioning
device. Now do the machining on the rest of the device to make it function
correctly (have fun with the berylium dust, if you go that route). Now do the
timing electronics, and the charge shaping (if you go _that_ route).

This.

Is.

Not.

Garage.

Technology.

~~~
losvedir
> _More troubling is the idea that frankly we've had the technology to build
> sports cars in our homes for decades and decades now. It's called buying a
> lathe, an arc welder, a mill, a forge. Despite the availability--cheap!_

Now I'm curious. How much would it cost to set up a reasonable home garage
with all this machinery?

~~~
ChuckMcM
Depends on your budget of course. Machine tools don't 'depreciate' past their
accuracy. Which is to say that a mill that can hold one thousandth of an inch
repeatably costs $X used and on that can do one ten-thousandth (tenth)
repeatably is $10X.

The challenge of building a sports car is generally building your own engine
(which people don't do often unless they are steam based). You need to get the
engine block cast and that requires a steel foundry. Building a blast furnace
in your garage is quite difficult if it needs to have the volume capacity to
make even a fairly small engine block. Once you have the castings however the
various other bits can fit in your garage easily and do cost between $8K and
$150K depending on newness, accuracy, control methodology etc.

I'm also assuming that you would use fiberglass layup for the body panels
since a sheet metal press that can make a 'hood' sized piece, or fender sized
piece, is also quite large (and tall).

~~~
mindcrime
_The challenge of building a sports car is generally building your own engine
(which people don't do often unless they are steam based). You need to get the
engine block cast and that requires a steel foundry._

Unless you just insist on starting with a home-made block, you can acquire
"naked" engine blocks fairly easily. Or at least you could a few years ago,
I've been away from that scene for a while, so my knowledge isn't completely
up to date. And even if you can't get a naked block you can certainly get
partially built engines (usually the block, crankshaft, pistons and connecting
rods) or "crate engines" (usually everything except intake manifold and
carburetor / fuel injector) straight from the manufacturers. For example, see:

[http://store.gmperformanceparts.com/store/SelectProd.do?prod...](http://store.gmperformanceparts.com/store/SelectProd.do?prodId=7704&redir=true&manufacturer=GM&category=Engines&name=LS327/327%20&model=%3C!
--19165628--%3E)

'course, starting with a pre-forged block and partially built engine isn't
quite like doing it from scratch, but then again, I'm assuming anybody
building their own car is buying pre-rolled tubing and sheet stock, etc., not
literally building everything up from iron ore, aluminum ore, etc.

~~~
ChuckMcM
Yes, any sane person would start there :-) I was just responding to 'build
from scratch' which can be interpreted quite literally. A friend of mine
builds race cars which are basically some 'regular' car, except all the
structural bits get replaced. I joked with him that it would be simpler if he
just took the engine out and built a frame around the engine, but he claims
the previous frame are his version of a story stick[1].

[1]
[http://www.woodcraft.com/Articles/Articles.aspx?ArticleId=28...](http://www.woodcraft.com/Articles/Articles.aspx?ArticleId=289&page=1)

~~~
mindcrime
_A friend of mine builds race cars which are basically some 'regular' car,
except all the structural bits get replaced. I joked with him that it would be
simpler if he just took the engine out and built a frame around the engine._

Cool. My dad builds and races late model stock cars and has been involved in
racing as long as I can remember... so yeah, I can relate to exactly what you
mean there. The race cars have almost nothing left of the original car except
the exterior sheet metal.

------
gwillis13
Oh... I do love everyone's opinion about this topic. It's interesting, but I
think everyone can agree that "human evolution is a train with no tracks".
Thinking there is a formula to solve it's equation by humans is pretty
amusing.

------
alexro
Maybe. On the other hand, which risks we estimate properly? What if humans are
incapable of properly estimating risks, especially related to themselves?

It all comes down to the ability to predict future, and we are really bad at
it.

------
akkartik
I was reminded of Bill Joy's article from 2000:
<http://www.wired.com/wired/archive/8.04/joy.html>

------
pjscott
Human extinction sounds a lot less plausible since it's never happened before.

------
joseph4521
Many people see technology as one of the biggest threats but don't forget
about our growing stupidity:

<http://www.youtube.com/watch?v=fMkrJOOTjj4>

It may be what will kill our civilization after all, if not mankind itself.

------
shingen
I'm always fascinated by the extremely long term horizon concerns that people
bring up in discussions about the future of humanity.

200 to 300 years out there is no humanity as we know it today. Whatever we are
at that juncture, it won't be "human" as we now define it. Our self directed
evolution has long since taken over, and it's accelerating at an extraordinary
clip. You can debate the merits of that, the details of it, but it's happening
either way.

In the next two or three decades we'll have begun to completely take over
genetic alteration / evolution / improvement / etc, of our species. Within a
few decades after that, we'll be severely altering what we are. Within 150 or
200 years, it'll be very hard for modern humans to relate to the ancestor
humans from 2012.

Concerns about asteroids or global warming and so on are moot. We won't be
here as a species pumping CO2 into the atmosphere or waiting around helplessly
for a rock to crash into the surface. It's not an issue of if, it's just an
issue of how long it takes and how many competing models of our self directed
evolution become options.

The sole threat to that future is super virus / disease, most likely man-made.
It's the only thing that could stop our evolution and wipe out our species in
that couple hundred year time frame. Even nuclear war isn't a threat to
species survival, you could detonate thousands of nukes simultaneously and it
wouldn't come close to killing us off.

~~~
nollidge
I think you vastly overestimate our ability to understand genetics. The genome
(any species' genome) evolved so haphazardly and with such deep and
interconnected causal spiderwebs I think it's going to take centuries to
understand it to the extent that we'll be able to intentionally influence it
more than genetic drift and selection currently do.

NINJA EDIT: Regardless, the certainty with which you seem to regard your
claims is hardly justified (unless you've got some concrete evidence you can
share). If you're wrong, we'll have wished we were thinking about these things
all along.

~~~
mattmanser
Given that we're doing it right now to plants and animals right now I think
you're very wrong.

~~~
refurb
What we're doing now is a pretty haphazard introduction of non-nature genes
into plants and animals often with unexpected effects.

Look up histone acetylation and DNA methylation. We're barely scratching the
surface of how gene expression works.

To say we're going to rebuild the human genome in the next 100 years is a
little fanciful. Reminds me of the "trains to the moon" that Popular Mechanics
said we would all be riding in the 1980s.

------
hristov
Wow an entire article about human extinction without a single mention of
global warming or climate change. The Atlantic are such reliable whores.

~~~
pjscott
We could totally survive any climate change that we've ever seen in the
geological record. Even if famine claims billions of lives, it wouldn't be a
species-ending event. Bostrom is worried about things that have considerably
more destructive potential.

