
On “Armchair Epidemiology” - ssklash
https://www.scottaaronson.com/blog/?p=4695
======
notacoward
There are two things seriously wrong with this article.

(1) It treats authorities and non-authorities as monolithic. Far from it.
There were plenty of serious, well qualified epidemiologists who were warning
of the danger, and they were right. There were even more non-serious people
using their popularity in completely unrelated fields to minimize the danger,
and they were wrong. Sadly there still are, most definitely including here.
Yes, the people in the actual seats of US federal power got it wrong, and
plenty of amateurs got it right, but it's more complicated than the "don't
trust experts" narrative.

(2) Sample of one. Even if it weren't for the above, and the two groups were
monolithic, this is just one case where the authorities were wrong and others
were right. Yes, it's a very _prominent_ case because of the consequences, but
prominence is not the same as statistical significance. Treating it so is a
form of cognitive bias/error.

The real lesson here is not to distrust experts or authorities. It's not to
trust conspiracy theorists or people with whom we empathize more because
they're in our own industry. It's to trust _science itself_ \- empirical facts
and analysis applied to facts. Both experts and others can help us to
interpret those facts, put them in perspective, correct common errors in
analysis, but ultimately it should be the facts that rule. The reason so many
authorities and non-authorities were wrong is because they rejected the
scientific approach, not because of who they are.

~~~
OrderlyTiamat
good points, though I'm a bit bothered with your second point here.

First of all, I think calling this a single event is reductionist. We're
talking about weeks- months, even- of experts and non experts warning about
this, and the systems of authorities failed completely to react in time. We're
not talking about failing to heed an important warning once, we're talking
about a long string of choices here.

Secondly, This isn't _just_ a prominent case, it is a very important case.
Treating this as a series of bernoulli trials with a single (statistically
irrelevant) failure is an extremely reductionist model of the reality here.

While your conclusion of trusting science, empirical facts and analysis rings
true to me, I think we should be wary of overly simplifying the subject
matter.

------
screye
What has struck me most about Epidemiology, are the qualifications needed to
be someone pursuing a career in it.

It seems like an Epidemiologist is most likely to be a doctor with some
knowledge of statistics. On the contrary, I would have thought that a
statistician/operations research person with domain knowledge of medicine
would be a lot better suited to study epidemiology.

A lot of the existing models also seem really naïve for something an entire
field depends on. Tradeoffs between human life lost due to the disease and
effective human lives lost due to economic downturns seem entirely unstudied.

Czechia's actions of forcing people to using masks seems to have worked
amazingly well for flattening out the curve. The "science" not seems to
support it, and official recommendations are now moving back to recommending
masks for all. The popular hypothesis for this, is that it stops
nonsymptomatic carriers from being spreaders.

Now this sounds like a fairly obvious thing to realize, and to an extent,
existing models should have caught this a lot earlier.

I grew up as a kid with an almost religious belief in Science. Over time, I
have come to realize that a lot of our sciences are at best empirical and at
worst entirely unverifiable. I am a 100% certain, that in a 100 years, we will
look at medicine and nutrition of the 2000s, the way we look at humors and
blood letting.

~~~
Engineering-MD
There is benefit to a team with diverse backgrounds. Mathematical and
statistical knowledge is vital, but so is understanding how hospitals and
humans react to disease and interventions. As for looking back at medicine in
200 years as blood letting and humours, I think that is to misunderstand
medicine in its entirety. Modern medicine is evidence based where it can be,
mixed with a high level of uncertainty and best application of basic science.
This has predictive power, but is obviously limited due to the complexity of
the system. A better analogy would be industrial era engineering vs modern day
engineering: the same aim with similar attempts but much better understanding
of the system at a higher level of resolution achieved by improvements in
technology and base understanding.

~~~
Engineering-MD
As an addition, I was browsing the r/physics subreddit and came across a very
relevant post from a physicist’s point of view.

[https://www.reddit.com/r/Physics/comments/frsd16/the_best_th...](https://www.reddit.com/r/Physics/comments/frsd16/the_best_thing_you_can_do_to_fight_covid19_is)

~~~
thu2111
I disagree with that post completely.

If you look at the stuff coming out of Imperial and Oxford in recent weeks,
there are a huge number of basic problems. Their papers can't be replicated
for multiple reasons. They're using known-bad input data. They're not
providing uncertainty bounds. There's no peer review. The Imperial paper
assumed constant hospital capacity. Their papers reach diametrically opposite
conclusions. They have a track record of model failure.

It's all practically a poster child for the replication crisis.

The /r/physics post criticises physicists who write some code to do basic
curve fitting. Has he looked at the Oxford epidemiology paper? It is by all
accounts literally curve fitting to the first 15 days of outbreak and reaches
diametrically opposite conclusions to the Imperial paper.

It criticises people who write papers and then upload them to arXiv because
reporters will find it and create panic. Would this person prefer people to
use the Imperial/Oxford technique of sending papers directly to journalists,
and entirely skipping the whole upload to arXiv step?

It blames physicists for creating "denial": _" You become a punchline to a
denier that says, they can't decide if there's going to be hundred thousand
cases or a hundred million cases! Scientists don't know anything!"_ \- guess
what, they're saying that already and are going to say it a lot more because
epidemiologists themselves keep contradicting and criticising each other for
doing bad work, in public. Additionally, there are enough people with
scientific literacy to understand the limits of statistics and modelling out
there. They aren't as easily confused as this person rather seems to assume
(does he have any data showing that people conflate epidemiology with
physics?).

Right now I absolutely want to see papers written by physicists studying
COVID-19. Why - because physics is a significantly more rigorous field than
epidemiology. I trust the average physicist to have at least slightly higher
standards, for instance I trust them to at least pretend to care about
statistical uncertainty, and I suspect many of them will upload their source
code. I don't expect any of them to email their paper straight to known-
friendly newspapers.

~~~
Engineering-MD
Tbh I don’t disagree with quite a few of your points. A lot of papers out
there atm are pretty poor quality. But having more similar papers won’t help
matters. Wait for the high quality papers that will come from more valid high
quality data and models that haven’t been rushed out. As for physicist have
higher rigour, I’m sorry but that’s just arrogance. There are good scientists
and bad scientists, and whether they studied physics or epidiemoology isn’t
the point. Stop assuming that just because people studied a subject you like
and are familiar with that they are better than the other group.

~~~
thu2111
I agree with you on one level - the difference in rigour I perceive is a
function of surrounding culture in a field, not the specific people who are in
it. On the other hand if you look at the confidence levels required to publish
something as a discovery, they're much higher in physics, partly that's
fundamental to the field and partly it's that in physics scientists are OK
with statements like "to make the next discovery we must spend 10 years
building a billion dollar machine that will require international cooperation
on a scale never seen before". Whereas in most other fields their ambition
stops with collecting a bunch of grad students, or downloading data from
sources that wasn't meant for the purpose to which it's put.

------
tjr225
If this current situation has taught me anything it is that the internet is a
mush of armchair garbage and it’s overwhelming to decipher- yes even what I’m
typing now.

The whole thing has me desperate to ignore everyone and read a book. What a
joke; to think we are all somehow experts.

~~~
naringas
I agree, I still cannot make up my mind about masks...

It seems like "back in the day" people trusted the experts, becuase well,
there weren't many alternatives (?); but now thanks to the internet everyone
can google for a while and feel like they have a worthwhile opinion.

It seems like now it's possible to find an expert (and/or data) to justify any
opinion.

I am not saying this is bad, but that it's just different. The game has
changed.

~~~
Gibbon1
After rolling things around my current thought is, what if this isn't the last
rodeo? What if there is another outbreak?

Because there will be. Of 'something' but we don't know what that will be.

Seems to me then what you want to a robust set of responses that do not
require people to make decisions based on god knows what. Because in the
future you don't know ahead of time what's going to be helpful and what won't
be.

What you want to public to do is reflexively anything that might help. And
then later tell them what measures they can back off on. Not the other way
around.

------
knzhou
This whole mess has been a repudiation of business-as-usual politics, but it
has _not_ been a repudiation of scientific experts.

Scientific experts across the world have been alarmed for months now. The only
problem is that, as with any new and uncertainty situation, the error bars
started absolutely enormous, so the range of expert opinion has been wide.
This is the case in any crisis. With the benefit of hindsight, you can
_always_ find _an_ expert who was wrong, but that doesn't mean anything.

It's important to say this because later, you're going to start to hear
bloviating from politicians who failed to act for months. "Nobody knew this
was coming." "Everything that went wrong was solely the fault of
$enemy_country or $rival_party."

~~~
professorgerm
It has been a repudiation of the communication layers between experts and
average people, as well, which I think might be more of Aaronson's ire than
experts qua experts (it's certainly more of my ire, but I don't have his
platform).

A standard of "scientists dabbling in journalism to improve communication"
instead of "journalists dabbling in communicating science" would likely have
gone a long way to improving the civilian response and trust in experts.

There's also the factor of the "Noble Lie" dishonesty around wearing masks. I
suspect intellectual-ish contrarians (like Aaronson et al) are more angered by
this than the average person, but we ended up with shortages and sellouts
anyways: a significant number of non-experts didn't buy into that noble lie so
the experts/organizations that pushed it (not to be confused with ALL experts)
burned a lot of good will to basically no effect.

ETA: I considered replying to your comment about the Harvard epidemiologists
but would rather edit it in here to avoid two replies to one person: I think
that's a great example of my point. The story wasn't _broken_ by bloggers, but
bloggers were more likely to be amplifying the concerned experts than our
"traditional media powerhouses."

------
wwweston
Relate to a lot of this (and it's especially worth noting there is something
about identity contempt especially on social media but also IRL that hobbles
our ability to coordinate understanding and action) but... let's also not
forget it wasn't only contrarians and autodidacts who were making concerned
noises -- there are plenty of professional epidemiologists and other
specialists who were beating the warning drums.

~~~
knzhou
Exactly. For example, back on February 3 we had this from Harvard
epidemiologists:

> “Many epidemiologists and people who are following this outbreak closely are
> assuming that it’s probably quite a bit more widespread than the case counts
> suggest,” said Michael Mina, assistant professor of epidemiology at the
> Harvard T.H. Chan School of Public Health. “Many people also think that
> there’s probably over 100,000 in reality out in mainland China, for example.
> We just don’t capture all of those through the case reporting we have. … We
> can assume that this is growing at somewhat of an exponential rate, and it
> will continue increasing in scale.”

This story wasn't broken by bloggers!

Source: [https://news.harvard.edu/gazette/story/2020/02/as-
confirmed-...](https://news.harvard.edu/gazette/story/2020/02/as-confirmed-
cases-of-coronavirus-surge-path-grows-uncertain/)

------
btilly
Looking back at my thinking at the same time I am also ashamed at how slow I
was to realize it.

I also trusted in our institutions. I knew what lengths that they went to in
SARS. I understood how much thought has gone into public health. I found
articles explaining why one would rationally ration available tests. Even when
it was clear to me that we were in for a pandemic and millions would die, I
completely failed to predict that we'd be willing to put Western countries in
lockdown - or that lockdown would be as effective as it has been.

I probably came to the right conclusion before Scott did. But still the only
thing that I did right with my knowledge was to tell my wife to buy a put on
the DOW back when it was still rising.

------
naringas
this whole covid19 debacle is exposing a very real change brought about by the
internet.

we are being forced to adapt and to develop new ways to relate to information,
to asses it's trustworthiness, etc...

it's an epistemic phenomenon. perhaps another facet of this phenomenon are the
filtered information bubbles.

But this pandemic really puts the pressure on us to face the reality of our
current information society.

------
jasonhansel
> A viral article implores us to “flatten the curve of armchair
> epidemiology”—that is, to listen only to authoritive sources like the CDC,
> not random people spouting on social media.

> This was notable to me for being the diametric opposite of the actual lesson
> of the past two months.

So, because some (unspecified, unnamed) experts weren't able to predict the
future with perfect accuracy, we should give equal credence to "random people"
instead?

Medical professionals, scientists, and public health experts are still the
best available source of knowledge about preventing the growth of this
pandemic.

Spreading the message that we should ignore epidemiologists is wrong at best
and dangerous at worst.

~~~
fiblye
>So, because some (unspecified, unnamed) experts weren't able to predict the
future with perfect accuracy, we should give equal credence to "random people"
instead?

That wasn't my takeaway.

My takeaway is that if you have some intuitive sense that shit's about to hit
the fan and authorities keep telling you to stay calm and don't take any
special precautions, all while danger is clearly growing, it's best to lean
towards following your intuitions gradually just in case things ramp up. You
shouldn't buy 10000 rolls of toilet paper, but all those people who thought,
"Hmmm, maybe I should buy some masks and a couple extra bottles of hand
sanitizer. Maybe not visit the movie theater anytime soon as well" while
authorities were saying "Go to work and be normal. Masks are useless"... well,
they were right and it would've been better to take those then "unnecessary"
precautions.

Reminds me of people talking about government surveillance pre and post
Snowden. If you wanted to encrypt all your messages and avoid Facebook, etc,
you were a conspiracy nut because it was impossible to surveil everything.
Overnight it became "Of course they monitor everything. Why wouldn't they? But
we have end-to-end encryption now and we can completely trust it."

------
empath75
There are people still, in every thread here about it, downplaying it.

I’m sure someone will be along momentarily to tell us that it’s no worse than
seasonal flu.

------
elchief
Who should we believe, then, the WHO? They told us China had it under control,
and not to wear masks

~~~
knzhou
The WHO has been urging everybody to get prepared for 10 weeks now, while
constantly giving the same good advice (test, trace, lockdown, social
distance).

People are pretending this is all WHO's fault because they said some wrong
stuff in early January, when the correct conclusions were _not at all
obvious_. And they've been completely correct and transparent since late
January. It's almost April now. At this point, it's just a worn out excuse.

~~~
adnzzzzZ
They literally just posted this a few days ago
[https://twitter.com/WHOWPRO/status/1243171683067777024](https://twitter.com/WHOWPRO/status/1243171683067777024)
when we know it's false and that masks do offer some level of personal
protection on top of making everyone around you safer, given that you can
infect others while asymptomatic.

~~~
knzhou
Okay, yeah, the mask thing absolutely blows my mind. I have absolutely no idea
what their motivation for saying this is, when just about every expert in
every east Asian country disagrees.

~~~
temp-dude-87844
That tweet and the attached video are entirely true. It's just that individual
people may wish to not chance it, and wear a mask nonetheless, and masks will
end up allocated by luck and purchasing power instead of need and risk
profile.

There's also some evidence that widespread usage of masks may be effective in
lowering the risk for such a disease, but deploying widespread usage of masks
equitably on a global scale is nigh-impossible right now, so it's
counterproductive to focus on it. Distancing is achievable instead, and leads
to a similar result.

~~~
zozbot234
> deploying widespread usage of masks equitably on a global scale is nigh-
> impossible right now

You don't need high grade medical masks to significantly reduce the amount of
transmission. Any sort of fabric-based covering on your face can help, just
make sure you can comfortably breathe through it. Fabric is cheap and
ubiquitous, it's literally everywhere.

~~~
dariusj18
That doesn't seem right. If the mask doesn't prevent droplets from penetrating
doesn't it actually act as more of a trap for potentially viral-laced droplets
to gather and get inhaled?

~~~
bmn__
Sounds like a hypothesis that affords to be tested.

Oh look, the results are already in!
[https://news.ycombinator.com/item?id=22727761](https://news.ycombinator.com/item?id=22727761)

~~~
dariusj18
So I think I inferred that PC was referring to wearing a mask to prevent
getting infected, which may have not been the case, but in a way needs to have
clarification. 100% mask usage would be great, then the particles never get
into the air from the infected. I think it's important to not let people think
that wearing a simple cloth mask will prevent you from being infected.

------
apatil
I sent a similar text to a medically vulnerable friend on Jan 25 saying I
thought it was "scary, but not the scariest thing in the world right now",
despite having spent some time working with an epidemiology research group. In
hindsight, I knew enough to take the risk much more seriously.

This article: [https://virologydownunder.com/past-time-to-tell-the-
public-i...](https://virologydownunder.com/past-time-to-tell-the-public-it-
will-probably-go-pandemic-and-we-should-all-prepare-now/), plus the rising
mortality numbers, spurred me to start warning the medically vulnerable people
in my life on Feb 23. I am very grateful to the authors for helping me do the
right thing.

At this point, it seems to me that our experts are still letting us down in
two important ways:

1\. They aren't rebutting this implicit notion that any mortality risk below
1% can be rounded down to zero. As a consequence, the idea that it might be a
good idea to let younger, healthy people just go out and get the virus and get
immune seems to still have currency, and this is really dangerous. We can't
open things back up until we have a credible plan for driving the infection
rate toward zero. Leaving aside the risk of life-changing injury, a mortality
risk of eg 0.5% applied to eg half of young adults would still be by far the
most dangerous thing that happens to most of them this year, and a huge
personal tragedy for many of us. Do the math and work out how many people you
know would die.

2\. Simple, inexpensive measures like widespread masking, handwashing, gloves
and eye protection will clearly be important components of any credible plan
for driving the infection rate toward zero, unless we get an early vaccine.
The experts should be hammering the point that we all need to be fastidious
about those things whenever we open back up again.

------
modzu
this is borderline terrorism, imo. right from the into, "Dr. A..." which to a
layman would reasonably interpet as meaning "M.D" rather than professor of
computer science. the one thing i have noticed in all this is the utterly
astonishing abandonment of science, reason, and rationality. that scares me
much more than disease.

------
zozbot234
Hindsight is always 20/20.

------
darawk
What's particularly impressive about this situation is that all the
institutions on all sides got it wrong. Trump and his administration
downplayed it, _and_ Vox, Wapo, NYT et al downplayed it. You would think these
two would act as a check on one another, but they completely failed to do so.
The one time we needed them to disagree with each other, they agreed
forcefully.

------
hirundo
"A viral article implores us to ... to listen only to authoritive sources like
the CDC, not random people spouting on social media. This was notable to me
for being the diametric opposite of the actual lesson of the past two months"

This is the opposite sentiment to "there are no libertarians in pandemics."
Keen perception of reality is unlikely to be concentrated at the top of
political hierarchies. Competent Authority isn't an oxymoron but it's
distressingly unreliable.

------
whatever1
Let's talk first about the domain experts, professors in prestigious
universities, that were reassuring the public that COVID-19 is no way more
fatal than the seasonal flu [1]. The same moment that the ICUs in Italy and
Spain were already overwhelmed and the physicians could not find the necessary
protective equipment to keep saving lives.

Are there gonna be any legal repercussions to these?

[1][https://www.wsj.com/articles/is-the-coronavirus-as-deadly-
as...](https://www.wsj.com/articles/is-the-coronavirus-as-deadly-as-they-
say-11585088464)

~~~
modzu
what are you talking about? that article is contemporary, they havent been
proven wrong.

~~~
whatever1
Art can be contemporary. Science not. We are currently looking at 200k deaths
in the USA [1] from COVID-19, in the good scenario. How is it even compatible
with their 0.01% mortality rates? Have we forgotten how to divide numbers?

[1][https://thehill.com/homenews/administration/490138-birx-
says...](https://thehill.com/homenews/administration/490138-birx-says-us-
could-have-up-to-200000-coronavirus-deaths-if-we-do)

P.S. They even make the claim that COVID-19 has 1/10th the mortality of
seasonal flu :facepalm:

~~~
petertodd
"We are currently looking at 200k deaths in the USA"

That is a prediction. It is not yet a fact.

Right now we just don't have a good idea of how many people 1) how many people
are currently infected, 2) how many people have already been infected and
recovered without being tested, 3) how many people were already immune and
would never have been infected, 4) how the _way_ you are infected changes the
outcome of the disease.

The mortality rate depends directly on those numbers. Until we know those
numbers, we just don't know for sure. With H1N1 initial estimates of the
fatality rate per infection were almost two orders of magnitude too high. A
similar error may or may not be happening here.

The best we have right now is a tiny handful of limited, non-representative,
"accidental experiments" like the Princess Diamond, that may or may not
represent how the disease spreads in the community (note question #4). This
will however soon change as more randomized testing is done, and in
particular, anti-body testing to determine who already got the disease and
recovered.

~~~
whatever1
This way of thinking is exactly what brought us here.

"We need more data".

No you dont need data, look at the ICUs you fool. They are struggling to find
ventilators and in worse cases with lower capacity, even just for beds to deal
with the influx of patients.

But you can wait for the data. They are incoming. Hopefully we will still be
here in 2 months to discuss.

------
alexandercrohde
This is a topic that seems to be in a perverse conflict of interests.

A. Politicians have interests representing themselves as well as their
country, and thus have given dishonest advice (masks don't help) with an
ulterior motive (save them for hospital), downplayed the problem, and covered
up the number of cases.

B. I think a lot of actually qualified experts (doctors, scientists) to think
the public can't handle the truth perhaps? I still have yet to see anybody in
any position of respect or authority simply say the bald truth: "A few million
Americans will die, at least, unless some radical cure happens. Social
distancing is a drop in the bucket, ventilators make a tiny difference. The
vast majority of young people will be totally fine"

I'm disappointed in our society for failing to bubble up and synthesize good
information quickly.

~~~
zozbot234
> Social distancing is a drop in the bucket

Sorry, but look at how China, South Korea, Taiwan etc. are doing right now -
this doesn't jive at all with "social distancing is a drop in the bucket".
Even Western countries/places which started social distancing early in their
local outbreaks look like they'll be a _lot_ better off than places which
didn't.

> The vast majority of young people will be totally fine

This is also quite wrong, unfortunately. A sizeable minority of young people
_will_ need hospital treatment for COVID, and perhaps intensive care. Given
overwhelmed healthcare systems, they won't be "totally fine" at all. Of
course, older folks are at increased risk.

~~~
JimTheMan
I highly recommend this video from Korea (Professor Kim Woo-joo from Korea
University Guro Hospital) that gives a Korean perspective on the disease and
the western response to it:

[https://www.youtube.com/watch?v=gAk7aX5hksU](https://www.youtube.com/watch?v=gAk7aX5hksU)

A few points of interest: (He is a very respectful fellow so apologies if the
summary seems a bit to the point)

\- Masks help and he is slightly puzzled why the west hasn't adopted that
culture

\- Their testing and tracing helps and they doubt the US would allow their
level of privacy invasion

\- Korea is doing well because they have had to respond to pandemics and were
prepared (as are Taiwan, Singapore etc)

------
bJGVygG7MQVF8c
> I sent a quick reply two minutes later:

> For now, I think the risk from the ordinary flu is much much greater! But
> worth watching to see if it becomes a real pandemic.

Nope. Lost all credibility right there. Do not pass GO, do not collect 2
minutes of attention. Sorry.

Oh, you've learned something from the error and you've updated your priors?
Great, good for you. You're still unreliable.

~~~
AlexMennen
This seems overly harsh. Few people took COVID-19 seriously on Feb 4, so this
standard makes almost everyone have no credibility, including people who don't
admit to the error. All humans are unreliable.

~~~
defen
Here's a blog post from February 6 by Greg Cochran, one of the people
mentioned. Looks pretty good in retrospect
[https://westhunt.wordpress.com/2020/02/06/strategy-not-
the-f...](https://westhunt.wordpress.com/2020/02/06/strategy-not-the-flu/)

