
No, I won't read your amateur Covid-19 “research paper” - ryanwaggoner
https://ryanwaggoner.substack.com/p/no-i-wont-read-your-amateur-covid
======
crusso
This screed is disappointing to me. We tell people to go out and do their own
research. We tell people to avoid just buying into group think. We want people
to be informed citizens who do research, share their research with others, and
are open to revealed facts vs just buying into the most appealing emotion-
laden rhetoric.

Some dude does a bunch of research, reasons as carefully as he can about the
subject, and openly provides his research and reasoning to others. The very
first note in his research is a disclaimer to say that he's a layperson with
no medical or scientific credentials. It doesn't appear that he's trying to
deceive anyone.

Why the piling on? Why the expenditure of so much effort to silence his
intellectual inquiry?

~~~
ryanwaggoner
The disclaimer was added after the fact.

I'm not trying to "silence" anyone. The author should be free to write
whatever he wants. I just think it's irresponsible and I'd like to encourage
people with any influence or audience to do better.

EDIT: lots of criticism of me not mentioning that the author added a
disclaimer later. It's a fair criticism, I've updated my article.

~~~
crusso
_The disclaimer was added after the fact._

So you're saying that the guy is open to criticism of his work and correcting
it.

You say that like it's a bad thing.

I don't see how it's irresponsible to work hard at being a more informed
citizen and helping to pass that information on to others.

~~~
btilly
The fact that he changed that without taking into account any of the _other_
flaws in his work suggests that he is not, actually, open to criticism of his
work.

Here are some examples. He shows charts estimating per age range fatality
rates and leaves out the most vulnerable group. He estimates an IFR and thinks
that is comparable with a CFR. (They are not.) He is apparently unaware of the
fact that IFRs have long been thought to be a bit under 1%. (For example
[https://www.imperial.ac.uk/media/imperial-
college/medicine/s...](https://www.imperial.ac.uk/media/imperial-
college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-
modelling-16-03-2020.pdf) in mid-March was the paper that convinced the UK to
do a lockdown - it used an estimated IFR of 0.9%.) He does his own IFR
calculation for the least vulnerable groups without looking at research
showing the full IFR. He complains about hospitals being underloaded while
refusing to acknowledge that hospitals would be overloaded without lockdowns.
He fails to admit that it is extremely hard to limit transmissions within
managed care facilities, and in an environment with lots of COVID-19 around
you, personal distancing measures provide very little protection. (In other
words we cannot simply "isolate the old people" and expect it to work.)

In other words he is ignoring every widely known fact that undermines his
position. Which not coincidentally are the facts that lead everyone else to
the opposite conclusion from his.

Given that he can't fail to have seen many points like these, his leaving them
out of his analysis shows dishonesty on his part. And no, he is not attempting
to correct this fault.

------
cousin_it
> _We can’t take a close look at all of that chatter before deciding._

Yes you can. For example, if there's a lot of chatter in favor of reopening,
you could find a thorough summary of arguments in favor of reopening - much
like the article you're complaining about! - and do your best to evaluate it
on merits.

> _I don’t know if he’s evaluating the data correctly, or if he’s ignoring
> other relevant data that doesn’t support his conclusion._

Then you'll have the same problem with writings by experts. They have written
plenty of articles with basic mistakes, see for example the Santa Clara
serology study [1] and the takedown by Gelman [2]. Knowing a bit of math, I
rechecked the calculations and figured out that Gelman is correct, even though
he isn't an epidemiologist. If you use "context" and "motivations" and
"heuristics" as excuses to avoid evaluating the argument on merits, maybe you
should rethink what you're doing.

[1]
[https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v...](https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v1.full.pdf)

[2] [https://statmodeling.stat.columbia.edu/2020/04/19/fatal-
flaw...](https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaws-in-
stanford-study-of-coronavirus-prevalence/)

~~~
sorum
> [...] excuses to avoid evaluating the argument on merits, maybe you should
> rethink what you're doing.

I think his argument still holds: experts having made mistakes doesn't change
the fact that you are unlikely to be able to evaluate based on merits in the
first place. Unless you are an expert in the field, you're highly unlikely to
know enough about the other research out there which is valid, applicable and
could contradict the findings; you're not capable of proper evaluation.

------
dtech
> The author even wrote the whole thing using “we” instead of “I”, which
> subtly gives the impression that a group of people collaborated to write
> this

Note, this is standard in the academic world, even if there is a single
author. But it is a further way in which a phoney might mislead readers into
thinking it is a credible academic paper.

~~~
bargl
He used We because his paper is open for PR and he was hoping to get
contributions. He offered to change it and even put a disclaimer at the top.

[https://news.ycombinator.com/item?id=23025809](https://news.ycombinator.com/item?id=23025809)

If you'd like to submit a PR to the original paper you can totally do that and
it'll be open for everyone to see. I may not agree with the author but I don't
think attacking him is fair (on these points). Based on reading his comments
he seems open to discourse and suggestions on his paper.

~~~
ryanwaggoner
Note that he added the disclaimer later, after being called out [1] for making
the specious argument that adding a disclaimer would be "too verbose".

1\.
[https://news.ycombinator.com/item?id=23030215](https://news.ycombinator.com/item?id=23030215)

~~~
bargl
So he didn't think he needed a disclaimer, had a discussion around it, then
added a disclaimer. That's exactly what I want from another human being.
Someone who is open minded enough to have a discussion, opinionated enough to
back up what they think, but in the end plyable enough to make an important
change when they see it.

I might suggest you edit your post to include a nod to his disclaimer, as
you've shown now that you know about it but don't mention it in your article.

~~~
ryanwaggoner
I didn't include mention of the disclaimer because it wasn't in place when I
saw the article in question, and I didn't link to the article because I don't
think it deserves any more visibility.

Also, just looking at the comments on the original article, I don't think the
author is open-minded or pliable, but that's just my .02.

~~~
bargl
But now you know, you could easily add an edit and an "This wasn't originally
there but after being outted the author added it."

No I'm not saying link the paper, I was saying your HN link that's all.

------
spekcular
There are some legitimate complaints here, but the point about the paper being
well-written seems weird. Would the author prefer the discussion of such an
important topic be poorly organized and perfunctory? Also, using "we" is
standard scientific practice, not some bluff.

~~~
AllegedAlec
> Also, using "we" is standard scientific practice, not some bluff.

No. The standard practice is to use passive forms.

EDIT: I've been informed by the people responding that passive form is now
considered archaic, so apparently using first pronouns is now acceptable. This
must've changed since I did my research.

~~~
dekhn
This is.. a bit historical. While the passive voice was recommended in the
past, most journals and advisors recommend using active voice in papers now.

~~~
AllegedAlec
Thanks for the update. Back when I did my studies (less than a decade ago) I
was told, and all papers were also written, passively.

------
lordnacho
I think Ryan is mostly right. We can't give everyone a shot at everything,
there simply is no way to evaluate every claim from everyone. Imagine if we
had to read papers proposing to inject people with disinfectant.

Of course that does leave is with the authority of guilds in each area. They
get to define who is a member and who is not, and where the research money
goes within each area.

The major problems I find with this approach is that we end up letting the
experts decide things that they are not experts in, and we have to trust their
own policing.

They cannot within their own guild decide what tradeoffs society should make,
it necessarily requires more than one expert opinion, and there's no way to
say the concerns of one group must outweigh that of another. It's always a
political decision by someone who isn't an expert in every field.

As for their own policing, there are plenty of examples of the experts being
wrong. How a non-expert is gonna fix that I don't know, but not having
external oversight is a due diligence failure.

~~~
zzzcpan
It's not a real problem though. We can give everyone a shot at everything
simply because very few will take it and even fewer will be able to reach the
point where their claims have to be evaluated by others or someone in a
position of decision making. Having authorities on subjects can't do anything
good, it only allows to silence and ignore uncomfortable questions and
opinions.

------
pjc50
The problem is essentially spam filtering with much higher stakes. A priori
the source of an unusual paper might be one of:

\- a genuine innovation from a polymath outside the field or undiscovered
talent (Ramanujan)

\- someone outside the field applying a well-understood technique from their
own field in a new area (used to be common in bioinformatics back when it was
done with Perl)

\- someone in a non-first-world country much closer to the problem or with a
connection to traditional relevant knowledge

Then there are the ones which turn out to be wrong:

\- respected but crank-ish behaviour within the field: someone well respected
who is extremely enthsiastic about an idea beyond all evidence, such as Linus
Pauling's enthusiasm for vitamin C

\- respected but ideological behaviour within the field: e.g. the warring
schools of economics

\- novices who are bad at checking their work: students who believe they've
solved a famous conjecture but left out a minus sign on page 65. Most senior
academics deal with a lot of these routinely.

\- field outsiders getting cranky in another field: William Shockley's
opinions on biology

\- freelance not-for-profit cranks: outsiders who are wrong, but simply
because of error and not malice

\- for-profit cranks: this is where it starts getting genuinely dangerous, as
these people can be high-output and are aimed at the public. All manner of
quacks are in this category, such as the "miracle mineral solution" people who
have been trying to get people to inject bleach.

\- culture war cranks: The Alex Jones and David Ikes of the world. Even more
dangerous as they are not afraid to libel people and destroy those who cross
them.

It is no more possible to submit every paper you see to rigorous review and
replication than it is to do your spam filtering by sending money to every
Nigerian prince who asks for it and seeing which ones send it back. They will
destroy you because their capacity to waste your time and effort exceeds
yours. You have to go Bayesian; look for red flags that indicate that it falls
into one of those categories above.

------
vikramkr
Another thing that really matters - Peer review from other experts in the
field. We have mechanisms whose job it is to do what this article was talking
about, see if the claims made add up/if the references are interpreted
correctly etc. It's obviously not perfect, but it's a hell of a lot better
than no peer review

------
djohnston
If you only permit established domain experts to voice ideas you will severely
limit potential advancements. Innovation is often the product of multiple
disciplines intersecting to produce something novel.

~~~
nil-sec
Do you have an example from the last, say 20 years, where an actual amateur
made any significant scientific contribution to an established field? I have
strong doubts such a person exists. Simply because science, at the very least,
requires you to be aware of what has been done already. Of course there are
collaborations between researchers but this is a totally different point.

------
andybak
Here's the article referred to:
[https://news.ycombinator.com/item?id=23025044](https://news.ycombinator.com/item?id=23025044)

~~~
D13Fd
Interesting - not quite what I expected from the article.

I think that lines like "So, we feel that we must say that the right to
freedom of assembly (along with all of our other rights) are not luxuries that
are graciously extended to us by the ruling class" and statements about the
"WHO’s pattern of lying" make clear that this is a political opinion piece not
a scientific paper.

------
threwawasy1228
This is just a hollow pro-credential posturing without any substance to it.
What is the point here supposed to even be? That software engineers too dumb
to understand statistics and probability, and should just leave this to the
'experts' like the ones the original author was citing in order to make his
argument?

I don't understand the critique of tone in this either. Is the idea here
supposed to be "I disagree with what the author was saying, so he shouldn't
have written it well with a concise tone" ? That is what this comes off
sounding like the author here is saying. Like he thinks if your idea is wrong,
you aren't allowed to write well or something.

------
lalaithion
Maybe I'm missing something, but this person doesn't appear to be an
epistemologist or even any kind of trained philosopher. I'm not sure why he
thinks that any of us should listen to the advice about who we should listen
to the advice about covid-19 from someone who isn't an expert in a relevant
field.

------
madmax96
So the original paper is sophistry because:

>the author linked to many research papers by experts

>wove an argument together out of all that research

>The tone was authoritative and confident

>it’s fairly long and comprehensive

>it’s well-organized

War is Peace, Freedom is Slavery, and Ignorance is Strength. The paper is bad
because it is good.

The author of the original paper was more thoughtful than this one's. A good
post would have investigated the original paper's claims and responded to
them. This is a low-effort take.

------
gridlockd
_" For example, when evaluating claims in a high-stakes, high-uncertainty,
high-complexity area, like the public policy approach to COVID-19, a good
heuristic might be “does this person have expertise or experience in a
relevant field?"_

Ironically, the author of this blog post doesn't exhibit expertise or
experience in _any_ relevant field.

------
alexandercrohde
"No I won't read your amateur Special Relativity 'Research Paper', you dumb
patent clerk"

~~~
wool_gather
Not at all equivalent. Einstein had a relevant degree -- albeit "only" a
teaching degree -- and his _annus mirabilus_ papers were not self-published;
they were submitted to and accepted by a respected scientific journal.

------
beanerboi22
This post is essentially saying don't even try to comment in an intelligent
formal tone, citing sources, etc, on things you don't have a credential for. A
surprisingly large number of fields are just applied statistics/probability
with some minor domain knowledge tossed in. So people who are familiar with
statistical methods can comment on things like this very fast. Further, that
guy was explicitly citing people from the field to make his argument, not
making some outlandish claim without backing it up.

In this post there is literally nothing about the actual content of what the
other person was saying. There is nothing here about what citations they maybe
were using incorrectly or misunderstanding.

There is nothing here saying what statistical tools they might have been
accidentally abusing. There is nothing here! I'll speak for myself and say
I'll judge ideas on their own merits not some empty credentialist garbage.

------
spacefearing
Sounds like we need better heuristics. It doesn't make sense to ignore
research papers based on their origin.

Each paper _should_ be evaluated on its merits. A system with submission,
voting, and comments (HN/reddit) is a good way to filter the wheat from the
chaff. It's hard to believe this basic technology hasn't reached the
scientific fields.

~~~
ar0
They have. It's called "peer review". It is just that peer review is a slow
process, so it doesn't really work in a fast-moving crisis environment.

Part of peer review being a slow process may be fixable (although it is
difficult to compel unpaid, volunteering reviewers to adhere to strict
deadlines), but part of peer review being a slow process is specifically that
if you do not want to rely on heuristics such as author credentials you need
to take the time to really analyze a paper to avoid falling into the traps
mentioned in the original article (confirmation bias, being misled by an
authoritative tone etc.). And HN/Reddit votes IMHO are definitely not a good
example of avoiding any of these.

------
D13Fd
It would be helpful if this linked to the original article/paper to put it in
context.

~~~
zymhan
I disagree, this post applies to many, many discussions/dissections/etc about
the current pandemic.

The skepticism in general applies even more widely. The point is to make you
think harder next time you read an amateur post on COVID.

~~~
D13Fd
The post makes a number of claims that I disagree with generally, like the
idea that a person who is not a formal expert on a subject has no business
offering evidence-based opinions on the subject.

I think a link to the original paper would help me better judge where the
author is coming from and whether, in context, he/she is making good points
here.

------
ubercore
Why is this story flagged? I think it's as reasonable to be on HN as the
original.

------
FpUser
I am not going to defend that particular guy. I am surely not going to take
this kind of advice from generic Joe Doe.

But here is the example that comes out of Chief Public Health Officer of
Canada: "Canada's top doctor told CBC News the federal government could have
made earlier efforts to keep the COVID-19 pandemic from sweeping across the
country — but moves to close borders and screen travellers for the illness
sooner _might not have made much of a difference_."

This _might not have made much of a difference_ really got me. What if it did
make effin' difference. When this kind of argument is presented by top health
official with top credential I do not even know what to think. Ok the CBC
could've misquoted/misrepresented her point of view but then where is the
correction?

------
rriepe
I've noticed the appeals to authority have ramped up since the authorities
have been wrong over and over, again and again. At what point do we just stop
calling them authorities?

At this point, the software engineer is more believable because he has fewer
reasons to lie to you. The "expert" is going to be held up and crucified (in
the style of this blog post) if he goes against the accepted narrative. And
that's if he's not bought off right at the start.

This isn't the software engineer's credibility problem. It's yours.

~~~
btilly
Note that the reply was itself written by a software engineer as well. There
is no particular reason to believe one over the other.

As for "authorities", I distinguish between people who try to understand and
think well, versus those who have been recognized by bureaucracies and
politicians. The first group has done astoundingly well. The second group has
failed over and over again.

Sadly it is the second group that is in charge. And the result has been
repeated institutional failures. :-(

~~~
crusso
Yeah...
[https://www.jerrypournelle.com/reports/jerryp/iron.html](https://www.jerrypournelle.com/reports/jerryp/iron.html)

------
newen
Why is some random non-academic's uneducated opinion about academic paper
relevant, important, literally anything at all?

~~~
zymhan
That's not what this is. It's a post explaining why and how you should be
skeptical of authoritative-sounding blogs about COVID.

It is more important than ever that people be able to understand what are
reliable facts and what is merely opinion.

------
generalpass
Can't help but notice the author's title, _No, I won 't read your amateur
COVID-19 "research paper"_, which suggests it should be about COVID-19,
immediately begins by criticizing an unlinked "paper" that seems to be about
public policy impact from government actions.

~~~
zymhan
And?

------
xiphias2
,,the author is a software engineer, not economist or epidemologist''

Great! I have tried economist math test for university students, and it was a
joke. I wouldn't trust any economist being able to analyze data, as they don't
have the scientific background. I would invite any economist to try to solve
my computer science calculus university test in 45 minutes.

