
Against pandemic research exceptionalism - SiempreViernes
https://science.sciencemag.org/content/368/6490/476
======
roenxi
My normal position is fairly anti-government, but there is a role for an
agency here that I wish they'd fill. It'd be fantastic to have a centralised
agency organise a panel of experts who have gone though all the known papers
and studies, synthesised them then just sat for a week doing a massive public
Q&A session. Targeted at people who are really quite clever rather than the
usual aiming-low that the mass media goes for.

The volume of research has gotten far too much for me to keep track of; the
media are in full swing polluting the channel with irrelevant and frequently
political noise. I really do not care about what politicians think about given
drugs, and honestly their justifications for why they are doing X or Y are not
going to matter to me for a while because politics doesn't happen quickly.
It'd be nice to have a centralised, low bandwidth well moderated information
source for the smartest 5-10% of people to get good information in response to
their questions and uncertainties.

There are great information channels for PhD medicine types & doctors (I
hope). There are good sources for people who are at risk of drinking bleach
because they heard it might help (try a local news channel). There is a hole
for, say, the top 5% smartest people where they really don't have an in to
figure out how they might help, because they need information hidden in very
technical articles or behind rather black-box modelling as described in the
media. But they are otherwise very educated, intelligent, capable, wealthy or
what have you. These are coordination problems a government could actually
help with.

~~~
knzhou
> It'd be nice to have a centralised, low bandwidth well moderated information
> source for the smartest 5-10% of people to get good information in response
> to their questions and uncertainties.

Any such channel which is truly open to questions immediately gets swamped by
political attacks, along with an endless horde who think they're in the
smartest 5% yet accuse the scientists of being in the pocket of the lizard
people. Malicious agents will be able to mine even slightly nuanced
explanations for out of context quotes to show that their political opponents
are stupid or evil. If the channel is to be useful at all, it must make
statements under uncertainty, but that gives it the chance of being
occasionally wrong -- and anything it ever gets wrong will be repeated ad
nauseum as an argument to defund it. If the channel is visible and trusted, it
also becomes a prime target for politicians looking to shift blame away from
themselves.

The reason I'm completely sure this will happen is that the information
channel you're asking for _already exists_. It's called the World Health
Organization, and they've been doing informative press conferences this whole
time. However, everybody on this site has decided that they're useless because
of a single, incorrectly read, out-of-context tweet from 4 months ago.

~~~
1996
> It's called the World Health Organization

That's a political organization, not what is being proposed here.

~~~
knzhou
You’re perfectly illustrating what I talked about.

~~~
leereeves
But the objection is perfectly valid. They are a political organization that
sometimes places politics over reality (for example, refusing to recognize
Taiwan, repeating Chinese reports unchallenged and praising China's
transparency, or as another comment pointed out, appointing Robert Mugabe as
goodwill ambassador).

That internal challenge should be added to your excellent list of external
challenges the proposed organization would face, and the WHO does face:
politically motivated people and powerful nations will capture the
organization from the inside and abuse any trust it has built up to serve
their own ends.

~~~
ShorsHammer
> refusing to recognize Taiwan

So it wouldn't be political if they did recognise Taiwan? No one on Earth
besides a few Caribbean islands recognises the ROC and yet somehow the WHO has
no credibility for doing the same.

Very much making the GP's point for them without even realising here.

Mugagbe is a perfect illustration of that. They were trying to tackle non-
communicable diseases in Zimbabwe, getting him on board and giving him a
meaningless title was their plan. Instead of seeing it as a boon for the poor
in the country, people decided to make it political and demanded he not get
the worthless honor. Either outcome had absolutely zero effect on the people
screaming the loudest about it.

Put the politics aside for a moment and ask yourself the question: Would
health outcomes for the poorest Zimbabweans be better or worse if Mugabe had
kept the title?

~~~
leereeves
The fact is, the Republic of China exists and is independent of the People's
Republic of China. They have a military and a government that doesn't answer
to Beijing, and have maintained that government in Taipei for 70 years.

That's reality.

Recognizing reality need not be political, but refusing to recognize reality
for political reasons definitely is.

But the main point is that politics, not medicine or science, is dictating the
WHO's decision here. Would health outcomes for the Taiwanese be better if the
WHO included Taiwan?

As for Mugabe, you're arguing that it was _good_ politics by some measure, and
it may very well have been, but it also erodes faith in the organization from
people who don't think it was good politics.

> Would health outcomes for the poorest Zimbabweans be better or worse if
> Mugabe had kept the title?

If we assume that outcomes would have been better, that implies that listening
to the WHO does improve health, and therefore we should also assume that other
people who would have lost faith in the organization would have had worse
health outcomes as a result. I don't know how it would all balance out, but
again, the point is merely that it's a political decision, not a scientific
one.

~~~
ShorsHammer
> Recognizing reality need not be political, but refusing to recognize reality
> for political reasons definitely is.

Getting China offside means 1 in 5 people on Earth would no longer get WHO
advice and state media would undermine them at every turn.

Getting Taiwan on side means appeasing political activists in the West and
having little if any measurable positive health outcomes, the Taiwanese are
used to this state of affairs and have more than competent doctors and
lawmakers, they are one of the smartest countries on Earth.

Despite the closeness and affection between my own country and them, when
their foreign minister visits they don't go near parliament or meet elected
officials in public, they go to trade shows and meet with industry.

As much as it might pain some to accept, compromises are the best outcome for
all here, the WHO is being realistic about the position they are in. Hopefully
more sideline commentators can be too.

~~~
leereeves
> Getting China offside means 1 in 5 people on Earth would no longer get WHO
> advice and state media would undermine them at every turn.

Basically you're saying the same thing I'm saying. Authoritarian nations are
allowed to dictate WHO policy in order to keep them "onside".

You're just arguing that it's necessary, and I'm inclined to agree, but it's
also the reason the WHO is not an impartial or trustworthy source of
information. And it's probably impossible for any such organization to be so
at this time.

------
ISL
As someone who was arguing for speedy reporting a few weeks ago: the longer
the pandemic goes on, the more important rigor becomes.

Waiting a short time to get things right today is a a very different matter
than a 4-8 weeks ago, where on-the-ground information was completely absent.
Speed still _really_ matters, but the longer this goes the more taking our
time and doing things right will pay off.

~~~
SiempreViernes
Are you arguing that in the initial stages _incorrect_ information is less
harmful than no information?

~~~
bo1024
There’s a difference between incorrect and incomplete.

Facts, like how many people in a trial of 23 recovered, can be shared and
useful early on even though they’re incomplete. That’s different from
attempting to draw a strong conclusion about what works from n=23, that might
be an incorrect conclusion.

~~~
mattkrause
The _form_ that the incomplete data took was part of the problem.

Adding numbers to a spreadsheet is one thing. When the same data takes the
form of an NJEM or Lancet paper, it definitely implies--and some cases,
outright says--more than "FYI: 37/53 people got better when we did [this]."
Part of this is how papers are ordinarily written, but I think the authors
(and editors) should have stopped all over the word "improvement" in those
manuscripts.

------
dahart
> Numerous trials investigating similar hypotheses risk duplication of effort,
> and droves of research papers have been rushed to preprint servers,
> essentially outsourcing peer review to practicing physicians and
> journalists.

I’m curious about this point. Often we have the opposite problem, that we
can’t get anyone to replicate studies with strong looking but potentially
specious results. Isn’t perhaps a small amount of duplication of effort a good
thing, especially if studies are rushing and letting standards slip a little?
Aside from coordinating and not using too much of the funding to investigate a
particular hypothesis, what are the other reasons to avoid duplication of
effort?

~~~
SiempreViernes
Their point is that there is actually a _very large_ amount of duplicate
effort is happening now, as everyone _separately_ goes for the first ideas:
this makes it more probable that a lot of effort will be wasted on super-
duper-triple confirming that the first thing everyone thought of didn't work,
slowing the exploration of more nuanced ideas.

But besides waste of time and resources, the only other problem they mention
is an increase of false positives. These naturally become more common as you
try the same thing more often unless you correct for it by keeping track of
all the other trials.

~~~
gus_massa
My problem is that there are too many inconclusive trial. Someone makes a
small trial without a control group, so the result is promising but difficult
to interpret. Someone else makes a small trial without a control group, so the
result is promising but difficult to interpret. Another person makes a small
test without a control group, so the result is promising but difficult to
interpret. ...

So after a few months, we only have promising inconclusive results.

Not that all the trials are different, so it is difficult to compare them.
IIRC There was a recent trial with remdesivir without good results, but they
only tried the drug in patients with strong symptoms. So people asked what
happens if they use the drug in the beginning of the infection.

------
kqr2
Not only research, but a lot of diagnostic companies are trying to take
advantage of the _gold rush_ :

 _Theranos Would Be Thriving in the Covid-19 Pandemic_

[https://elemental.medium.com/theranos-would-be-thriving-
in-t...](https://elemental.medium.com/theranos-would-be-thriving-in-the-
covid-19-pandemic-63ec6d45c96a)

------
bo1024
Really interesting question here. I wish it went a bit more into specifics of
how to navigate the tradeoffs. Maybe one way to put this is that it’s fine to
do preliminary studies that motivate and direct the actual rigorous research,
but we’re stuck in a place where people are only doing the preliminary
studies, a lot, and not getting to the rigorous stage.

------
hartator
> perpetuate the perception that, when it comes to the rigors of science,
> crisis situations demand exceptions to high standards for quality.

Where are the high quality studies warranting strict lockdowns?

~~~
pnako
In fact I would be curious if a single medical textbook anywhere in the world
mentions this as a possible solution to any epidemic. It seems to be a
innovation, coming from politicians more than scientists.

The practice so far, still uncommon, was to quarantine sick people, not force
the entire population to stay at home.

------
pmiller2
What happened to the title? The actual title is " _Against_ Pandemic Research
Exceptionalism". If this was intentional, it was a bad choice, because
information was lost.

------
dia80
I had wondered if a bandit algorithm would be an effective approach. Can
anyone who knows more about it than me comment?

~~~
gus_massa
The name for an "A/B test" is "randomized control trial" (with two arms).
There are also trials with more arms, like

* standard treatment + placebo

* standard treatment + some drug

* standard treatment + double amount of drug

and other combinations, like using two drugs in one arm.

~~~
marcosdumay
Bandit algorithms aren't different only because they have multiple arms. They
also follow the least possible amount of harm by updating the conclusions in a
Bayesian style during the entire experiment, not only on the end.

They also can stand more arms appearing during the experiment after people
know more.

