
My Institutional Review Board Nightmare - apsec112
http://slatestarcodex.com/2017/08/29/my-irb-nightmare/
======
ak_yo
The author doesn't seem to take the history of medical research ethics (e.g.
the Tuskegee syphilis experiments [1]) seriously. Having a sophisticated
understanding of the ethical risks of research is a huge part of doing good
science, and this post certainly doesn't communicate a high level of
sophistication. If the author thinks that ethics review is "Blindly trusting
authority to make our ethical decisions for us", then he clearly doesn't
understand what ethics review is or how it works.

IRBs have well-documented flaws (for a more comprehensive look, read Laura
Stark's book on IRBs [2]) but "IRBs are bad because they say I can't do
whatever I want even though it's obviously the right way to do it" is not one
of them.

[1]:
[https://en.wikipedia.org/wiki/Tuskegee_syphilis_experiment](https://en.wikipedia.org/wiki/Tuskegee_syphilis_experiment)

[2]:
[http://press.uchicago.edu/ucp/books/book/chicago/B/bo1218257...](http://press.uchicago.edu/ucp/books/book/chicago/B/bo12182576.html)

~~~
kinkrtyavimoodh
While I agree that the author is being a little facetious (with the constant
Nazi references and such), I don't think he is arguing in favor of removing
ethical considerations in research, or even arbitrarily weakening the
constraints put on scientific studies.

But what purpose is having a review board made up of people who can think for
themselves if their behavior is going to be indistinguishable from a program
running through a checklist without any care whatsoever for whether or not the
checklist makes sense in that context?

Massive inefficiencies in bureaucracy are always hidden behind the fig leaf of
consistency and accountability, but it's worthwhile to consider what aim they
serve and at what cost.

~~~
icelancer
>>Massive inefficiencies in bureaucracy are always hidden behind the fig leaf
of consistency and accountability, but it's worthwhile to consider what aim
they serve and at what cost.

This problem was largely solved by the development and acceptance of private
IRBs. Going with private IRBs that meet 2x weekly, turn forms around quickly,
and are willing to pre-qualify your work - at cost, of course - is the road
many researchers take now.

~~~
Sir_Substance
How is that solving the problem???

~~~
icelancer
....how does it not?

If you are asking the question: "How does this private market solution solve
the problem of government/education/medical bureaucracy being inefficient,"
then I guess it does not solve the root cause. But trying to solve IRB
problems involving that three-headed hydra is not likely to succeed. Going
around the system is a feasible workaround.

~~~
jessriedel
Implementing a terrible process faster at great cost is not the same thing as
solving the problem, nor is it the best possible solution short of "fix all
bureaucracy".

~~~
icelancer
I wouldn't call my experiences in private IRB to be terrible at all, honestly.

~~~
jessriedel
My comment does not conflict with that because by "process" I mean the
mechanism purporting to increase subject safety, not your experience.

~~~
upvotinglurker
What basis is there for the assumption that private IRBs have a "terrible
process" and do a worse job at protecting subject safety?

------
88e282102ae2e5b
The initial meeting with the IRB where they proposed ridiculous and irrational
changes to the consent forms was strikingly reminiscent of every conversation
I've had in a third world country where someone was asking me for a bribe, but
couldn't say it directly.

~~~
gwern
I've noticed how often big projects will allocate something like 5% of their
budget for a few things like 'bioethicists' and 'community' grants; you never
seem to hear about what they produce or what important findings they make, but
the money does get spent...

~~~
sorokod
Money always gets spent. It is in its nature.

------
schoen
I thought I remembered a proposal to create some new categorical exemptions
from IRB review, one of which was somehow related to studies where you just
talk to people. (I've been interviewed by several social scientists and very
often their "protocols", consisting exclusively of informal interviews, had
clearly been through extensive IRB reviews, which felt super-odd because
journalists wouldn't need any ethical review at all in order to carry out
exactly the same interview in exactly the same way.)

On the other hand, you clearly _can_ harm people in various ways by or as a
result of interviewing them, for example by breaching confidentiality, by
being forced to breach confidentiality, or by making them feel bad about
themselves by insulting them during the interview or just by bringing up
painful and distressing memories. And also it's hard for researchers who
aren't doctors to extend the same level of legal protection to what their
research subjects tell them:

[https://www.socialsciencespace.com/2017/01/social-science-
ne...](https://www.socialsciencespace.com/2017/01/social-science-needs-doctor-
patient-confidentiality/)

So maybe one idea would be to create a categorical IRB exemption for studies
that combine non-invasive observations, or existing datasets, with interviews
not reasonably expected to be distressing to the subjects, and conducted
subject to otherwise existing norms on patient confidentiality and medical
privacy? (recognizing that limited legal protection for that confidentiality
still poses problems for some studies)

(I'm certainly happy to apply the Chesterton's fence principle and learn more
about the history of unethical experimentation before seriously advocating
this.)

~~~
munin
> which felt super-odd because journalists wouldn't need any ethical review at
> all in order to carry out exactly the same interview in exactly the same
> way.

Journalists don't have a history of injecting you with syphilis just to see
what happens, though.

~~~
occam
Who has ever done that?

~~~
schoen
As mentioned elsewhere in this thread,
[https://en.wikipedia.org/wiki/Tuskegee_syphilis_experiment](https://en.wikipedia.org/wiki/Tuskegee_syphilis_experiment)
is one of the most important parts of the history of IRBs.

~~~
occam
Tuskegee syphilis experiment, ok. However, as bad as that was, no one was
injected with syphilis.

~~~
schoen
I thought I remembered that the Public Health Service intentionally infected
people in order to see what would happen, but Wikipedia seems to say that they
intentionally withheld treatment from people who were already infected and
deceived them about this, rather than causing the infections themselves
(although some of the study subjects spread the disease to other people during
the course of the study, which were new infections that could have been
prevented by treatment).

~~~
mjevans
I believe a modern design for this study would compare different kinds of
treatment (but not the elimination of treatment... yikes).

~~~
goialoq
Modern policy for medical research is that everyone gets a best available
known treatment, and some people also get an experimental unknown treatment.

~~~
incompatible
There must be cases where this isn't practical. E.g., if the normal treatment
for greyscale is to cut off the arm before it spreads, how could you try any
alternative treatment?

~~~
YawningAngel
It isn't, and the answer is "you can't get good data". A good example would be
immunotherapy for cancers, which has to be tested alongside treatments with
adverse impact on immune function

------
xr4ti
I can't agree. The authors wounds at every stage in the narrative were self
inflicted.

(1) The study title doesn't work at the top of the form? Change it! Surely
there is some other arrangement of English words that would satisfy both the
author and IRB.

(2) They leave the risk section blank because minimal risk, and then complain
about the requirement for pens because the patients might stab themselves if
given pens. Looks to me like they didn't spend much effort thinking about
risks to the patient. If they had just mentioned the pen stabby thing as a
risk factor, and given "don't use pens" as a risk minimization plan, they
wouldn't have had a problem.

(3) They poopoo the training because Hitler, and then get upset when they
unknowingly violate a personnel rule that is most definitely covered in any
decent IRB/GCP training.

I could go on, but every new thing I enumerate makes me angrier. Life is too
short. Look. IRB applications suck, but human experimentation is serious
business.

The rules aren't there to make the study easy, the are there to (a) look out
for the patient (respect), (b) evaluate risk and benefits of the study
(beneficence), and (c) ensure that the benefits are distributed to the group
bearing the risk (justice).

~~~
sevenfive
Yeah, the author was not quite as bureaucracy-savvy as you, it's clear. You do
realize that the whole study was nothing but _matching the results of a
questionnaire that was already being used against interviews that were already
being done?_. Literally, all he's doing is observing. And maybe asking a few
extra people to complete the questionnaire, the wording isn't clear.

~~~
xr4ti
Said this in another comment, but I'll reiterate here. The big difference is
purpose/intent. Looking under the hood of someone's car can be totally fine or
a violation depending on whether or not I'm being paid to fix it.

Same thing applies here. In one case the questions are being asked to provide
treatment the patient is paying for. In the other it's being done to collect
data for a publication.

~~~
aidenn0
Bullshit.

If my mechanic wants to publish a paper on wear-and-tear on 4 cylinder engines
using just the information he gleaned from doing work on cars that I (and
others) paid him for, he has caused zero damage to me versus just doing the
work on my car.

The same applies here. Patients come in for treatment, and they provide
exactly the same questions as if they weren't doing research.

~~~
euyyn
Private information about my mental health is much more personal and sensitive
than the wear of my car's engine. Handling that data to an untrained student
intent on publishing a paper can cause plenty of harm.

I agree, though, that this study could have been done more easily by accessing
the patients' records post-facto. But it's not the IRB's fault that the
experiment could have been designed better.

~~~
aidenn0
Yes, mental health information is personal and sensitive, but it looks like
the IRB didn't even address patient privacy, it was instead something that
came out of a later audit.

[edit]

Also it seems that post-facto experiments are being encouraged over controlled
trials, which seems the opposite of what you want from an experimenters point
of view.

~~~
euyyn
More privacy protections were enforced later according to the account, but the
very first protection of them all is asking for consent to use the data.
That's what I was getting to. Consent to use your responses to treat you is
different from consent to use them for research.

------
aidenn0
IRBs for psychology really do seem needlessly bureaucratic. This is a case
where a controlled experiment is not allowed, but the corresponding natural
experiment would be (They would likely have gotten quick approval to
investigate long-term outcomes of patients who had charted bipolar on this
test). This is not the only time that I've heard of professional psychologists
not being allowed to do something considered mundane and ethical as a course
of their job _if_ it were part of an experiment.

Yes I am aware of trauma caused to people in not just Nazi experiments but in
many post-war studies (many of which were of dubious value). It seems like at
some point we should decide one of the following:

A) Psychology has the potential to significantly alleviate suffering, so it's
worth a risk of small harms in order to collect good data.

B) Psychology doesn't have the potential to significantly alleviate suffering,
so we shouldn't do any of these studies.

------
hirsin
My freshman year of college I proposed a study to our hospitals IRB to strap
small lasers to three week old infants in an effort to measure concentrations
of a chemical in their blood. The most frustrating part was not the arcane
insistence on ink and bolded study names, but the hardline insistence that it
was impossible (illegal) to test the device before getting IRB approval - even
on ourselves. Meaning that without any calibration or testing, our initial
study would likely come back with poor results or be a dud, but we couldn't
find out until we filled out all the paperwork.

Incidentally, I recall it being much less painful than this, at least to get
to the approval step. I would expect that each IRB varies greatly from group
to group.

~~~
jpatokal
What stopped you from testing on yourselves anyway outside the study?

~~~
hirsin
Our PI was perhaps stricter than was useful. I believe it was intended as a
teaching tool to force us to deal with the irb

------
yk
Well my story with the research bureaucracy is, a coworker had to fill out a
notice that he got some money for a conference from outside the university and
he dutifully filled out everything. Thing is, as a theoretical physicist you
are supposed to fill out the first page, check the box on page 27, that there
is not conflict of interest, and sign the thing. Any answer on certain
questions will trigger a public hearing by some very obscure board.

Public hearing means, that poor guy has to go there in case the public has
questions. He filled out everything, so he had to go to the anti-corruption
board and the nuclear safety commission and so on. After roughly three weeks
of him whining that he always sits there like an idiot, I got curious and came
along to one of the hearings, definitely for some reason other than watching
him sit there like an idiot, or so I claim.

That was the animal cruelty (against) commission and at the hearing were the
committee, a very nervous working group, one unhappy postdoc and a grad
student who just wants to watch the show. The PI of the working group then
gave a presentation on the sad reality, that you can't study pain receptors if
you anesthetize the mouse. (Apparently that was not the first time they were
there, so I got just the part on why they are certain that there is really no
way to get a statistical meaningful result with less mice.)

Really drove home the point that these procedures are not designed for the
convenience of theoretical physicist, but are designed to tackle really hard
problems.

~~~
hawkice
If the ethical problems of rat suffering are really hard problems, why is it
utterly mundane for people to buy products at home depot to deliberately
poison them for days and days until they succumb to internal bleeding, for no
other reason then "it was bothering me, existing in my house"?

Perhaps we should or could start caring, but we definitely do not even
acknowledge there being a meaningful question there in any other element of
human life.

~~~
munin
Fortunately for us, the IRB only governs human experimentation. You need very
little approval to experiment on animals, in the US at least.

~~~
euyyn
What a medical professor at MIT HST told me is exactly the opposite, at least
for "higher order" mammals like pigs. The checks are more strict because the
animal can't possibly consent to you doing the experiment on it.

~~~
munin
I dunno what to tell you, the Belmont report was about humans. It's right
there in the name.

Animal research is often regulated under a framework, just not the IRB. My
training was pretty clear on that.

~~~
euyyn
I mean, he didn't say it was handled by the IRB. He just told me it was more
strict. Maybe that was specific to MIT.

~~~
munin
Interesting. At the institutions I'm familiar with, you can do all kinds of
unspeakable crap to animals and no one cares unless PETA gets wind of it.

~~~
euyyn
If memory serves me well, a friend of mine there killed mutant zebrafish
larvae, after being done with them, with bleach (as the approved protocol).
And another friend at Harvard cut the heads off lab rats with scissors like
Heidi plucking flowers.

My understanding is the less you look like a human in the grand scheme of
things, the more unspeakable crap we're comfortable with doing to you.

------
terminado

      “YOU LET PEOPLE SIGN CONSENT FORMS IN 
       PENCIL, HOW CAN YOU JUSTIFY THAT?!”
    

The funny thing is, contracts are legally binding agreements, whether they are
verbal, scribbled on a napkin, signed in pencil, pen or blood, and whether the
signature is an easily forged ' _X_ ' or elaborate calligraphy. Consent forms
are a slightly different use-case, but I think the premise is the same: to
show a meeting of the minds.

Knowing this, the pencil signature actually protects the patient more than the
institution, since the patient could theoretically forge a revocation of
consent by erasing their mark. Meanwhile, ink isn't as binding, without some
sort of objective third-party official (e.g. a notary public, beholden neither
to patient nor to institution) serving as formal witness, further validating a
non-forgery, in the case of a simple ' _X_ ' or erasure thereof.

In practical terms, one solution to render the pencil signature binding would
be to create a tamper-evident photocopy, with fraud-marks like a UUID moire
pattern sprayed by an ordinary desktop printer, for each serialized document,
readily detectable in the photocopy but permanently printed into the
background of the form (e.g. yes, this is a photocopy of the same piece of
paper, and the forger failed to notice the lightly tinted pattern, when
erasing their signature), and retain the pencil copy, to ensure a match. In
the event of an erasure, the paper might show an abrasion, and the photocopy
would act as a snapshot/backup.

Meanwhile, it should be possible to access existing, archived, historical
questionnaires and actual clinical diagnoses from normal records, in a de-
identified fashion, and compare results, no?

Am I misunderstanding the goal of the study? Didn't he want to compare the
assessment of the survey, with actual patient dispositions, and leave the rest
of all treatment untouched, undisturbed and otherwise normal? Ask the
questions, do the rest, same as ever, but analyze the opinion formed by the
score of the answers to the questions, with the opinion of the professional
caregiver beyond the scope of the simple Q&A exam?

------
paulsutter
Here's a story about sidestepping IRB processes by going offshore

[http://khn.org/news/offshore-rush-for-herpes-vaccine-
roils-d...](http://khn.org/news/offshore-rush-for-herpes-vaccine-roils-debate-
over-u-s-safety-rules/)

"Neither the Food and Drug Administration nor a safety panel known as an
institutional review board, or an IRB, monitored the testing of a vaccine its
creators say prevents herpes outbreaks. Most of the 20 participants were
Americans with herpes who were flown to the island several times to be
vaccinated, according to Rational Vaccines, the company that oversaw the
trial."

~~~
tim333
Discussed on HN yesterday
[https://news.ycombinator.com/item?id=15126557](https://news.ycombinator.com/item?id=15126557)

As the people doing that one argue, there is an argument for balancing the
risks of the test against the risk to suffers of illness and the like of not
doing the tests.

------
alexandercrohde
For those who think the author is in the wrong

1 - Do you think this story indicates a success or a failure of the IRB?

2 - Would you rather a better expedited review process AND more studies
(remember the personal effort cost killed this study), neither, or do you see
some 3rd option?

~~~
euyyn
1\. It's a success of his IRB. Of the many requirements he described, both by
his IRB and the auditors, he didn't make the effort to try to understand how
any of them could be good for his subjects. Instead he focused on how they
affected his study, and tried to find the bare minimum he had to do to get
approved. If the result of it is he doesn't want to deal with IRBs anymore, it
might be for the best.

2\. I believe this study could have been done with much less personal effort.
His hypothesis is the screening question leads to unwarranted diagnosis. He
already had Dr. W. convinced that it's plausible, and willing to do full
diagnosis to his patients. And he had other doctors that would only use the
screening question. All that's required to refute the null hypothesis is
compare the rate of "yes" vs "no" in the set of Dr. W's diagnoses, with the
rate of the doctors that just use the screening question. If his hypothesis is
true, the other doctors would get more positives. That's 1 bit of information
he had to extract anonymously from the patients' existing records, which is a
strict subset of the methodology he developed for the study.

~~~
skybrian
If you don't test both kinds of diagnostics on the same patients, that doesn't
sound the same at all?

As another example, if you have a ten-question survey, asking 1000 people one
question is not the same as asking 100 people all ten questions. You don't
learn nearly as much about the correlation between answers to different
questions.

~~~
euyyn
Assume the patients are assigned to the doctors in a way that's not affected
by whether they have the condition or not (e.g. at random). If the screening
question doesn't lead to over-diagnosis, then you'd expect the same ratio of
positives/negatives from both sets.

You need more subjects to attain the same statistical power, but the data is
much easier to obtain. 100 subjects already sounded like overkill anyway.

~~~
YawningAngel
It doesn't strictly pertain to his hypothesis, but presumably you would like
data on false positives and negatives as well as overall outcome percentages.

~~~
euyyn
Granted. But to get that you need to jump many more hurdles (which are there
for good reasons).

------
omegaworks
A take from the opposite end of the spectrum (ethically and logistically):

[https://arstechnica.com/science/2017/08/bucking-fda-peter-
th...](https://arstechnica.com/science/2017/08/bucking-fda-peter-thiel-funds-
patently-unethical-herpes-vaccine-trial/)

I wish the author took more seriously the warnings around patient health and
safety. Yes, they may not apply to his particular project, but that doesn't
make asking about them useless. To take past abuses lightly undermines his
overall point.

It's done precisely to avoid a situation like the way he delegated
responsibility to his PI: "Dr. W had vaguely hoped that I was taking care of
it. I had vaguely hoped that Dr. W was taking care of it." The review board
shouldn't make vague assumptions about something as critical and necessary as
ethical study design.

~~~
ScottAlexander
I take the possibility of serious ethical violations...well, seriously.

I'm not sure whether the global cost of IRBs outweighs the benefits. But I'm
pretty sure that the sort of regulatory changes OHRP proposed (mentioned in
the linked NYT article), where studies like mine with minimal possible risk
are exempted from IRB requirements, seem good and important.

I also wonder whether it's possible to monitor even the scary studies with
lots of potential risks - like the herpes vaccine trial you link to - with a
little more common sense and a little bit less insistence on everything being
signed in pen.

~~~
zaroth
BTW, welcome [back] to HN. And just wanted to say, I am immensely enjoying
very many of your articles and glad to see them making the front page!

------
dunkelheit
The problem with specialized committees that review and approve something is
that they have absolutely no incentive to allow the new thing to go forward
and every incentive to drag it into the ground with requests for more and more
changes and precautions.

Well, except when they are paid per review in which case they just rubber-
stamp everything.

------
paultopia
It's kinda amazing any research gets done at all, with all the bureaucracy.

My university requires official approval and a full process even to apply for
a small non-institutional grant. I was going to apply for that tiny little AI
Grant that was posted on here a while back, then I concluded that the rules
probably would require me to get approval to apply for that. So I said "fuck
it."

------
mjevans
I suspect that the proper ethical thing to do depends on the risks to the
patients, and the benefits to the subjects and overall population.

In my own opinion the author is quite correct in classifying the medical
benefits to the subjects (practically speaking, zero; equivalent to talking
with any random untrained individual), and to future patients (potentially
great benefits).

The risks seem to be much more closely associated with accidentally exposing
patient information. This is the section that their study protocols (at least
what was described in overview) need review and revision.

The unblinding documentation should be kept in a secured, designated area. A
safe with a very limited ACL would be ideal. Unblinding information could be
either the consent forms with the unique non-PII ID information on them or a
list for matching unique non-PII IDs to (hard copies of) existing patient
record stubs attached to the consent forms.

If possible the study should have actually been designed to use as much of the
already collected data as possible, and possibly ask the small additional
information of all, but not record the answers (or destroy the collected
information if they do not consent). Hypothetically they could be asked if
they consent anonymously providing some of the questions they were already
asked to an informational medical research study. If you ask them for consent
AFTER having asked (and gotten answers on) the questions, then you're even
actually safe to tell them which data are being collected (what their answers
were as well) and what you're attempting to study.

I believe that the above is the most ethical way of designing this type of
study, and that it sounds similar to a term I've heard professionals in this
field use 'retrospective study' (collecting study data based on previously
collected medical data).

~~~
harryh
Why should study data subject to higher security standards than information
that is collected on the patient during the process of providing care that is
much more sensitive?

~~~
mbreese
All PHI is subject to the same data security standards, regardless of if it is
from research or clinical sources.

~~~
harryh
That's not what the blog post linked here says.

~~~
mbreese
I didn't see anyplace in the blog post where they say how the data in the
chart (that they couldn't use) was stored. Did I miss that part? Are we
talking about the online chart that they used?

Just because they couldn't use the online (clinical) chart to store research
data doesn't mean that the data that backs the clinical chart application is
stored in any less secure way. There are very specific methods that are used
to store PHI electronically. The fact that the clinicians (who needed access)
could access it doesn't mean it wasn't stored securely -- they were supposed
to have access to it.

In all clinical research there is a clear separation between clinical (as in,
used for treatment) and research (as in, should never be used for treatment)
data. When you try to hit that gray in-between area is when you start to have
issues with IRBs. In particular, when you are both the treating physician and
the researcher, you have to be particularly careful about keeping the two
sides separate. There is no way anyone should be allowed to store research
data in the clinical record -- even if the researchers otherwise have access
to the clinical chart.

Also -- it should be mentioned that not all study data is automatically PHI.
In fact, most of it shouldn't be. Once it's been de-identified, then the data
isn't PHI any more and can be more freely used for research. This is why there
is so much emphasis on the security for storing the "unblinding"
documentation.

------
epmaybe
I'm in the process of getting two IRBs approved. I am somewhere in the first
quarter of that author's story. I agree that it's a big hurdle to get past,
and writing a good consent form is hard.

All of this being said, though, it was/is a great experience to have under my
belt. And, next time, I'll be able to tackle the regulatory hurdles with far
less confusion.

------
icelancer
Much of this reminds me of what I have been going through. What SSC should
have investigated and wrote up is that hospital and medical IRB is fraught
with bureaucracy and that a private IRB industry has developed, entities
inside that handle wide ranges of studies and turn around informed consent
docs and things very fast (within a week).

Our first IC forms got banged multiple times for the same reasons his did. And
it’s annoying as shit. But my ethical training focused not on something stupid
like Nazis, but something real like American scientists unethically hiding
treatment from diseases they were studying, like the Tuskegee Syphilis
Experiment (the actual reason IRB exists as is). Or the Milgram Experiment.
Etc.

A lot of IRB review is stupid. But messing with human subjects doesn’t have a
good track record in this country. Our predecessors earned every bit of
bureaucracy and oversight based on their absurd experiments. Such is life.

~~~
SilasX
That sounds, to me, eerily close to the mentalities that Alexander rightly
criticizes:

\- "It's no big deal because you can just pay off someone's who's in the know
and can work the system rather than read the rules at face value."

\- "No amount of kafkaesque absurdity is too much, because Nazis."

~~~
icelancer
What specifically sounds like that? I cannot decipher the context you are
using those quotes in regarding my comment.

~~~
SilasX
For the first one:

>"What SSC should have investigated and wrote up is that hospital and medical
IRB is fraught with bureaucracy and that a private IRB industry has developed,
entities inside that handle wide ranges of studies and turn around informed
consent docs and things very fast (within a week).

That is, even though Alexander had a long wait time just to hear responses,
there are expensive services you can use that make all that wait time and
pushback from hidden rules disappear, so you don't see it as a problem.

For the second one:

>A lot of IRB review is stupid. But messing with human subjects doesn’t have a
good track record in this country. Our predecessors earned every bit of
bureaucracy and oversight based on their absurd experiments. Such is life.

You don't show a standard for the upper bound on when the absurdity is too
much, dismissing any such concerns with appeal to some historical atrocities.

~~~
icelancer
>>That is, even though Alexander had a long wait time just to hear responses,
there are expensive services you can use that make all that wait time and
pushback from hidden rules disappear, so you don't see it as a problem.

You can make the wait time disappear. In my experience, you cannot make the
pushback disappear. I've been scolded multiple times by our IRB for improper
and too-technical language in our Informed Consent forms, amongst other
violations/gaffes.

IRB is poorly-designed. Should it go away? Sure, in its current incarnation,
yes. Will it? Like anything government + university designed, there is a
snowball's chance in hell that anything with that many moving parts and
bureaucracy will ever die a true death.

The private markets designed a workaround for this and they are accepted as
every bit as good as the university and medical IRBs.

As far as "expensive," your time is valuable if you are a scientist. You
should act accordingly. The IRB fees we pay are less than 7% of the fee it
costs to publish in major peer-reviewed journals, which is another racket that
is propped up by the government + university complex. So I don't consider it
expensive in the least. (It is well under $500, to give you a hard number.)

~~~
SilasX
>You can make the wait time disappear. ... IRB is poorly-designed. Should it
go away? Sure, in its current incarnation, yes.

So, you agree with my points then?

>As far as "expensive," your time is valuable if you are a scientist. You
should act accordingly.

I didn't say otherwise, and that's not relevant to Alexander's claim that the
many of the standards are excessive and not merited.

------
Nition
Someone on Reddit says[1]:

> Amateur. What you do is you sweet talk the clinicians into using their
> medical judgement to adopt the form as part of their routine clinical
> practice and get them to include it as part of the patient's medical
> records. Later... you approach the IRB for a retrospective chart review
> study and get blessed with waived consent. Bonus: very likely to also get
> expedited review.

[1]
[https://www.reddit.com/r/slatestarcodex/comments/6wtylk/my_i...](https://www.reddit.com/r/slatestarcodex/comments/6wtylk/my_irb_nightmare/dmas5l5/)

~~~
j88439h84
What form?

~~~
haltingthoughts
The author was trying to test whether a simple common test for bipolar was
accurate. Presumably, the suggestion is to make the test a form filled out by
each doctor for each patient. Then months or years later the author could do a
chart review and see whether the test results were accurate.

~~~
nohat
To potentially clarify: the test is just a form. It's a form with some
questions about mood, and is currently being used to diagnose bipolar disorder
(inappropriately, the author believes).

------
smithkl42
Commenters have mentioned "Chesterton's Fence". I'd add a reference to another
of GKC's observations:

"When you break the big laws, you do not get liberty; you do not even get
anarchy. You get the small laws."

~~~
penpapersw
I can't help but wonder if the Fence comments had anything to do with my blog
post last night. Kind of like watching social osmosis in action :D

~~~
slavik81
Chesterton's fence has long been a staple of HN discussions. Unfortunately, it
involves no risk/reward analysis and it invariably leads to needless
conservatism. That's probably not a big deal in medical research where risks
are often high, but it's a problem when applied as a general principle.

(The opposite wisdom is probably grandma's ham [1]. Consider that if grandma
had passed away, they might never be allowed to try cooking the whole ham.)

I quite enjoyed that blog post, btw.

[1]:
[http://www.angelfire.com/ma/artemis9/humor/joke7.html](http://www.angelfire.com/ma/artemis9/humor/joke7.html)

~~~
throwanem
The point of Chesterton's fence _is_ cost/benefit analysis. It's not saying
the fence mustn't go; it's saying you need to know why the fence was put there
in order to evaluate whether it's best removed or left in place - that is,
your analysis needs to be informed by understanding of the status quo in order
to be accurate.

Grandma's ham is a great example. It might have been some subtle trichinosis-
related failure case that cutting the end off a ham reliably prevents, and
it's a shame we had to lose your great-uncle Joe to find out about that. Or it
might have been an issue of pan length. You can't know until you ask, so you
ask if you can. If you can't ask, then sure, you do the best you can with what
you have - I should like to hope there are no blind dogmatists here. But if
you _can_ ask, you'd be a fool not to.

~~~
slavik81
My only objection is the dogmatism. Asking why something is the way it is is
obviously worthwhile.

Perhaps you just have more faith in people to behave reasonably than I do.

------
ryandrake
> Faced with someone even more obsessive and bureaucratic than they were, the
> IRB backed down and gave us preliminary permission to start our study.

I love it! You can’t fight mindless rule followers with logic or reason. You
need to use even more detailed rule-following. This is true in every
bureaucracy where checklists have replaced good judgment.

------
maxxxxx
Mandatory IRB story:
[http://www.npr.org/templates/story/story.php?storyId=1023936...](http://www.npr.org/templates/story/story.php?storyId=102393652)

------
tom_b
And it's only worse at the staff level. Even getting patient data variables
for patients consented to IRB-approved studies is hit-or-miss. Groups build
barriers by wanting to "work with" principal investigators (PIs) instead of
providing data _to_ PIs and study staff. Data moats are everywhere.

My own group (at a research center attached to a university school of
medicine) has switched our focus from helping PIs with clinical data
management and applications to helping manage research samples and sample
processing, mainly by doing housekeeping around genomic processing across
different lab teams.

It's kind of hard to convey just how much I would warn devs away from
healthcare and healthcare research IT gigs.

------
Glyptodon
Maybe he should have called it a care audit and made it look like they were
auditing the doctors success at administering screenings instead of doing any
kind of patient research.

------
michaelbuckbee
A separate discussion of this happening in /r/slatestarcodex

Top comment there right now:

"Amateur. What you do is you sweet talk the clinicians into using their
medical judgement to adopt the form as part of their routine clinical practice
and get them to include it as part of the patient's medical records. Later...
you approach the IRB for a retrospective chart review study and get blessed
with waived consent. Bonus: very likely to also get expedited review."

Which is an extremely clever "hack" (in the way we like to use it on HN) of
the IRB setup.

------
sprague
You can't stop science. Someday, somebody, somewhere will answer the questions
that this IRB requirement silenced. Whether it's via underdeveloped countries,
or through the use of technologies too new to catch the regulator's eye,
science will progress.

~~~
euyyn
Or via a procedure that complies with the requirements of that IRB :)

------
msteffen
Why do a study in the first place? Particularly when the outcome of the study
is a foregone conclusion as far as the author is concerned. Just go to
professional conferences and say "researchers say this works, but those of us
with clinical experience know it doesn't. We write the diagnostic manuals,
let's write a diagnostic procedure that actually makes sense to us"

------
SilasX
I can't tell how much of this is exaggerated and how much really happened.
Sometimes I have a hard time parsing the difference.

They demand that you reveal what the study is on and sign in pen? Because the
Nazis didn't disclose that stuff and get signatures?

~~~
tedunangst
That's kinda the point. We need rules because Nazis (and Tuskegee, etc.) but
it's not clear we need _these_ rules about pen vs pencil.

~~~
vacri
The pen vs pencil thing is a total canard by the author, who is looking to
'dress-up' his indignation. Hand the form to the subject. Hand a pen to the
subject. Subject reads and signs the form. Take the form from the subject.
Take the pen from the subject. Job done.

It's not like the patient will go bonkers stabbing-crazy the millisecond their
flesh touches a pen.

------
chi17
Man. This sounds just like my workplace.

------
vacri
The author complains about there not being enough of an ethics gauntlet with
the bipolar test in question, and then complains of having to run an ethics
gauntlet to do the study.

The author keeps on bringing up nazis and suggests he knows about study
methods, when it's clear he's pretty clueless about doing studies. Several of
the "this is stupid" questions really aren't that stupid, given that these are
generic forms for any study. Why the fuck do you care if you're being asked if
you're removing organs? Just tick 'no' and move on to the next question. Does
the author also complain that other genders are selectable on online forms,
and not just his own gender - after all, it's equally useless information? The
setup to the story is: 'waaah, this application form isn't tailored
exclusively to MY study'.

I ran into problems with ethics committees myself when doing an (aborted) PhD
that caused significant needless delays. They're annoying as hell, I agree,
and there are some genuine complaints in the article, but most of what the
author is complaining about makes it clear that he doesn't understand the
point of the questions posed.

------
karmajunkie
I thought this was going to be about Ruby...

~~~
civilian
Hah, I think your joke got stomped by a mod re-writing the headline. :]

~~~
karmajunkie
yeah, they probably saw that coming a mile away... :)

------
jlarocco
Poor choice of headline for this site, IMO. I can't be the only one who
clicked expecting something about the Ruby REPL.

For the benefit of anybody else confused by the title, IRB in this article
refers to an "Institutional Review Board"
[https://en.wikipedia.org/wiki/Institutional_review_board](https://en.wikipedia.org/wiki/Institutional_review_board)

------
waisbrot
It sounds like the study was poorly designed. Having an actual PI would have
helped. The IRB probably would have seemed a lot less painful with a better-
designed study and an engaged PI.

~~~
civilian
You're probably getting downvoted because you're making a casual armchair
judgement and this is _Scott fucking Alexander_ we're talking about.

//edit: I only just got to this section of the article, but it feels good to
include as a counter-example.

 _> During that year, Dr. W and I worked together on two less ambitious
studies, carefully designed not to require any contact with the IRB. One was a
case report, the other used publicly available data._

 _> They won 1st and 2nd prize at a regional research competition. I got some
nice certificates for my wall and a little prize money. I went on to present
one of them at the national meeting of the American Psychiatric Association, a
friend helped me write it up formally, and it was recently accepted for
publication by a medium-tier journal._

 _> I say this not to boast, but to protest that I’m not as much of a loser as
my story probably makes me sound. I’m capable of doing research, I think I
have something to contribute to Science. I still think the bipolar screening
test sucks, and I still think that patients are being harmed by people’s
reliance on it. I still think somebody should look into it and publish the
results._

~~~
tallanvor
Sorry, but this is a real question: Why would we know who Scott Alexander is?

~~~
civilian
He's written a lot of great and influential things, and I feel like he's
entering the "nerd/intellectual" common knowledge.

The biggest example for me is Meditations on Moloch, which is the only thing
that's ever made me deeply question my libertarianism.
[http://slatestarcodex.com/2014/07/30/meditations-on-
moloch/](http://slatestarcodex.com/2014/07/30/meditations-on-moloch/)

"I can tolerate anything except the outgroup" is a depressing piece on
American politics. [http://slatestarcodex.com/2014/09/30/i-can-tolerate-
anything...](http://slatestarcodex.com/2014/09/30/i-can-tolerate-anything-
except-the-outgroup/)

My original comment is not the best, it was a kneejerk reaction. I'm not
representing the fandom well. :-/

------
alexashka
The real nightmare is having read through 3/4 of this, only to realize it's
the longest bitchy complaint I've ever read through.

No substance and a lot of arrogance.

The more curious take-away is that these extremely long winded rants of his
have younger people fooled as being worthwhile.

If you have something of value to say - you distill it, condense it, so as to
ease the mental effort required for someone else to understand the point
you're trying to make.

It reminds me of what philosophers do - they exhaust you into thinking they're
making some sophisticated point simply because they're using fancy words that
produce the thought of 'am I missing something? Surely no one is spouting
complete nonsense for 400 pages, it must be me'. It isn't you. It's them on a
massive ego trip - this guy being a great example.

~~~
sctb
A more charitable interpretation might be that a detailed and clinical account
of the absurdity of the most tedious bureaucracy may have made for rather...
dry reading.

~~~
alexashka
I can summarize what he's said in the 3/4s that I've read, down to 2 or 3
paragraphs, with a few bullet points.

The author complains about tedious procedure, while writing the most tedious
blog I've had the displeasure of reading.

The obliviousness and callous arrogance of the author, combined with cult-like
reverence some people seem to have for his babbling is what makes this at all
worth commenting on.

