
Facebook and OkCupid Broke the Law When They Experimented on Users - jboynyc
https://medium.com/@JamesGrimmelmann/illegal-unethical-and-mood-altering-8b93af772688
======
greglindahl
Basically, they're claiming that the law says that any experiment on people,
by anyone, requires informed consent. Every business with customers breaks
this law. "Let's see if substituting medium roast espresso beans raises
sales!" Sorry.

~~~
thrownaway2424
It's also ridiculous because it presupposes an objectively correct way to rank
results. The way Facebook places items on the page isn't the result of some
natural law, it's the result of code they wrote. Nobody can seriously say that
That Way of ranking results is superior to This Way, and certainly not that
This Way is somehow illegal.

~~~
scarmig
I wonder if a retailer hiking or cutting prices would fall afoul of the legal
idea being pushed here. After all, it amounts to an experiment on people to
discover the ideal price of a good.

~~~
colbyh
This blog post's interpretation of "illegal" would include any change to any
product with the intent to stimulate user behavior. It's a very slippery
slope.

------
namecast
This isn't the law; it's a set of guidelines for federal agencies to follow.
It'd be illegal if the department of health and human services or the FDA
pulled this type of stunt, because they've voluntarily agreed not to do that
by amending their rules and regulations to comply with the common rule that
this blog post refers to
([http://www.hhs.gov/ohrp/humansubjects/commonrule/](http://www.hhs.gov/ohrp/humansubjects/commonrule/))

Facebook and OKCupid are, at least not at the moment, part of the federal
government.

~~~
jwise0
I recommend reading the rest of the article; the author describes multiple
state laws that apply in this case.

~~~
j2kun
This is actually not so clear to me. The law in question [1] specifically
states that it defers to the federal law [2], which clearly states in section
§46.102(e):

> "Research subject to regulation," and similar terms are intended to
> encompass those research activities for which a federal department or agency
> has specific responsibility for regulating as a research activity.

Furthermore, the definition of "minimal risk" is given in part (i) of the same
section,

> the probability and magnitude of harm or discomfort anticipated in the
> research are not greater in and of themselves than those ordinarily
> encountered in daily life or during the performance of
> routine...psychological examinations or tests.

The argument against Facebook and OKCupid are pretty slim here... How could
you experience significantly more discomfort in daily life if OKCupid slightly
changed their rating system? Any reasonable interpretation of this risk is
negligible.

[1]
[http://mlis.state.md.us/2002rs/bills/hb/hb0917t.pdf](http://mlis.state.md.us/2002rs/bills/hb/hb0917t.pdf)
[2]
[http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf](http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf)

~~~
dalke
It's a bit tricky to understand the details, but I think I've got some of it
figured out.

There's "research" and there's "research subject to regulation." The federal
law says that all research by or directed by federal departments is subject to
this law. It also says that everyone, including you and I, who are doing
research subject to regulation (which includes drug development) are also
subject to the law.

That is, even without MD 917, all research in MD which is subject to
regulation would have been subject to the law.

However, this leads to a gap. Anyone doing research, but not research that is
subject to regulation, and not research which is funded or directed by a
federal department or agency, is not subject to these ethical requirements.

The MD law (see
[http://www.oag.state.md.us/Healthpol/hb917letter.pdf](http://www.oag.state.md.us/Healthpol/hb917letter.pdf))
is meant to close the gap, and include research which is not subject to
regulation by a federal department.

The "minimal risk" section you pointed out is for expedited review. That is,
the research still needs an IRB, but the IRB can use the expedited review
process instead.

OkCupid didn't have an IRB, so the minimal risk section doesn't apply.

~~~
j2kun
Thanks. It's really hard to understand what MD917 is saying with all the
strikethrough.

~~~
dalke
I agree. The problem is that MD917 is the bill, but not the actual law. Bills
are often structured as a delta to the law.

------
scarmig
"from the horrors of Nazi medicine to the deliberate withholding of care from
syphilis victims in the Tuskegee experiment"

This is a deeply offensive comparison. Reading the complaint letter makes me
wretch.

~~~
MetaCosm
Yeah, it was horrible to read. It is a rare mix when someone is -- self-
important, offensive, long-winded, ill-informed, and manages to invoke
Godwin's Law.

For awfulness: 5/5

------
jawns
The letter that the authors submitted to Maryland's attorney general
([http://james.grimmelmann.net/files/legal/facebook/MDAG.pdf](http://james.grimmelmann.net/files/legal/facebook/MDAG.pdf))
point out that Maryland's HB 917 -- which essentially applies the federal
Common Rule to private companies who do business in Maryland -- defines
"research" as "a systematic investigation ... designed to develop or
contribute to generalizable knowledge."

That definition is so vague that practically any sort of interaction with a
person in which any information about the interaction is recorded and analyzed
can be construed as research. If a Girl Scout troop makes a graph of which
neighborhoods buy the most cookies ... boom, that's research on human
subjects.

I'd be interested to know whether any entity has been prosecuted for violating
HB 917. I can't imagine that the extreme vagueness of the definition would
hold up against a determined challenge.

What puzzles me is why James Grimmelman, J.D., and Leslie Meltzer Henry, J.D.,
think that the ultra-vague definition of "research" is legally tenable. Like,
can't they anticipate the objections to the definition -- that it effectively
bans all sorts of everyday actions that no reasonable person wants to subject
to these sort of legal hurdles?

~~~
j2kun
More likely they want exposure and a high profile win rather than that they
really think this is as immoral as they claim (via their comparisons to Nazi
experiments). If they can get away with it and boost their career, why should
they care about the impact?

------
Someone1234
"This is why we cannot have nice things."

OKCupid's blog is super interesting but they might have to discontinue it if
it is going to generate legal challenges like this one. The only reason we
know about their A/B testing is because they openly told us about it, which if
they get prosecuted for it might be a mistake.

I kind of liken it to the "never apologise" rule many companies now have. It
is when common sense/public interest run into cold legal restrictions.

------
terravion
It seems like this "law" as interpreted the authors to ban A/B testing would
violate the 1st Amendment rights of companies to uncensored speech with their
customers. I hope that they challenge FB in court so it can be overturned.

------
nostromo
A/B testing is also illegal by this standard.

~~~
jboynyc
It is not, because it is not "designed to develop or contribute to
generalizable knowledge" in the sense that it seeks to communicate to wider
scholarly publics.

------
barnabee
This makes no sense to me. The algorithms employed by Facebook, OkCupid et al.
are not published and presumably changed regularly both to better serve their
users and to advance commercial imperatives - apparently this is OK but if you
measure the outcomes and call it an experiment it's not?

I don't buy this, as long as all representations made to customers about the
service are met, it seems perfectly reasonable to tweak the already unknown
algorithms behind the scenes for whatever reason.

~~~
pdkl95
> as long as all representations made to customers about the service are met

Exactly this. The thing that some people seem to be missing is that there is a
(very) good chance nothing is banned. It's just that you have to actually
_ask_ the customer. And IRB could waive a lot of the usual requirements (that
are primarily intended for experiments with a LOT more risk).

The Common Rule, in these cases, is primarily about enforcing that stuff like
a boilerplate EULA does _not_ count as properly informing the customer and
getting their approval. When humans are involved you have to get explicit
approval each time as a precaution, which isn't actually that hard.

------
dalke
The Maryland House Bill 917 is at
[http://mlis.state.md.us/2002rs/billfile/hb0917.htm](http://mlis.state.md.us/2002rs/billfile/hb0917.htm)
. The gist of it is:

(a) Compliance with federal regulations. -- A person may not conduct research
using a human subject unless the person conducts the research in accordance
with the federal regulations on the protection of human subjects.

(b) Scope to include all research. -- Notwithstanding any provision in the
federal regulations on the protection of human subjects that limits the
applicability of the federal regulations to certain research, subsection (a)
of this section applies to all research using a human subject.

My problem is, I'm not sure what MD 917 (b) means. I'll start with §46.102 (a)
(1) at
[http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#...](http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#46.101)
:

> (a) (1) Research that is conducted or supported by a federal department or
> agency, whether or not it is regulated as defined in §46.102, must comply
> with all sections of this policy.

Is MD 917 (b) supposed to mean that §46.102 (a) (1) applies to all human
research in Maryland? I believe that's the case, because §46.102 (a) (2) only
applies to those organizations which are regulated by a federal department, so
would already be covered under the law.

Section §46.102 (b) has a list of human research which is exempt. The most
relevant for Facebook and OkCupid appears to be:

> (2) Research involving the use of educational tests (cognitive, diagnostic,
> aptitude, achievement), survey procedures, interview procedures or
> observation of public behavior, unless: (i) information obtained is recorded
> in such a manner that human subjects can be identified, directly or through
> identifiers linked to the subjects; and (ii) any disclosure of the human
> subjects' responses outside the research could reasonably place the subjects
> at risk of criminal or civil liability or be damaging to the subjects'
> financial standing, employability, or reputation. [Note: does not apply to
> children.]

If A/B testing falls under 'educational tests' or 'survey procedures' then
it's exempt.

Otherwise - and this is another tricky part - §46.102 (c) says "Department or
agency heads retain final judgment as to whether a particular activity is
covered by this policy."

Who is the department head for Facebook and OkCupid? §46.102 (a) defines it
as:

> (a) Department or agency head means the head of any federal department or
> agency and any other officer or employee of any department or agency to whom
> authority has been delegated.

MD 917 doesn't have the information for how to interpret §46.102 outside of
the federal context for which it was written.

An alternative interpretation of MD 917 (b) is that _no_ human testing is
exempt, including §46.102 (b) (2) which allows surveys. Instead, _all_ testing
must go through an IRB.

However, if that's the case then the other exceptions of §46.102 (b) are also
no longer allowed, including

> (4) Research involving the collection or study of existing data, documents,
> records, pathological specimens, or diagnostic specimens, if these sources
> are publicly available or if the information is recorded by the investigator
> in such a manner that subjects cannot be identified, directly or through
> identifiers linked to the subjects.

which would require IRB review for every historian working out of previously
published materials.

Therefore, I don't think this alternative interpretation is correct.

Which means I don't know if MD 917 really applies to OkCupid or Facebook. (The
Facebook research was published in a journal which follows the Common Rule
compliance; I presume the journals have a way to resolve the question of what
"department head" means when there is no department head - it must have come
up before. I only investigated MD 917 compliance here.)

_EDIT_ : The MD attorney general clarified this at
[http://www.oag.state.md.us/Healthpol/hb917letter.pdf](http://www.oag.state.md.us/Healthpol/hb917letter.pdf)
.

> Implicit in the federal regulations, and therefore in House Bill 917's
> mandate for compliance with these regulations, is the requirement that a
> person have in place a reasonable process for determining whether research
> falls within one of the exempt categories.

So determining if A/B testing is exempt comes down to if there's a reasonable
process for determining if it's exempt. It's not clearly illegal.

~~~
dragonwriter
> My problem is, I'm not sure what MD 917 (b) means.

Probably Annotated Code of Maryland Section 13-1601(b), added by House Bill
917, means exactly what it says. " _Notwithstanding any provision in the
federal regulations_ on the protection of human subjects _that limits the
applicability of the federal regulations to certain research_ , subsection (a)
of this section applies to all research using a human subject." (emphasis
added)

> §46.102 (b) has a list of human research which is exempt.

But those exemptions are a provision of the federal regulations which limits
the applicability of the federal regulation to certain research, and so are
_expressly excluded_ from the part of the regulation incorporated into
Maryland law by Section 13-1601(b).

> Otherwise - and this is another tricky part - §46.102 (c) says "Department
> or agency heads retain final judgment as to whether a particular activity is
> covered by this policy."

This, again, is a provision of the federal regulations which limits the
applicability of the federal regulation to certain research (to those things
not excluded by the relevant funding department/agency head), and so are
_expressly excluded_ from the part of the regulation incorporated into
Maryland law by 917(b). So there's no need to figure out how it applies -- it
doesn't.

> However, if that's the case then the other exceptions of §46.102 (b) are
> also no longer allowed

Note that, if you go through the link you posted and click on the final passed
(Third Reading) version of the bill and note the edits from the prior version,
that _prior_ version of the bill had express language incorporating the
exceptions in 45 CFR § 46.102(b) as exceptions to the Maryland law, and this
was stricken in favor of the language now present expressly disregarding any
exceptions in the federal regulations. This makes it pretty clear that,
_whether or not this is desirable_ , the legislative intent was very expressly
_not_ to incorporate the exceptions in 45 CFR § 46.102(b).

~~~
dalke
I suspect my edit was too late for you to read. I added a link to
[http://www.oag.state.md.us/Healthpol/hb917letter.pdf](http://www.oag.state.md.us/Healthpol/hb917letter.pdf)
, where the attorney general describes exemptions:

> Therefore, under the applicable federal regulations, a person may conduct
> research in an exempt category without, for example, obtaining review by an
> institutional review board. A Maryland researcher likewise may do so under
> House Bill 917, because such a person is conducting the research “in
> accordance with the federal regulations on the protection of human subjects”
> and, consequently, does not violate the prohibition in § 13-1602(a) of the
> Health-General Article.

This is dated May 2, which is after the bill passed both houses and before it
was signed by the governor.

This seems like the exemptions in §46.102 (b) are all allowed under MD law.

~~~
dragonwriter
There seems to me, on the surface, to be some very odd statutory construction
underlying that analysis (which is presented in a much less full form than a
typical opinion letter, because that's not really what this is, so its really
hard to form much of an opinion of its conclusion) -- but certainly it at
least indicates how the legal authorities of the State may be likely to
_apply_ the law, presuming that their construction of the statute hasn't
changed in the intervening decade.

------
xux
So.... are A/B testings now banned?

------
vonklaus
Maybe they were illegal. The point is they are companies and can do whatever
they want, largely without scrutiny. I mean this in the sense that it is
difficult to bring charges against them, as evidenced by basically the entire
financial sector.

When things like this are published it just serves to dissuade them from
PUBLISHING the results of the experiments but not from actually CONDUCTING
THEM.

I would much rather know when they do things like this than have them continue
to conduct them in secret.

------
j2kun
I encourage people to read Section 46.102 [1] (page 4) of the federal
regulations that Maryland is enforcing. It lays out the definitions of all
these terms (as a mathematician, I put the highest weight on definitions), and
in particular section (i) on "minimal risk" is clearly in Facebook/OKC's
favor.

[1]:
[http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf](http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf)

------
nernst
The Common Rule AFAIK only applies to places that get government funding, so I
don't see how this applies to research FB does on its own.

~~~
scarmig
Two Cornell researchers were collaborators in the experiment. So it's possible
that its IRB screwed up somehow. Hard to see how Facebook or OkC is at fault
for that, though.

------
acqq
Here

[https://medium.com/@JamesGrimmelmann/illegal-unethical-
and-m...](https://medium.com/@JamesGrimmelmann/illegal-unethical-and-mood-
altering-8b93af772688)

the author explains more clearly his position:

"Federal law — primarily the so-called “Common Rule”— (
[http://www.hhs.gov/ohrp/humansubjects/commonrule/](http://www.hhs.gov/ohrp/humansubjects/commonrule/)
) regulates research on people in the United States. The details are
complicated, the gist simple. If you engage in “research involving human
subjects,” you must have two pieces of paper before you start. You need a
signed informed consent form from the person you’re experimenting on, and you
need approval from an IRB (short for “institutional review board”)."

~~~
Someone1234
The Common Rule linked doesn't apply to private businesses at all. So
therefore I don't understand the author's point here?

~~~
acqq
That's what I also wanted to point to, you were faster.

The "Common Rule" appears to be a rule for the Federal agencies, and it is a
rule, not a law.

Long jump from that to the illegality? But I'm certainly not a lawyer, whereas
the article author seems to be "James Grimmelmann, Professor of Law,
University of Maryland."

Strange.

~~~
aidenn0
According to TFA, Maryland law codifies the common rule.

~~~
acqq
Thanks. So the author could have stated from the start that it's (I'd say
"just") "illegal in Maryland." Why then "primarily" mentioning the "rule" and
"federal?"

And is the Maryland law relevant? By the author (from the article I've linked
previously):

"But wait, you may be saying, Facebook isn’t in Maryland, and neither is
OkCupid. True. But they have users in Maryland, and given the size of the
experimental groups, it’s overwhelmingly likely that they experimented on
residents of the state. Facebook manipulated with hundreds of thousands of
News Feeds; that’s thousands of Marylanders. OkCupid gave bad recommendations
to about five hundred users."

Well... "likely."

------
hyperpape
The phrase "generalizable knowledge" seems crucial. I am not a lawyer, and
have not done human subjects research, so hope someone else can comment.

However, looking around, it sure sounds like this would not normally cover A/B
testing, a business changing its prices and tracking results, or similarly
"local" investigations.

Here is one IRB's interpretation of that term:
[http://www.irb.wsu.edu/definitions.asp](http://www.irb.wsu.edu/definitions.asp).
Others seemed similar.

------
smoyer
The typical TOS these days turns users into corporate serfs ... I'll bet they
respond to these complaints by alleging the users had granted consent.

------
brg
Would it be correct then that any change in layout or diversity in Maryland
newspapers, periodicals, and publications would also be illegal? For instance,
asking people they prefer a Sports section prior to Entertainment, or having
Politics or Local Interest above the fold? Would it be ok to make the changes
given that the effects were never measured?

------
michaelochurch
OP's position is extreme, and I feel like any negative publicity to Facebook
or OkCupid over this is going to ultimately have the effect of punishing them
for _saying_ that they ran these experiments.

Changing the way content is ranked by your private service is not in the same
ethical territory as the Tuskegee Syphilis Study. It's just not.

That said, there is an interesting discussion here, which is: _where is the
line?_ Netflix runs an experiment (and whether it chooses to collect data)
every time it changes its ranking algorithm, but I think we'd agree that it's
ethically OK in that case. What about those sleazy mobile game companies that
use data science and heavy experimentation to find and target "whales" who
have an in-app-purchase addiction problem? Not ethically OK. I don't know
where the line is, but it's clear to me that some tech companies are on one
side and some are on the other.

------
dang
We changed the url from
[http://laboratorium.net/archive/2014/09/23/facebook_and_okcu...](http://laboratorium.net/archive/2014/09/23/facebook_and_okcupids_experiments_were_illegal),
which points to this longer piece.

