
What if we tested laws before passing them? - robg
http://www.boston.com/bostonglobe/ideas/articles/2010/12/12/law_lab/?page=full
======
DougWebb
Or, we could pay attention to the 10th Ammendment:

"The powers not delegated to the United States by the Constitution, nor
prohibited by it to the States, are reserved to the States respectively, or to
the people."

Instead of randomly applying variations of a law, don't federalize it. Let the
states pass their own variations, and see how that works out. Sure there are
other variables at play, but they mostly revolve around how suitable the law
is for the people it applies to, and that's a legitimate reason for the law to
vary and part of the purpose of the 10th Ammendment: local self-governance.

I believe most federal laws should be repealed and replaced with guidelines
for the states to follow when designing their own laws. These would be "Best
Practices" for governments. Most of the world treated the US Constituion this
way as they wrote their own constitutions, and I think that worked pretty
well. It should work for State governments too.

~~~
ddlatham
Using the states to perform the experiments is useful, and we've been doing
that for hundreds of years, but it's besides the point of the article.

The whole point is to produce sets of people that are drawn from an identical
distribution where the only difference is whether the law applies to them or
not. That way any changes that result can be confidently attributed to the law
itself.

Right now, when we see differences between California and Iowa, there are a
million other factors that may be causing different outcomes than a single
law.

~~~
DougWebb
Random testing, particular in cases like drug trials where the technique
started, also relies on neither the testor nor testee knowing who is being
tested. That can't be the case for laws. So, if someone behaves differently,
is it because of the law that applies to them, because of the law that applies
to others but not them, or because they're being watched to see how they
behave?

You know, we used to have laws like this: segregation laws. The people who
weren't subject to the laws became abusive to the others, and the people who
were subject to the laws rebelled against them. The situation was very unjust.

~~~
lutorm
Segregation laws are by nature nonrandom. It doesn't seem very apt of an
analogy to me. You're right about the fact that people might react differently
when they know they are being singled out, though.

~~~
DougWebb
My point in bringing up segregation wasn't about the selection process, but
the outcome. I believe that if two groups of people are given different
variants of a law to live by, one of those groups is going to discover a way
to use the difference in the laws to gain an economic, social, or political
advantage over the other group. As soon as that happens, you've got state-
mandated discrimination, and we already know how that turns out.

~~~
Travis
Not necessarily always the case. Anyway, the author isn't suggesting that we
do this in all areas of society. Just the ones that fit the experimental
structure pretty well. Using the articles examples of education and crime --
seems pretty difficult to imagine criminals gaining advantages over each other
b/c they had different rehab experiences.

Also, "[using] the difference in the laws to gain an ... advantage" does not
equal state mandated discrimination. It's more of a state-accepted, or
allowed, discrimination. State mandated discrimination is something a little
more explicit, like when Zimbabwe's President Mugabe enacted laws that took
land from one racial group and bequeathed it to another.

------
hugh3
Note how the example is lower taxes: why don't we give _lower_ taxes to a
hundred thousand people and see what they do? But now suppose the proposal is
_higher_ taxes for a hundred thousand people. How do you think the hundred
thousand people in the test group are going to react? Not too well, and
probably not nearly the same way as if you raised taxes on everybody.

Isn't this one of the supposed advantages of having states, though? So that
different laws can be tried out in different places under the same general
societal conditions and we can learn what works and what doesn't work? And yet
doesn't the Federal Government keep trying to subsume more state powers into
itself?

~~~
bhousel
You're asking how people would feel if they were forced into high-tax and low-
tax test groups.. But suppose people are given a choice opt-in to a tax
lottery system, such that instead of paying a fixed rate the taxpayer would
have a random chance of paying either a higher or lower rate.

As initially reprehensible as this sounds to me, the government could make
quite a bit of money by taking clues from the gaming industry. Gamification of
the tax system?

~~~
DavidSJ
Perhaps it should be required that the net income from this lottery be 0.

~~~
bhousel
That would be nice, but then you're not doing an experiment.

~~~
adrianN
Why not? You can still observe how people behave differently if they have
higher or lower taxes.

~~~
bhousel
I guess that's true. But what I meant to say was that in order to arrange for
the results to net a zero difference from what the government would collect
normally, you're not really randomly assigning tax rates or running a lottery,
you're just fixing people's rates at something other than what they would
normally be.

Something about manipulating the conditions of the experiment (even a stupid
thought experiment) to force a desired outcome doesn't sit right with me.

------
tibbon
YES!

We should approach this from a two-tiered testing platform.

The first tier is rather like how you'd do it in Rails or any other TDD
environment.

We write a list of tests that we know should pass as true at the end. Let's
say, start with the US and State constitutions as the basis.

    
    
        def must_not_prohibit_exercise_of_religion()
        {
            People.can_exercise_religion?
        }
    
    

We then write in the test laws and see what side effects occur. When those
side effects occur, we must make note of them and consider the potential
exceptions.

    
    
        def freedom_of_speech()
        {
            unless(crowded_theatre && People.speak == fire)
                People.can_speak?
            end
        }
    

As a second layer then we must test it on smaller populations to determine the
effects. This has worked wonderfully in some situations. "Gay Marriage will
destroy marriage and society" is a claim I've heard prior. So test it in
Massachusetts. Several years later, no marriages on record have been destroyed
due to gay marriage, and the metrics of society's stability have not moved
significantly in any direction that we can attribute to same-sex marriage. So
it (should) pass and become law nationally because we've observed its effects.
We can observe similar for the long term effects of required health insurance
in Massachusetts as we debate how things should work out nationally.

Just as we have 'human-readable code' why shouldn't our laws be machine
readable? Computers can remember/understand 200+ years of case law probably
better than humans. Again, this is something that we could test.

One of my friends is working on a law firm run by computers
(<http://www.robotandhwang.com/>). Of course the machines can't show up in
court or legally offer advice, but they can advise the attorney. (Law.com and
ABA Journal have cited him doing this, so maybe it isn't as crazy as it
sounds).

~~~
jgfoot
I was about to respond until I realized your post was a Swift-ian parody so
sophisticated as to be almost over my head. Well done.

But if we were going to try to reduce the complicated, inherently human, and
uncertain concept of law to computer code, surely you'd agree that a
declarative programming language would be a better choice. Something like:

prohibits_free_exercise_of_religion(X) :- unconstitutional(X).
respects_establishment_of_religion(X) :- unconstitutional(X).
unconstitutional(X) :- not_law(X).

Now, all you have to do is write Prolog code to test for when a law prohibits
the exercise of religion or respects the establishment of religion!

~~~
rntz
You have your Prolog backwards.

    
    
      foo(X) :- bar(X).
    

means that foo(X) is true if bar(X) is true, not the other way around. So your
code would indicate that anything that's not a law is unconstitutional, and
anything unconstitutional prohibits free exercise of religion (for example).

~~~
jgfoot
oh! that explains why my Prolog hacking has been so unsuccessful. Thanks very
much.

------
messel
Why stop at new laws? Existing laws should be relentlessly reviewed for
relevance.

One issue I see with random sample law experimentation is blindness to effects
which only crop up when everyone is subject to a law. Take for example gun
control. If a small sample of folks aren't allowed to have guns there may be
little negative fall out. But taken to the extreme if gun ownership is illegal
than by definition only law breakers would have guns making the random
burglary or home invasion far less dangerous for criminals and far more
dangerous for law abiding citizens.

~~~
Zak
Gun control is a case where there's probably enough data to do interesting
analysis if someone were to collect it and study it without trying to make it
show a specific result. Noteworthy data points would be the first derivatives
of gun crime, non-gun violent crime and burglary rates before and after
specific gun control measures were passed or repealed.

~~~
Retric
There is a fair amount of evidence that gun owners are less safe than non gun
owners, and societies with higher levels of gun ownership tend to be less safe
overall. But, changing gun laws don't alter the prevalence of gun ownership
vary quickly so it's hard to such analysis with idential populations.

~~~
Zak
There's also the fact that correlation does not imply causation. The
possibility of a common cause is fairly obvious here: people who are at higher
risk of being the targets of crime may be more likely to buy guns for
protection, and people who are armed may be more likely to become involved if
they see someone else being attacked.

~~~
Retric
Yep, while hiding and calling the cops is generally safer during a robbery,
the attitude that causes people to defend their property is the same one that
results in gun ownership. Add in people being shot with their own weapons and
the home protection benefit becomes very complex. There is an idea that gun
ownership deters criminals but at the same time it promotes criminals buying
guns escalating the situation.

PS: For a somewhat less contentious example, video games have been linked with
a significant drop in crime. However, because that does not agree with their
biases people focus on the direct effects to support their argument.

~~~
yummyfajitas
_Add in people being shot with their own weapons..._

This is a pretty minor effect, unless you are including suicides. In 2007,
there were only 613 accidental firearm deaths. For comparison, there were 1039
"transport, other" deaths (I believe this means plane/boat crashes).

~~~
Retric
All of these numbers are fairly low. Adusting for the percentage of gun owners
and you end up with a higher rate than just 5% of all deaths from firearms
being accidental. And when you start looking at low risk populations (for
violent crime), accidental death from gun ownership is fairly significant
_relative_ risk.

All homicides Number of deaths: 18,361

Firearm homicides Number of deaths: 12,632

vs: Motor vehicle deaths 33,808

Death rate extrapolations for USA for Smoking: 440,000 per year (probably
exagurated) <http://www.wrongdiagnosis.com/s/smoking/deaths.htm>

------
btilly
It is an interesting idea.

However I'd be concerned that the law would work differently with a subset
than the whole population. For instance take a law banning cellphones while
driving. If you try to apply this to half the population, then the police
aren't going to try to apply it because at least half the time they would be
pulling over people who are allowed to use cellphones. But if you pass the law
for all of the people, then the police could enforce it. So you try to solve
this by saying that it will be selectively enforced according to location. But
now you've gotten rid of the random selection that is the heart of evaluating
the statistics.

For another example, look at the tax cut example they were discussing. The
challenge here is that the touted economic benefit of tax cutting is that
people with lower taxes spend their money, increasing circulation, and general
prosperity. (And eventually improving tax revenue.) However there is no way to
economically separate out the people you're giving the cut to from those you
aren't. Therefore any economic change cannot be attributed to the tax cut, and
you're unlikely to get a clear economic difference. (Other than that the less
taxed have more disposable income.)

~~~
Symmetry
You can't see the change in the overall economy, but you can see if the people
who you gave the tax cut to are investing or spending more.

~~~
btilly
We already have that data.

The debated question is how many times said money continues to circulate.

------
Goladus
I don't think you could test taxation, for a number of reasons.

First, you don't have time to measure the effects of funding changes. How do
people behave with the new services funded by the taxes, when you aren't
collecting enough to provide them anyway? Does the government go bankrupt? No,
because revenue was not changed significantly due to the small size of the
test.

Second, sample populations are unlikely to exhibit the same behavioral traits
of large-scale populations. For example, if you applied a tax increase to a
limited segment of the population the first behavior change you would see is
that people would try to escape the test group.

For something like a sales tax this is even more obvious. The overall rate of
taxation matters much less than the rate relative to a competitor. It's a free
competitive advantage and of course behavior will change-- the untaxed
businesses will be more likely to thrive and grow while driving their
competitors bankrupt. When everyone has the same tax rate they have to find
real market advantages.

The whole point of taxes is that they are applied universally as much as
possible. Everyone contributes a little bit so that the society can have
things that benefit everyone but the market is unlikely to ever provide. NPR
just discussed autopsies as an example of this. Understanding why people die
is good for the public, yet only the most altruistic individuals are inclined
to pay for an autopsy of their own loved ones.

While I suppose it is possible to design test runs of tax laws, I doubt it
will ever be as simple as picking a "representative sample of 10,000." There
are probably only certain kinds of laws that can be tested, and tax laws
probably aren't among them.

------
CWuestefeld
What is the purpose of passing a law? I submit that its primary purpose may
not be to bring about a change in the behavior of those subject to it, but
rather, as a signaling mechanism by which a politician can say to those in his
party, or his constituents, "I'm playing on your team".

We see time and time again regulation that is known to be worthless or even
destructive even before it's implemented (e.g., CAN-SPAM, airport porn
scanners), yet bureaucrats stand behind them, and even trumpet their
"success", just the same.

Recent threads here have discussed whether voting behavior is really based on
weighing issues, as opposed to signaling one's identity as a member of the
"club" (party, special interest, etc.). If this is true, then it's only
logical that the politician's job is to make it know that he is the
representative of that "club", and actual goodness of his work isn't
particularly relevant.

~~~
anamax
> I submit that its primary purpose may not be to bring about a change in the
> behavior of those subject to it, but rather, as a signaling mechanism by
> which a politician can say to those in his party, or his constituents, "I'm
> playing on your team".

Another important purpose is encouraging support.

For example, one of the best ways to get bankers to contribute to your
campaign is to propose banking regulations.

------
jessriedel
The RAND study is famous for trying this on a pretty large scale with health
insurance:

[http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experimen...](http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experiment)

Robin Hanson has been a big proponent of resuming similar experiments:
<http://www.overcomingbias.com/2007/05/rand_health_ins.html>
<http://www.overcomingbias.com/2007/05/rand_health_ins_1.html>
<http://www.overcomingbias.com/2007/05/rand_experiment.html>

Worth reading even if you're of a different political persuasion.

------
JimboOmega
I've advocated this for a while in a somewhat obvious field, traffic law.

However, there's a flaw; science in the political arena is subject to
interpretation. In VA, we've gone through this with red light cameras. For a
while they were authorized at a few intersections to see if they reduced
accidents.

They didn't (<http://www.thenewspaper.com/news/18/1844.asp>).

But a few years later, red light cameras are back, the whole study thing
having been an annoying thorn in the side of politicians, who like red light
cameras (because they are perceived as good for safety, for money, whatever -
who knows?)

So an article comes out like this: [http://www.washingtonpost.com/wp-
dyn/content/article/2009/02...](http://www.washingtonpost.com/wp-
dyn/content/article/2009/02/18/AR2009021801108.html) which contradicts the
link I posted before.

How can the same study come to opposite conclusions? That's politics. In any
case, the politicians decide what they want to do, and then change the science
to suit.

After all, can you really imagine a politician saying "I supported X fully,
until the results of the study came out, and clearly, X is not effective, so I
no longer support X?"

Only if the study is poll numbers, I imagine.

------
pierrefar
Surely this is a prime candidate for the Hawthorne effect?

From Wikipedia: <http://en.wikipedia.org/wiki/Hawthorne_effect> _The Hawthorne
effect is a form of reactivity whereby subjects improve or modify an aspect of
their behavior being experimentally measured simply in response to the fact
that they are being studied, not in response to any particular experimental
manipulation._

~~~
metageek
From the article:

> _But in other cases, participating knowingly in an experiment could distort
> the result (this phenomenon even has a name, the Hawthorne effect)._

------
tomjen3
this sounds smart until you realize that would mean giving up the principle
that the law treats everybody the same, which we always complain about the
politicians doing.

~~~
ugh
I see this as a possible problem that would have to be considered when such an
idea is implemented but not something that stops the idea dead in its tracks
because it is fundamentally opposed to the principle of equality.

You would have to make sure that the tests are truly random (everyone has an
equal chance of being selected), that they run only for a limited and pre-
determined time which cannot be extended and that the differences to existing
laws is not too big (e.g. people shouldn’t suddenly become multi-millionaires
because some law was tested on them).

Such tests would then no more or less contribute to inequality than, say, jury
duty.

------
jokermatt999
Hmm..Hacker News' love of A/B testing also applies to laws, apparently.

Also, isn't one of the arguments for stronger state laws and weaker federal
laws very similar to this proposal? That is, if you give states greater
autonomy, it's easy to test the effects of laws and give citizens choice over
which laws they'd like.

------
brudgers
IANAL, but I don't think this meets the Equal Protection Clause of the 14th
Amendment.

------
qjz
I'd be happy if they developed a protocol for testing the constitutionality of
a law. That would have to be part of any experiment, since the effectiveness
of a law is irrelevant if it's unconstitutional.

------
RyanMcGreal
>Randomly assign a representative sample of the population - say, 10,000
taxpayers - a lower tax rate, and see what happens.

This assumes the tax rate _only_ affects the individual paying tax; but taxes
are used at least in part to finance public goods - positive externalities
that also affect individuals. If a random group of people personally pay lower
taxes but still benefit from the available public services that other
taxpayers are still funding, that doesn't tell us anything useful.

------
Aaronontheweb
We have a system for doing this - it's called Federalism.

~~~
jimmyk
Unfortunately, the federal aspect of the US government has almost disappeared
as a result of the 16th amendment and the 17th amendment and a few other
factors such as mandates that the states must follow in order to receive
federal funding. The US government is now plainly a national one.

------
gills
Experimenting on humans with a system backed by the use of force would be
despicable. What happened to basic respect for each other?

<snark> Anyway, if we want to know how a group behaves when lawbreaking goes
unpunished, we need look no further than the FIRE industries, Congress, and
the current Cabinet. </snark>

------
gibsonf1
The idea that raising taxes increases revenue has been shown to be untrue with
actual taxation data time and again. A good resource for cause and effect with
taxation is "Hauser's Law": <http://en.wikipedia.org/wiki/Hausers_Law>

------
Natsu
Maybe we could also put the laws into source control, be able to roll back to
a "known good" state and have a "blame" command for who introduced changes
that screwed everything up.

Unfortunately, politics is largely in the realm of emotion. And people _hate_
being proven wrong.

------
rwhitman
I firmly believe that the next major system of government to emerge in the
future will be heavily influenced from the lessons learned in governing social
media communities.

The idea of A/B testing laws seems like it would fit into this pretty well.

------
ph0rque
I was hoping the proposal would be testing laws on a virtual US, something
commented on before: <http://news.ycombinator.com/item?id=1724778>

~~~
Benjo
How do you know we're not just part of a simulation to determine the outcome
of Bush v. Gore?

------
j_baker
I would imagine now is when the Haskellers start claiming that laws wouldn't
need to be tested if they had a good enough type system. :-P

------
wazoox
Problem : most people aren't interested in reality and truth. See the the
"teach creationism" debate. It would be great, however.

------
vhackish
This would be a great way to test out new jails too - when you break a test
law, straight to the test jail with you!

------
stuaxo
I've been talking to people about exactly this sort of thing, I'd make the
trial groups bigger though.

------
giardini
At least we could do A/B testing with them! Isn't that what states are for?

------
albemuth
I thought that's what California was all about

------
ChristianMarks
Effective methods to elevate political and social discourse above the level of
ideological debate would be revolutionary. Empirical testing of proposed
legislation should be implemented as soon as possible. In controversial cases
where experimentation doesn't lead to satisfactory outcomes, thanks to
ideological intransigence, the method should be supplemented with a proposal
that I call Bimodal Politics.

Technology could be used to manage controversial political issues, for which
the distribution of voters is bimodal and for which there is essentially no
middle ground. Such issues include abortion rights, stem cell research and gay
marriage. Through the system of bimodal politics, voters would live in
parallel legal and political worlds, with different rights and obligations to
the state depending on how they voted.

Where controversial legal and political matters are concerned, the
distribution of opinion involves roughly equal and opposite numbers of
equally-informed, impartial rational voters. (Some would contend that for
issues such as abortion, a few vociferous opponents give the misleading
impression of a much larger opposition, but until this is established, we will
proceed as if public discourse reflects a genuine controversy.)

Although the voters on each side of a controversy may not see the other side
as rational or equally informed, one essential feature of controversies is
that they cannot be decided by common morality, and they have to be
transferred to the legal and political system for resolution. The legal and
political system can attempt to resolve controversial issues one way or the
other, but it is understood that the resolution is provisional and does not
resolve any underlying moral issues.

Deciding controversial issues one way or the other is inherently unstable.
What if it were possible to decide issues in parallel, in such a way that each
group in a controversial issue imposes rights and obligations only upon itself
and no group imposes its vote on the members of the other group? Is there a
procedure that would result in less political and social instability than
deciding one way or the other for all voters?

Bimodal politics attempts to provide a legal and political mechanism for
resolving politically controversial issues, with the understanding that it
provides a political decision procedure: it does not address the underlying
moral issues. In outline, a database is maintained of voter preference on
controversial issues that are designated bimodal issues. Your vote is recorded
by the bimodal voter database. Your vote determines your rights and
obligations to the state on that particular issue in parallel with those
voters who voted oppositely, and who may have (and probably have) different
rights and obligations under the state.

Consider stem cell research. Under the proposed system, stem cell research
would be designated a bimodal issue. During an election, your vote on stem
cell research would be entered into the database. If you voted in favor of
stem cell research, you may be taxed to support it, your embryos may be
harvested for stem cells (these may be from embryos slated for destruction in
any case) and if you develop a disease that requires stem cell derived therapy
for its treatment, you will be eligible for it.

If you voted against stem cell research, you will not be taxed to support it,
your embryos will not be harvested for stem cells and if you develop a disease
you will be prohibited from pursuing treatments derived from stem cell
research.

Abortion is another issue that would be designated a bimodal issue under the
system of bimodal politics. If you vote against abortion, your tax dollars
will not be used to support state-sponsored family planning programs or sex
education, and you will be legally barred from having an abortion if you are
female. If you are male and you impregnate a woman who has an abortion, and
you voted against abortion, you will be held legally liable.

In either case, if you voted against abortion and your fetus or your partner’s
fetus is aborted, you will be prosecuted by the state. However, if you voted
in favor of abortion, your fetus can be aborted, and your tax dollars may go
to support state-sponsored family planning programs and sex education.

These examples illustrate the slogan that under bimodal politics, you live in
the world you voted for.

