
Open data is not a panacea - chrismealy
http://mathbabe.org/2012/12/29/open-data-is-not-a-panacea/
======
AnthonyMouse
I think the premise is mostly true (people with resources can use data more
easily than those without), but I'm not whether the conclusion follows. The
alternative to open data is closed data. That doesn't get the unwashed masses
access to data engineers, it just loses them access to data that corporations
and wealthy people continue to have.

If I was to point out a problem with open data, it's that it has the potential
to break our social institutions by facilitating aggressive min/maxing
behavior on a large scale. It's not that Goldman Sachs is going to use the
data against the general public -- they're going to do that whether the data
is public or private -- but you make it public and then the middle class can
use it against the poor (and then the poor are going to revolt).

For example, suppose we create a good, easily accessible map of places to live
and put down all the information someone in the housing market would like to
know: Home prices, tax rates, quality of the schools, etc. Now home buyers can
min/max those variables when making a purchasing decision. In general that's
good -- when we buy a home, having that information benefits us. We can find a
place that has extremely low taxes when we don't have kids in school, and
choose between the places with excellent schools to find the one with the
lowest taxes when we do.

But then local governments react by trying to find a niche. They let their
schools go straight to hell in order to attract childless professionals
through low tax rates. Or they zone all the low cost housing out of the school
district so that they can maintain good schools at a low mill rate because
everyone who can afford to live there is affluent. What they most certainly
don't do is provide good schools or other government services to poor people,
because that has a deleterious effect on the value for money that middle class
people are looking for when they're shopping for a tax jurisdiction, and when
they do that they lose their tax base and thereby their ability to continue
doing it.

Of course, such things happened before public data, but you make the data more
widely available and you make the behavior more common.

And before the libertarians get too happy about this state of affairs, recall
that the de facto default defense against aggressive competition is collusion.
Which we [pretend to] ban in the private sector, but between governments they
just call it centralization and "federal standards" and impose it on you from
the top down the second they can break a filibuster. Which is, of course,
miserable and inefficient because it destroys local control and creates a
systemic failure if they get it wrong.

Which isn't to say that the data availability is net negative. The incentive
it provides is for governments and corporations to be efficient, and
efficiency is _usually_ (indeed almost always) beneficial. But the data lets
the user of the data decide what to optimize for. So the problem comes when
the thing we choose to optimize for as individuals is not the thing we would
choose to optimize for as a society.

~~~
quanticle
Another point (raised in the comments to the original article) is that open
data, like "full disclosure" security practices, levels the playing field. In
other words, investment banks, city governments, and who have you already are
doing these sorts of things. Open data allows us, the general public, to _see_
them doing it. Yes, in your dystopia of Hobbesian all-against-all competition,
we end up in an _Snow Crash_ -esque agglomeration of specialized communities
for increasingly narrow demographic segments.

But there is an alternate future. With open data (and increasingly cheap
computing and communication devices), community organizers and the poor
themselves can see exactly how they're being screwed over by the government.
It will no longer be a nameless, faceless _they_ that's inflicting pain and
misery on them, but instead, specific actions, undertaken by specific
individuals in specific offices. Corporations and governments won't be able to
hide behind their usual practice of security by obscurity, since, with open
data, it'll be possible for any citizen with a modicum of spare time and spare
computing resources to run data analysis and come up with possibilites for why
things are as screwed up as they are. We're already seeing this, with tech
related laws and regulations. Open data makes it easy for any individual with
an internet connection to track the progress of legislation. This was a major
factor in galvanizing opposition to SOPA and PIPA. Making more data sets open
and freely available will, in my estimation, benefit individuals far more than
corporations or governments. Open data will do to closed-door decision making
what full disclosure did to 0-day vulnerabilities.

~~~
AnthonyMouse
>It will no longer be a nameless, faceless _they_ that's inflicting pain and
misery on them, but instead, specific actions, undertaken by specific
individuals in specific offices.

You're assuming that there are actually specific fat cats smoking cigars in
back rooms planning all of this out. That pretty much never happens.

What actually happens is that different tax jurisdictions have different laws
and people self-select into the jurisdiction that provides them personally
with the best value in government services for their tax money. That means the
affluent taxpayers go to the jurisdictions that take only a modest tax bite
and use it to provide only services that the affluent consume. The
jurisdictions that don't do that are then forced to cut services after their
tax base is eroded by so many of the high income taxpayers leaving.

The local officials in those places are not sitting in their offices trying to
think up ways to screw over poor people, they're just trying to figure out how
to keep the lights on. They have to do something to attract and retain more of
the taxpayers who pay as much or more in taxes as they consume in government
services or they fall into the death spiral of service cuts and tax increases
that induce more emigration and capital flight which erodes the tax base and
requires more spending cuts and tax hikes.

------
tspike
Similar complaints were lodged against open source software when it was
emerging, especially with respect to security. I assert that open source has
been better for security than ever imagined and nearly always produces
solutions superior to closed alternatives.

I do think it's important to draw attention to the potential downsides, so
thanks for contributing this article.

Given the upside of open data I can't imagine a scenario where closed data >
open data with respect to the general public.

------
d4nt
I agree there are dangers, but I think the way to protect the poor or
technologically unskilled is not to keep data sets locked away where only
government officials, illegal hackers or bribe paying business people can get
at it. The answer is to get it out there and ensure all interests have
information parity, surely.

If you have to be a techie to get value from this data then there must be a
market for easier to use data analysis tools for non-technical people, which
I'm sure the market will fill. (Checkout my profile page, I'm building one
right now).

------
lampe
I think this is not a big point to dont make data available to everyone.

making data available to everyone for free is the better way.

1\. If i got the data i can make a list of strange companies and put this
information out to the Internet.

2\. How says that this Information is correct? you can scan my facebook
profile what you will find: that i live in Australia but i dont. Yes my Ip
give it away where i live but i can use a proxy. I like this and i like that
but do i Really?

3\. School/University Rankings are full of facts that are wrong and dont cover
the reality.

Sry but if you only see data in the aspect to analyze it you could miss the
beauty of the data. MP3 Files are just data but what makes them special? the
noise we get out of the speaker.

So lets not think how we can abuse the data. Lets think about how with this
data we can make a better world FOR EVERYONE !

------
dbecker
The "information war" seems like a strange way to look at the world.

I use data, and I benefit from access to data. But my use of data doesn't make
someone else worse off, and others' use of data doesn't generally make me
worse off.

If they have faster computers than me, that's great.

Equating research and analysis to war sounds silly, and I think it's a really
unhealthy worldview.

------
njyx
it was interesting that when we tried to pull together the best blog posts on
APIs across the web this year (<http://www.3scale.net/2012/12/top-10-api-blog-
posts-2012/>) the strongest theme was exactly about this - the push and pull
of private data and platforms.

