

We've got thousands of pages of contracts with Booz Allen: Help us analyze - morisy
https://www.muckrock.com/news/archives/2013/jun/25/booz-allen-hamilton-contracts-help-analyze-thousan/

======
patdennis
People get excited about things like this and say things like "this is REAL
investigative journalism". Or "I wish mainstream journalists would start doing
this".

Look, I read documents like this all day long. So do investigative
journalists. It's just that very few people care about the stuff we find until
it turns into an international spy thriller.

Just because you don't care about it on the state/local level doesn't mean we
don't put in the work.

Edit: You can look at some of the great investigative journalism you're
probably not paying attention to here: [http://www.ire.org/blog/extra-
extra/](http://www.ire.org/blog/extra-extra/)

~~~
morisy
"I wish mainstream journalists would start doing this".

Spin it however you want, fewer and fewer mainstream journalists are able to
focus on this. I work with a dozen or so newsrooms a year on FOIA, CAR, and
other investigative reporting training, and in most newsrooms only 1 or 2
hands go up when I ask who has filed a public records request in the past
year.

MuckRock was built to change that: To make it easier to add document-
requesting and analysis to the daily news cycle, to help non-journalists get
interested, excited, and able to request and share documents, and to enable
large scale-FOIA projects like request BAH files from dozens of agencies.

And we've found that lots of people _do_ care about it at the state/local
level, but news organizations are a) terrible about marketing their work. b)
assume readers aren't interested in the nuts and bolts. c) let that work die
in obscurity, content to try to win awards with it rather than to truly engage
the public.

So yes, this is a flashy dump of documents, but we've helped hundreds of
others do meaningful, smaller scale stories, and whatever way you cut it, I
don't think there's enough groups doing that.

I also keep my own list of interesting, public-records driven stories:

[http://pinboard.in/u:morisy/t:to_foi/](http://pinboard.in/u:morisy/t:to_foi/)

Encourage anyone who is interested to reverse engineer them and try those
requests in their city, state, and country.

~~~
patdennis
Don't get me wrong, I love Muckrock! Wasn't trying to say you guys aren't
necessary. I file FOI requests for a living, and I'd love to be in touch with
you guys about a number of things - best practices for turning government
records into structured data, the importance of reforming records retention on
the government side to reduce the burden on public records officers, fixing
FOI on the local level, etc.

Feel free to get in touch with me on Twitter: twitter.com/patdennis

------
rdl
This is the kind of stuff I wish Wikileaks had stuck to -- mostly or
completely legal, beneficial to national security, etc. They are still
challenging powerful entities and supporting change, but without becoming the
story themselves.

~~~
mpyne
Likewise. I really liked WikiLeaks at the beginning but it became clear after
some time that there was a definite agenda behind their actions. The
impression I've had for years is that they will oppose the U.S. for no better
reason than that it's the U.S. Which is, if you think about it, just like
opposing any maligned group for what they are instead of what they _do_.

In other words if we start from the assumption that the U.S. should be a
"bastion of the free world" then it would make sense to tirelessly note
instances where the U.S., corporations, states, etc. fall from that ideal.

But diplomatic cables describing what kind of women Berlusconi likes, while
interesting, are hardly indicators of the collapse of a free society. Those
types of things were released because they were embarrassing, nothing more.
And while I'm certainly glad that Assange is able to stroke his ego by
thumbing his nose in Uncle Sam's face, I don't actually _hate_ my country; I
want it to be better.

~~~
Amadou
I have to strongly disagree. Wikileaks has released minutiae about other
countries as well. It isn't only about showing up hypocrisy from the US, it is
about lifting the veil of secrecy that all governments misuse whether it is
hiding big stories or trivial ones.

Saying that wikileaks has an agenda to oppose the US is basically confirmation
bias. Leaks about the US get news coverage in the US. Just for starters, look
at the 2 million emails from Syria - clearly most of those are about trivial
matters too, but most people in the US haven't even heard about the release at
all.

~~~
mpyne
> I have to strongly disagree. Wikileaks has released minutiae about other
> countries as well.

Doing bad things to _other_ countries (again, because of what they are instead
of what they've done) doesn't make me feel better about when it's done to
mine.

Here's some other countries I don't hate: Russia, China, the U.K., France,
South Africa and in fact the list could go on as long as I have time.

> Just for starters, look at the 2 million emails from Syria - clearly most of
> those are about trivial matters too, but most people in the US haven't even
> heard about the release at all.

There's no polite way for me to say this so I'll just spit it out; _no one
cares_ about Syrian government emails. That's why no one in the U.S has heard
about the release. Gossip is as destructive for international politics as it
is for small towns.

If there's big news to release then by all means go and do so, but governments
should be able to maintain diaries too.

~~~
Amadou
I'm having a hard time understanding your point of view, it seems to have no
unifying principle. But if I take your last sentence as what you really
believe then the obvious response is that once governments can be relied on to
not abuse their ability to keep secrets then they will have earned the
privilege to maintain their diaries in secret.

~~~
mpyne
And anarcho-libertarian types will claim that a government can never be relied
on to do that, which is just as well as saying again that the government could
not be allowed to keep secrets.

~~~
Amadou
Since I'm not an anarcho-libertarian type and you clearly are not, I don't
think what they might hypothetically say matters all that much.

------
morisy
A good way to search through the text of the documents is DocumentCloud, which
also offers some analysis tools:

[https://www.documentcloud.org/search/group:muckrock](https://www.documentcloud.org/search/group:muckrock)

Modify the search terms as desired (i.e., "Booz")

------
stfu
Is there already some decent crowd sourcing text analysis software out there?

Some simplified version of Nvivo with a wiki-like user management would do
wonders for something like that.

1\. Button "I commit to analyze 2 / 12 /22 pages". 2\. You get a very short
tutorial on how to do it. 3\. User is assigned relevant pages, starts tagging
the documents for specific content and highlights + comments lines worth a
closer look.

I would be definitely up for that. With enough traction one could just compare
inter-coder reliability and establishing a high quality analyzing process
(i.e. getting the same document analyzed by multiple users).

~~~
solstice
How about django-public-project? The following is from their description at
[http://django-public-project.readthedocs.org/en/latest/](http://django-
public-project.readthedocs.org/en/latest/):

 _Django Public Project (DPP) is a specialised content management system
written in Python /Django for building an information website around big
projects. It is open source (BSD licence) and can be used by civic groups to
monitor the progress of publicly funded projects but also from within
governmental institutions itself to bring transparency in the proceeding of a
project, inform the interested public about what is going on and thus enhance
the acceptance of the project by the different stakeholders._

It is currently being used by the investigative comittee of the Berlin city
parliament in order to investigate the massive clusterf __k that is the city
's new and very disfuntional airport (BER). The source code can be found at:
[https://github.com/holgerd77/django-public-
project](https://github.com/holgerd77/django-public-project)

Regarding participation of ordinary citizens, the docs say:

 _On DPP websites users can comment on nearly everything and connect their
comments to various elements of the project. Comments are then shown alongside
every connected element. All admin users having activated the “Receive new
comment emails” in the admin user settings receive an email after someone sent
a comment and can then activate the comment to be shown on the website._

[http://django-public-
project.readthedocs.org/en/latest/users...](http://django-public-
project.readthedocs.org/en/latest/users.html#user-comments)

------
jashkenas
If anyone wants to search through MuckRock's recent documents via the
DocumentCloud interface, they're also available over here:
[http://www.documentcloud.org/public/search/group:muckrock](http://www.documentcloud.org/public/search/group:muckrock)

 _Edit_ : Whoops -- I see that Michael's already commented along the same
lines.

To expand a bit, you'll be able to do boolean searches among the documents,
scan through excerpts around your search query, look at extracted entities
(people, places, company names), and plot any dates mentioned in the documents
on a timeline.

------
scottshea
Now this is investigative journalism!

------
patrickg_zill
[http://taxanalysts.com/www/features.nsf/Articles/557540D8F53...](http://taxanalysts.com/www/features.nsf/Articles/557540D8F539A26685257B94004CD0D1?OpenDocument)

I think this might provide some interesting background.

------
Spooky23
What exactly is the point of this exercise? Booz Allen is a big consulting
company. They have lots of contracts and SOWs, most of which are for really
lame things. You are not going to find a contract that says: "provide 3 big
brother spies to read everyone's mail".

Of you're trying to prove that that government spends lots of money for
consultants... Well, we all know that already.

Also, unless you really know what you are talking about, you are not going to
be able to spot most of the petty, obvious or really offensive contracting
practices. The Feds have their own language re: procurement.

~~~
rdl
Based on the ones I've seen in other contexts, the key is to be able to
aggregate. If you do, you can often find really interesting stuff from
unclassified contracts around the periphery, then presumably FOIA more, but
knowing how the system works helps a lot.

------
rhizome
They should be using plagiarism-detection software to strain out the
boilerplate commonalities and focus on the differences (as well as a single
copy of the commonalities).

