

Census Bureau's American FactFinder software cost taxpayers $33.3 million - wmhartnett
http://www.wmhartnett.com/2012/07/11/the-u-s-census-bureaus-american-factfinder-which-everyone-in-the-universe-hates-cost-taxpayers-33-3-million/
Did you catch the part where it said the Census Bureau paid IBM $33.3 million for the poo bucket that is the American FactFinder? Ha ha. That was my favorite part, too.
======
JPKab
The software is a "steaming pile of shit"? Of course, it was built by IBM
Federal. Building software for U.S. gov't contractors is an awful, awful
experience. The end result is the talented leave as soon as they can, because
working in a red tape environment with ridiculous Information Assurance
(that's gov't speak for their own ludicrous security theater that is their IT
security process) policies makes you want to kill yourself.

My wife has a friend who barely made it out of high school, believes in
psychics, believes that the U.S. gives an annual payment to England for our
freedom (this was revealed on the 4th. I spit out my beer laughing, but then I
realized she was serious). This person works as an Information Assurance
person in charge of making sure software is "secure." She knows nothing about
software, but she has passed a few tests, courtesy of prep courses which
guarantee you can get the cert. Because she was in the Air Force for 6 months
(she got pregnant and was honorably discharged a couple of months after her
first duty), she is a veteran and is therefore fully qualified to classify
software as secure. Or in reality, defense contractors have to meet quotas for
hiring veterans, and they put her in the easiest butts in seats job they could
find, IT security.

This, my friends, is the system you have to deal with in the federal fucking
government.

~~~
wpietri
I'm sure that IBM Federal is as bad as you suggest. But part of that is due to
how government works, which is in turn due to how citizens react to things.

Think about it like a government employee. If you do your job perfectly well,
nobody notices. Despite the bureaucracy, despite unclear success criteria,
despite insane budgeting. Nobody notices. That's just what's expected.

When something goes wrong, though, you get hammered. God help you if something
comes to the attention of the public or makes the news. Nobody will take the
time to understand the context; everybody just looks for the most plausible
person to blame. If that's you, then you've got a black mark for the rest of
your career. Welcome to the basement!

It's the total opposite of a startup context. And in some ways it should be.
But it does mean that government projects drown in red tape and politics and
procedures up the wazoo. Which is absolutely a recipe for shitty software and
overpriced contracts, whether you're in government or a megacorp.

I'm in startups for a reason, and I have a lot of sympathy and respect for the
good people who keep plugging away in government despite the fucked-up
incentives.

~~~
grassclip
If your a government employee and something goes wrong you just claim that you
didn't have enough money or resources to do the job. At least that's what
everyone seems to do and they get away with it.

~~~
wpietri
_Agencies_ can claim that. Because we never fire or sideline agencies.

But individuals don't have the same out. Somebody who took a risk and failed
may never get fired, but for somebody with ambition and vision, getting
demoted to a pointless job with no power is _worse_ than getting fired.

------
unreal37
Working in a big corporate environment which I feel is similar to the way the
government must work, you must understand that the "cost to develop the
application" does not mean what you think it means. It's not the "cost of the
development phase".

Coming up with the idea and getting it approved takes months. Wrapped into the
project of creating the web site will be the "what should we create" phase and
dozens of iterations on design and IA, which takes most of the work. There
will be 20+ people involved in this process and one line of code hasn't even
been written for the first 6 months of the project. The idea they had last
week is different than the idea they are talking about this week, which will
be different than the idea they will talk about next week. Imagine this goes
on for 6 months.

Once the code is written, testing, security, infrastructure setup and all the
sign-offs on that also takes months.

One person full time for a year will cost IBM $250,000 in real costs (salary,
benefits, overhead) that they bill out to the Federal Government at $500,000
per person per year. Can I imagine 25 people working on this for one year,
where only 3 of them are developers? And the rest of the money is hard costs
and infrastructure? Yes I can. Easily.

Sad though.

~~~
bobbles
Generally (in Australia anyway) government software also has to meet a series
of strict accessibility guidelines before it is considered complete.

Things like high-contrast modes, adjustable font sizes, naming conventions
(for screen readers, etc) all have to be taken into account on EVERY dialog.

~~~
adestefan
That's called Section 508 compliance in the US.

[http://en.wikipedia.org/wiki/Section_508_Amendment_to_the_Re...](http://en.wikipedia.org/wiki/Section_508_Amendment_to_the_Rehabilitation_Act_of_1973)

------
cobrien
I haven't researched the full contract award or pulled a copy of the original
RFP, however, work on the Census FactFinder website was part of an $89.5mm
contract awarded to IBM Federal in 2007 (for, potentially, 9 years) [1]. I
wonder if the $33mm quoted in the FOIA request response is the total cost
incurred to-date on that award (including services performed for the entire
Statement of Work) rather than the actual cost of ONLY building/updating
FactFinder.

I wouldn't be surprised if the $33mm price is correct ... but not 100% sure it
is without spending more time looking around and, potentially, seeking further
information through FOIA.

[1] <http://www-03.ibm.com/press/us/en/pressrelease/22467.wss>

~~~
wmhartnett
Good point, and thanks for digging that up.

------
Mizza
Awesome! I wrote the original FOIA that he references in the beginning of this
article, I'm extremely pleased that somebody else took the same initiative!

Is there anything else about the government any of you guys want to know
about? I'm getting pretty good at FOIA'ing now.

~~~
wmhartnett
Thanks for the inspiration! As someone just said on Twitter, "we really should
just FOIA the cost of everything with a .gov, just for fun."

~~~
Mizza
As much as I'd love to do that, I think casting a broad net in FOIA is very
difficult and potentially counterproductive. FOIA offices have very limited
resources already, so asking for literally everything at once or as lots of
little requests performs a DoS attack on the one office of the government that
I really like! It does make a good point philosophically, but practically, I
think it might be counterproductive. The FOIA law itself also requires
requests to be fairly specific.

I bet there are some _specific_ things that members of this community want to
know (cost/sources of software, policy memos, etc.) that could be requested,
received and processed in a more timely and useful fashion than if we just
asked for "(asterisk).(asterisk)".

~~~
freehunter
Or a better solution, everything that's eligible for a FOIA request should
just be made available on the existing Open Government page [1] rather than
making people request it and go through the process. What is the point of this
security by obscurity nonsense? If it can be released under FOIA, there's no
reason to not release it without a FOIA request.

[1] <http://www.whitehouse.gov/open>

~~~
sliverstorm
_there's no reason to not release it without a FOIA request._

How about the cost of digging up that information? Or the effort to ensure
each piece of information released is indeed safe for release? Or the effort
to put it all together in a coherent set?

The government has a _TON_ of records. Releasing them is not as simple as
picking the "public" checkbox in the settings pane.

There might also be a general concern that if someone had access to the
_entire database_ of records, they could mine that database and start to infer
other information that is not supposed to be known. This sort of tactic has
been used in wars past. Compiling supply chain records to infer troop
movements, for example.

~~~
specialist
I accidently upvoted you. So please mentally -1 for this post.

I'd characterize your reply as concern trolling.

I've done my share of FOIA requests. I imagine I've heard most every excuse.
Including "We lost the backup tapes."

Being charitable, the reason public records requests are hard, expensive to
fulfill is because efficient records management is rare, making finding and
retrieval difficult.

Someone may chime in suggesting CMS, workflow, sharepoint, whatever. Yea. If
it was just that easy, everyone would be doing it.

~~~
sliverstorm
I'm not sure I follow. Are you agreeing with the things I mentioned, or
claiming they are not valid?

------
jashkenas
For folks interested in helping solve the root cause of these sorts of things
... the White House is looking for people to help build a streamlined
procurement process:

<http://www.whitehouse.gov/innovationfellows/rfpez>

... hopefully making it possible for vendors build government web projects
with bids for less than $150k, instead of millions of dollars per web site.

~~~
tomjen3
Sorry, but for a hacker streamlining procurement procedures sounds like a huge
and extreemly boring timesink.

Wouldn't this be something an mba would love anyway?

~~~
wpietri
I think it would be something awesome to work on. I'm in software to make a
difference, and dropping a digit or two off the cost of government projects
could be quite an impact.

~~~
BHSPitMonkey
More money to allocate to the defense budget!

~~~
mseebach
If that's the democratically decided priority and it can be achieved without
cutting other budgets, raising taxes or running a deficit, then that's a good
thing.

Conversely, fighting a bloated military by wasting money on IT boondoggles is
not a strategy with a whole lot of long term potential IMO.

~~~
bradleyjg
Tell that to the people being bombed.

Defense my ass.

~~~
mseebach
Tell _what_ to the people being bombed? That you're hard at work changing
their fortune by refusing to reform the IT procurement process of the US
government? They will be _eternally_ grateful.

~~~
wpietri
As a USian, sometimes it seems like the high cost of war is the only reason we
haven't invaded yet more countries. The vast cost is certainly one of the few
arguments that has resonance across the political spectrum.

~~~
mseebach
Given the willingness of the US government to run huge deficits, I don't see
how fighting waste in completely different areas of government would enable
the country to go war.

Never mind that we're taking about saving millions, while the last two wars
each cost _trillions_. Or 100,000 Census Bureau websites as discussed in the
article.

------
danso
On a positive note...

I don't know how much it cost to redesign the Federal Register, but
federalregister.gov is one of the most usable, data-intensive sites around.
Especially compared to what it used to be.

GAO.gov is also a hallmark in usability and information-taxonomy.

~~~
knowtheory
One of the dudes who developed the Federal Register site (i think as a code
for america fellow) is now at AirBnB too. This is a world we can and should
get involved in.

Ah i wasn't quite right. You can check out the full story here:
<http://news.ycombinator.com/item?id=2839137> sunlight labs was involved
though.

------
cdk
For simplicity let's say that IBM charges $300/hr for a consultant
(developer/PM/etc). That would be about 100,000 hours of work. Let's say a
consultant bills 2,000 hours a year and it takes them 18-24 months to build
this thing. That would be a team of 25-30 (developers, PMs, account managers,
etc).

The time frame is realistic considering poor original design, scope creep and
federal government counterparts not being particularly quick in gettings
things done (getting access/fighting through bureaucracy, etc).

So why would the federal government hire IBM over you; who can build a totally
bug free version of this in half the time for half the price? IBM is: \- Not
likely go out of business anytime soon. \- Not be unavailable for support work
because you decided to start/join a startup \- Not be able to deal with
turnover since they have 100k+ employees, steady stream of college recruits
and have been around for 100 years.

~~~
wpietri
And you forgot the most important part. If the project goes to shit, people
can say, "But we hired IBM!" If the government employee in charge hires a
smaller company that does just as well as IBM would have, then everybody yells
at the government employee.

------
aggronn
This is disgusting. I've had to use it extensively for research, and while it
'gets the job done', its such a pain to use. I've asked myself who put this
together, and I always assumed it was a team of interns working across a
decade for each version.

I can't believe this is the best they could do for $33 million. I just can't.
They better have some slick stuff under the hood.

~~~
wpietri
Good UI comes from collaboration and iteration with users. Apple did something
like 100 iterations on the first iPod. Typical consumer electronics companies
do 3 to 5.

The government RFP and budgeting process is practically designed to prevent
iteration and exploration. I can just hear Senator Blowhard of Cornhusk now:
"So you want an unknown amount of money to keep building things until users
are happy? That's an obvious boondoggle. Come back with a complete
specification and put it out for bids from major contractors. We must be
responsible with every penny of taxpayer money. Harrumph!"

So of course the software has terrible usability. Anything built like that
will be terrible.

~~~
mbesto
> _Apple did something like 100 iterations on the first iPod._

I don't doubt you, but do you have a reference for this?

~~~
wpietri
For the Apple figure, I'm pretty sure it was _Inside Steve's Brain_. This was
some years back and I'm away from my books, though. I'm sure it was 100
iterations on some Apple device, but I won't swear it was the iPod. The 3-5
number I got from a consumer electronics designer I took a class with.

------
laironald
This isn't surprising. I've done work in the federal government. Recently we
built some products that did more, looked better and cost far less ($200k)
than the other big name consultants. Our risk for further funding is that it
didn't cost enough although most people love what we've done. The federal
government IT market works on a contracting model, which means the people
working under the system do things at a minimum OR do things to ensure that
they'll receive further funding down the line. It's incredibly time consuming
and difficult to nail one of these contracts (usually requires a combination
of relationship building, slow moving people and lots and lots of politics) so
once you get into that zone, the incentive for the contractor is to layer as
much stuff into the contract as possible.

~~~
crusso
Yet when anyone talks about cutting a dime out of the federal budget, the
cries of "You're starving poor people!" and such dominate the political
discussions.

There isn't even a little bit of common sense or honesty in the way the
government spends our money and people wonder why some of us are so adamantly
opposed to raising taxes.

~~~
_delirium
This doesn't really seem like a public/private difference, but an
enterprise/non-enterprise difference: Fortune500 firms are no more efficient
than government contracting is, really. While it's a lot in other contexts,
$33m for a custom enterprise software project, even one that seems like it
ought to cost much less, is pretty middle-of-the-road as a price.

~~~
crusso
Having worked at a company that's top 20 in the Fortune500 and having a good
friend who does government contract work... you have absolutely no clue about
what you're talking about in terms of the relative amount of waste exercised
by government vs corporate entities.

The fundamental reason why there's such a difference in efficiency between
private efforts and government ones is this:

Private companies that are horrible at managing efficiency go out of business.
People in private companies who are horrible at managing efficiency lose their
jobs.

Government agencies that are horrible at managing efficiency get bigger
budgets. Government employees who are horrible at managing efficiency rarely
lose their jobs. The GSA scandal is the only one in recent history that has
received any kind of real attention; and that's only because the idiots at the
GSA made videos the went viral.

~~~
Spooky23
I worked for a successful Fortune 50 company whose products you use every day.
I worked out of an office in NYC, and flew to the west coast every two weeks
to attend a staff meeting that lasted about 2.5 hours. I literally earned
enough frequent travel points that I didnt pay for a vacation from 2003-2009.

If you work for a company like Microsoft or IBM or Bank of America, you work
in a bureaucracy at least as dysfunctional and Byzantine as an average US
state. The Federal government is a whole other beast unto itself, but there
are probably companies as screwy as they are too.

The fact that the private entity purges some folks doesn't make them better.

------
scarmig
If you're interested in making actually good software for the government, an
agency that looks like it's genuinely dedicated to that end is the Consumer
Financial Protection Bureau.

[http://www.consumerfinance.gov/jobs/design-technology-
fellow...](http://www.consumerfinance.gov/jobs/design-technology-fellows)

For their Design + Technology Fellowship, the deadline is this Friday.

You can complain about how government does things, or you can do something to
fix it.

------
carson
From the site in question:

"American FactFinder works with Mozilla Firefox 3.6 and Microsoft Internet
Explorer 7. Other browsers may not perform as expected. "

I wonder what it would cost to get support for Chrome.

------
jevinskie
I'm curious to see what caliber/capabilities HN thinks software that costs $33
million should offer. I have experience with cycle accurate emulators that
cost $100k/floating license but never anything at the $33 million level!

~~~
GFischer
Well, the company I work for has quotes from both CSC and Guidewire for their
insurance packages (kind of an ERP for insurance companies), which would
handle all of our processes, and they ran in the "tens of millions" category
between licensing, implantation and other costs.

You can check <http://www.guidewire.com/> or <http://www.csc.com/> if you need
a multimillion dollar insurance solution :)

Guidewire at least has hundreds of people working on their insurance suite,
some of then I've written to sound very capable and I really like their
development process. They even created their own language based on the JVM
(Gosu).

<http://guidewiredevelopment.wordpress.com/>

I'm sure CSC has some equally good people, but I haven't come across them :)

~~~
JPKab
CSC was the prime contractor for the Air Force's attempt at an ERP. 5 years
and $1 billion later, there is no ERP to speak of.

~~~
GFischer
Good thing we didn't go with them, then :) .

Personally, I thought Guidewire fit our needs very well (but I'm at the very
bottom of the totem pole).

------
character
Seems like a similar situation to the student info system at my school, which
supposedly cost $8 million of our tuition money despite its terrible UI
(scrollbars inside scollbars inside scollbars) and inability to store three-
digit course numbers (making the entire university have to switch to four-
digit course numbers).

------
slurgfest
That figure is clearly not for an Android app, that is a clear
misrepresentation.

Who knows how it breaks down. This exposes a vast amount of queryable census
data to the public (something which would normally be regarded as a good
thing, particularly if you are a startup which uses data).

But, to be sure, we can just punish the government for ever releasing any data
unless it had such a high fee attached that only huge corporations could
afford to pay it.

You should FOIA some kind of itemization of how that huge number breaks down,
rather than running to the press with a sensationalized version tailored more
to partisan politics than to informing the public

~~~
schwanksta
You misread the post. He didn't query data about an Android app.

You really think $33 million is a reasonable number for building a web
interface to existing data? Please, hire me for your next job.

And I'll just leave this here: <https://github.com/ireapps/census>

~~~
crusso
> You really think $33 million is a reasonable number for building a web
> interface to existing data?

Apparently, he/she does.

I really wish I could understand why, though. Government corruption and waste
hurts us all. Why do some people blow it off so easily?

~~~
jbooth
Yeah, they should just have private subcontractors handle this stuff. Like IBM
Federal or someone.

~~~
crusso
Well, since the government is the one that set up the whole game including how
much they would pay for the work, how sub-contractors would be chosen, what
methods would be use to ensure accountability and a quality product, etc. -- I
think the target of your sarcasm is misplaced.

By the time IBM Federal got the project and could start coding, some
bureaucracy of failure had already decided that $33M was a fair price for the
specifications that they provided.

I know people who work on government projects like these. I know how the
subcontractors are chosen. I can almost guarantee that the subcontractor who
received this job had some special "in" with the decision maker on the
government side through nepotism, trips to strip clubs, cash payouts,
political connections, or something.

------
HotKFreshSwag
Does anyone know what something similar would cost a large private
corporation?

~~~
tlear
For just webapp? about order of magnitude less I think 3-4mil . Still way too
much

~~~
wpietri
That's what it _should_ cost. But I have seen large companies blow similar
amounts of money on even less useful systems. And heard enough stories from
friends to know that such fuckups are common in the corporate world.

------
ameasure
I have 2 theories for why this happens:

1) The people drawing up these contracts know nothing about software. They are
often non-technical, elderly, upper-management types that have never written a
line of code in their life. For all they know, adding a link to a webpage is a
2 week project.

2) Congress punishes federal agencies for not spending all of their budget
each year by cutting their budgets. As a result, towards the end of the fiscal
year every federal agency goes on a crap buying spree to make sure they've
spent every last penny.

------
DanielBMarkham
It's really hard to overstate how bad the government is at software
development. I've worked in several contracts with various agencies, and I
know people who still stay in the game, and the _best_ ones I've seen are ten
years behind everybody else and spending about 50x what they should to get
things done. They're simply swamped in bureaucratic molasses.

For instance, even though the "business" of the government (things like tax
collection or tracking shipping containers) uses much less data than say a
Google does, not only do they have a CIO, they have an entire battalion of
CIOs. There's a whole club of just government CIOs. Each of those, of course,
has a full staff. When contracts are paid, it usually goes through some
completely different set of people than the people who actually receive the
value.

If you're a company trying to help the government, you'd laugh your ass off if
it weren't so damned sad. Your programmers -- and you gotta love this -- are
all considered little cardboard cutouts, like cookies. If you have a problem
to solve, the correct answer isn't what technology you would use to solve it.
Usually that's outside your control anyway. The correct answer is how many
"standard" programmers it would take to get it done. The quality of a
"standard" programmer varies, but it's usually somebody with a few weeks of
training and a desire to be anywhere but here. Many times your government
partners are incompetent and a danger to themselves and others when it comes
to technology. They want the moon but they don't want to take any political
risk at all. I had one guy just flat out tell me: you guys make all the tough
choices and if it goes well I'll take all the credit. If it fails it's all
your fault. At least he was honest.

I could go on, but I don't want to get into a rant. Ever wonder how the IRS
spent billions on upgrades with nothing to show for it? Or how the FBI's new
case management system was a total Charlie Foxtrot? Geesh. 33.3 million is
getting off _lucky_.

~~~
yen223
Ex-employee of Accenture here. Accenture basically earns most of their money
selling horrendously overpriced "solutions" to customers who don't care about
cost.

The thing is, the bureaucracy that you mentioned, it's not just the
government. Too many large companies, especially non-IT ones, assume that you
can complete a project by throwing 100 mediocre programmers at it.

~~~
DanielBMarkham
Your point is well taken. It is not just the government.

The thing is that most private systems operating like this go broke -- unless
they have lots of federal protection. You can't spend billions in the private
sector without people noticing. The same is not true in the government sector.
While all big organizations are like this, the government wastes money on IT
at another level entirely. (I know, I've worked with lots of both government
agencies and large organizations)

------
jcromartie
On the other hand, we're doing some public-sector development, and it's fixed-
cost and the end result will be public domain.

~~~
sp332
Maybe you're just not very good at negotiating?

------
pcrh
Interesting use of the gif there. I wonder if there will be a migration of
these gifs from meme-based humor sites to more "legitimate" blogs and news
sites. They are, after all, the visual equivalent to using quotes by famous
people.

~~~
alexismadrigal
Sheeeit. We've had animated gifs on TheAtlantic.com HOMEPAGE since like 2011.

~~~
pcrh
Well... ain't that a thing!

But, seriously, someone is going to write a thesis on gifs one day. It is
"literally" an innovation in communications. While we currently associate gifs
with memes, etc, you can see quite a few very creative ones on ffffound.com,
for example. And,as you say, TheAtlantic now uses them.

------
benthumb
My calculations give me approx $0.29 / household... I'm not sure I understand
the fuss.

~~~
benthumb
AND if IBM Federal built it, why is the government being blamed exclusively...
government incompetence is an overhyped meme.

------
TheAmazingIdiot
Can somebody explain please what makes FactFinder so horrible (well, aside it
was made by the federal govt)?

