
The staggering impact of IT systems gone wrong - lukebennett
http://spectrum.ieee.org/static/the-staggering-impact-of-it-systems-gone-wrong
======
rabbiterm
I clicked on this curious as to whether I had worked on any of these. And yep.
3 of them -- 3 out of the top 4 worst in the Americas. The DoD has some
successes though.

~~~
ep103
well, and this is just a personal suggestion, maybe you should stop sabotaging
the companies you work for? Or hire yourself out for your ability to sabotage
competitors?

: D

------
i_feel_great
The Queensland Health payroll failure is now estimated to be $AU1.2b all up,
with nothing to show for it.

~~~
joshuarrrr
One of the authors here- we have another graphic coming about Queensland
Health. The craziest part was when IBM estimated it at $6 million.

~~~
i_feel_great
$6m to $1.2b cost overrun. That must be some type of record.

~~~
joshuarrrr
Yeah, it seems like it. You can see how the budget ballooned here:
[http://spectrum.ieee.org/static/the-life-cycles-of-failed-
pr...](http://spectrum.ieee.org/static/the-life-cycles-of-failed-projects)
(select the 5th project)

------
dijit
But there is a profession dedicated to preventing these kinds of failures:
Systems Administrators.

Proper infrastructure engineers (or "site reliability engineers" as google now
calls them) are key for helping architect highly available systems like this;
Yet the trend seems to be forcing software engineers follow tutorials from
google on setting up key pieces of infrastructure nowadays.

 _Edit:_ it's ettiquite to say why you're going to downvote someone, if what I
said wasn't relevant please let me know.

~~~
mannykannot
I did not downvote you, but I disagree with your premise. Systems
administration is only part of the problem, and only in some cases. A non-
inclusive list of other causes include failing to understand the intended
purpose of a system and the environment it is to operate in, and losing
control of the design.

~~~
dijit
I would agree with what you said, but I would argue that the points you raised
still fall into the realm of systems administration.

Identifying scope and purpose, and it's impact and documenting the design are
just some of the things an platform operations team is -supposed- to take care
of.

~~~
mannykannot
If you expand the definition of systems administration to include everything,
it doesn't get you any closer to a solution.

The largest problems of development have nothing to do with administering
anything (edit: for example, bad design often results from bad design
decisions that are neither prevented nor fixed by seeing that the required
documents have been written.) The idea that Winslow Taylor's principles can
solve these problems is a fallacy - a commonly held one, but a fallacy
nonetheless.

------
jmnicolas
I believe that these complex multi-billion $ projects fail for 2 reasons :

\- corruption : "let's milk the customer for all they have"

\- personal ambition : "let's choose what's best for my career instead of
what's best for the project"

Unless we radically change the way we do things the IEEE will need bigger and
bigger green circles on their charts.

~~~
awakeasleep
You're definitely falling prey to a short-sighted and overly cynical point of
view.

"Never attribute to malice that which is adequately explained by stupidity"
and never use the stupidity explination to overlook the fundamental complexity
of a software project.

What you have here is a lot of non IT professionals, without siginificant IT
experience, attempting the most complex IT projects ever considered, without
the pressures of ship-or-sink, investors looking for progress reports, or even
the benefit of working with their own employees (usually multiple groups of
contractors work on these big projects). Also, they're not trying to build a
MVP and iterate. They're trying to build Rome in a few years.

When you think about the hurdles these projects face, of course they fail. The
government trying to do a huge software project is like a kid seeing a master
level parkour video on Youtube and not understanding the lifetime of practice
that went into the sport. They're just going to jump off the roof and break
their ankle.

Of course in a sample size this large corruption is an issue. However, with
zero corruption and only the best intentions, we'd see substantially the same
results.

~~~
tomaskafka
Also, “Never attribute to stupidity that which is adequately explained by
complexity.”

[http://www.cennydd.com/blog/its-not-what-you-
think](http://www.cennydd.com/blog/its-not-what-you-think)

~~~
devonkim
Only those that haven't worked in DoD / federal systems could ever think that
laziness / stupidity is _not_ a major root cause of so many DoD IT project
failures. There's a lot of successes when using smaller, elite contractors but
complexity intertwines with stupidity as larger contractors must be brought in
to handle the bureaucracy and their hiring standards are basically abysmal.
Why? They were very much incentivized to hire butts in seats on contracts
historically, not to be effective at executing them to technical satisfaction.

Someone needs to call out DoD for what it really is in the US - federal Public
Works program that's approved by neoconservatives.

------
ArkyBeagle
1st Lt. Milo Minderbinder: Is it really that bad? Yossarian: It's cotton! 1st
Lt. Milo Minderbinder: They've got to learn to like it!

