
Why NASA Projects Miss Deadlines and Blow Budgets - rbanffy
https://spectrum.ieee.org/riskfactor/aerospace/space-flight/gao-warns-of-deteriorating-costs-and-schedules-in-nasas-major-project-portfolio
======
whatshisface
> _NASA managers have come to believe―usually correctly, Martin said―that
> “projects that fail to meet initial cost and schedule goals will receive
> additional funding and subsequent scientific and technological success will
> overshadow budgetary and schedule problems.”_

There's something else hidden behind this statement: NASA managers have also
come to realize that proposals that under-estimate budgets are likely to look
more attractive than proposals that accurately or over-estimate them. As a
result, the equilibrium shifts and a certain percentage gets taken off the
estimation for everything.

~~~
AngryData
Yup, if you need 10 million for a project, you quote 3 million and ask for
more funding later on, otherwise you will never get any funding at all as they
balk at the true costs of research and development.

~~~
sgt101
Where I work this gets you fired.

We have a benefits tracking requirement, you put in a 3 gets you 9 case
(borderline fundable - probably too small and not sufficient payback, but you
might get it) and they go back and ask for another 7 over 2 years and you will
leave on the ask. If you put 3 gets you 30 and then ask for another 7 you may
well leave on the ask, you may get to stay for the realisation of the 30, but
then you go. Probably you go on the ask because someone else will be
parachuted into the project (if it's believed that the 30 really is on the
table).

~~~
buro9
I worked in a place (a bank) where an estimate that was more than 10% out in
either direction would get you fired. Their view was that they valued accurate
estimations that highly.

But when I was there it was already being worked around... managers would
create lots of smaller projects that delivered cross-functional behaviour to
multiple projects and allowed them to externalise and hide the overages in
what might appear to only be dependency lines to existing internal projects.
The managers of multiple projects would collude to sponsor each others
externalities so that they all had a layer of protection from being fired,
obscuring what the costs were actually contributing to.

The reason I was there was precisely to build a revenue recognition type
system for their project management system to try and identify which projects
had asked for some other project to start.

But after that I worked at a UK govt department and they'd become so good at
tracking who had been first to demand the creation of another project that
their problem became one of legacy... no-one could sponsor even upgrading
their long obsolete software or hardware, because even though it was a thing
that benefited everyone, the project that demanded it was going to be the one
that paid for it (literally paying for the certification, hardware, software,
training, etc) which meant if you were the team that said "We can't support
this on IE6" then you had to pay to replace laptops, OS, to get onto a later
version.

My conclusion... if we don't embrace a pragmatic reality of what things cost,
if we try and manage the costs tightly... the side effects usually have
unintended consequences that are worse than facing or embracing the original
costs.

~~~
Someone
_”I worked in a place (a bank) where an estimate that was more than 10% out in
either direction would get you fired.”_

If so, I would see only two rational approaches: find another job, or hugely
overestimate, and, unless things go very wrong, twiddle your thumbs for most
of the time (or think of/work on a side project or study)

It’s very hard to estimate even the simplest ‘projects’ within 10%. For
example, try estimating how long your trip home or your next bathroom break or
coffee machine visit will take, and see how often you get within 10% (if you
say “5 minutes” that gets you 30 seconds either way)

~~~
AtlasBarfed
I'd overestimate and then invent work to get to 5% under budget. You can
always test more.

------
tomohawk
The kind of engineering you have to do to make a project successful at a place
like NASA is very different than many other places. If it's a larger program,
you need to make sure you have participation scattered across congressional
districts, or at least certain important ones. This reduces defunding risk,
but introduces lots of program management risks.

Once you've successfully become known as a jobs creator, then your job is to
spend as much as possible. You can't be looking at the efficiency of getting
the program completed - you have to be looking at the efficiency of spending
taxpayer money as efficiently as possible in congressional districts. The less
work performed per dollar, the more efficient you are in getting the money
into the economy. Hiring an engineer at $300/hour instead of $100/hour - why
risk getting the cheap guy when there is no benefit to you?

Efficiency in government programs has a very different meaning than in other
programs.

------
acidburnNSA
Important to note. A lot of complex systems people turn to the NASA systems
engineering handbook [1] thinking it will solve all their cost and schedule
issues. It may help and is insightful. But there's no silver bullet.

[1] [https://www.nasa.gov/feature/release-of-revision-to-the-
nasa...](https://www.nasa.gov/feature/release-of-revision-to-the-nasa-systems-
engineering-handbook-sp-2016-6105-rev-2)

~~~
iancmceachern
Thank you for posting this, ordering my copy now!

------
forapurpose
My projects are very hard to budget and schedule accurately, especially to the
degree they involve skilled labor, and my projects tend to be much smaller
than NASAs, with far less cutting edge tech and development and with much less
demanding requirements.

Can any large organization reliably and accurately budget, schedule, and then
implement large, complex projects? Which ones? How do they do it?

~~~
iancmceachern
This was my thought.

Could it be that NASA takes on the toughest, most daring and most technically
complex projects around?

On the medical device projects I work on delays, overruns and issues come into
play often. I can only imagine if the medical device I was working on not only
had to not injure the patient, but do it in a vacuum, survive the radiation
and survive launch!

~~~
forapurpose
And don't forget the 'serviceability' of your product in the Kuiper Belt.

------
greeneggs
"Too big to fail" makes it sound like a fixed-cost fallacy; we've spent $X
billion on this project, so we had better not cancel it now!

But in fact it could also be the opposite of the fixed-cost fallacy. Project X
is worth $1 billion, and is funded for $1 billion. A year later, the $1
billion is spent, but another $1 billion is needed to finish. At this point,
your choice to spend $0 and get nothing or spend $1 billion and get the
project done. The choice is obvious; we've already decided the project is
worth $1 billion.

The problem is that a year later you face the same choice again. And then
again. Provided the marginal increment is always $1 billion, you'll always
agree to it, because the project is worth that.

~~~
Eridrus
Running projects where you only make decisions when you run out of money seems
like the sign of incredibly poor management.

Once you start noticing that budgets/timelines are slipping you can start
revising your estimate. If revised estimates don't start converging, you
basically don't actually know how much the project will cost, and saying it
will cost another billion is just straight lying.

~~~
CamTin
"Running projects where you only make decisions when you run out of money
seems like the sign of incredibly poor management."

You're right, but this is the kind of management that the U.S. Constitution
requires. The Executive branch "runs" the program, but the people allocating
money (Congress) are only asked for input when the cash box is empty.

In a smaller (say, pre-Civil War-sized) country with a much much higher
(constituent/representative) ratio, and with far fewer things for the Federal
government to keep track of, Congress could reasonably exercise oversight of
all or most of the things that they had appropriated money for. When 535
people are trying to oversee the largest, most expensive, most geographically
distributed organization ever, in the history of the world, it all breaks
down.

The best a legislator can reasonably do is, when somebody comes to Congress
for a top-up on their bank account, to farm out a staffer to read some reports
about a project and make a suggestion about whether it should get more money
or get cut off. Most of the millions of line items in the Federal budget don't
get even this much attention.

~~~
Eridrus
Sure, there are a whole host of reasons Congress is bad at this, but there is
nothing rational about repeatedly authorizing $1bn in funding for something
with a projected benefit of $1bn.

Congress can't manage it, but in an ideal world they could demand good
management practices and refuse to fund things that go over budget without
evidence of good management.

~~~
CamTin
Yeah, not in this specific example, but under certain (somewhat artificial)
circumstances, it could be.

Imagine that your car doesn't start and you have it towed to a 100%
trustworthy mechanic. He says that he knows what is wrong and it will cost you
$500. You need the car, and $500 is far less than what it would cost you to
replace the car. You authorize the mechanic to fix what he thinks is wrong for
$500.

He calls you the next day to say he did the $500 worth of work but, just as he
was finishing, he found something else wrong. The car won't work at all
without this new work. It also costs $500. Because it is still worth more $500
to you to have a working car, you authorize this work as well.

This can continue indefinitely, even far past the point where you could have
just bought a new car with the money you spent fixing this one. At every point
along the way, the most prudent-, rational-seeming thing to do was to spend
the $500, but this ultimately leads you to ruin: no working car, and your
savings totally depleted.

This outcome is similar to that in the (also artificial) circumstances of a
dollar auction
([https://en.wikipedia.org/wiki/Dollar_auction](https://en.wikipedia.org/wiki/Dollar_auction)).

Still, many things is real life approximate this dynamic, if not exactly, then
in part. It's valuable to recognize the pattern so that you can recognize and
short-circuit it in situations where you would otherwise "rational" yourself
into a corner. In a dollar auction, you can pick the optimal move, which is
not to play at all. In certain real-life circumstances, that isn't really an
option.

~~~
Eridrus
The rational thing to do is to not be limited by the go/no go choice and seek
more complete information.

This game illustrates the limits of blind cost-benefit analysis, but that is
not the entirety of rational thought.

------
mcguire
" _Another reason NASA management plays down a project’s technical complexity
and risk is to get the project sold, Martin noted. Once sold, a project almost
immediately becomes too big to fail._ "

One example: Ares
([https://en.wikipedia.org/wiki/Ares_I?wprov=sfla1](https://en.wikipedia.org/wiki/Ares_I?wprov=sfla1)).
Originally sold as reusing the shuttle's SSRBs, it was too heavy. So they
said, "we'll just add another segment to the SSRB" and didn't mention that
that would mean a complete redesign and effectively a new booster.

I work as a NASA contractor on enterprise software and see the same issues:
things are sold as new, exciting, cutting edge, fast, and easy. Then they turn
into a fiasco.

------
woodandsteel
On the other hand, deep space probes like the Mars rovers seem to be generally
on time, and at least reasonably within budget (and fabulously successful
scientifically). I wonder what is making the difference.

~~~
wolf550e
NASA Science Directorate, which operates Mars rovers and sends planetary
probes, is mission driven. NASA human spaceflight is a jobs program (it
invents make work in order to excuse spending money) that is fatally afraid of
taking on a project they might fail at, because that might endanger the spigot
of endless government money for all the pet technologies (for the people
inside NASA) and jobs in districts (for the funders in congress).

See this talk by Dan Rasky (inventor of PICA heatshield) for a perspective
from the inside:
[https://www.youtube.com/watch?v=g3gzwMJWa5w](https://www.youtube.com/watch?v=g3gzwMJWa5w)

See almost any talk by Robert Zubrin for a perspective from outside, for
example:
[https://www.youtube.com/watch?v=EaNKtIY6JpQ](https://www.youtube.com/watch?v=EaNKtIY6JpQ)

~~~
jonhendry18
"is fatally afraid of taking on a project they might fail at, because that
might endanger the spigot of endless government money for all the pet
technologies "

And people could _die_

~~~
rbanffy
A lot of projects fail without taking human lives with it. Ares was dropped
after one single test flight of a simpler Ares I.

Arguably it shouldn't even have been attempted - the sheer stupidity of the
whole idea of having a single large solid rocket propelling you up without the
possibility of being throttled down and with the chance of having burning
solid fuel raining on your parachute in case you needed to bail out was
probably swept under the rug too many times for comfort.

And that clusterfuck was stopped before it'd be able to kill anyone. In the
meantime, it produced some knowledge on large solid fuel 1st stages that may
someday compete with SpaceX's tech.

~~~
wolf550e
This is only re: your last sentence.

It won't, solid rockets are inherently limited in their safety (for the
reasons you pointed out), fuel efficiency and ability to be rapidly reused.

The fuel efficiency will never be that of kerolox, methalox or hydrolox
engines [1].

Landing a solid the way Falcon 9 lands is impossible (there is no engine on
the way back, and if you can shut it down safely and then relight it then you
can't throttle it), and when you recover the parts of the rocket that are not
fuel, "refueling" solid rocket is basically re-manufacturing (the STS boosters
were re-manufactured, without any economy).

In general, the use of solid boosters for crewed NASA missions is a folly that
was mandated by congress to funnel money into ATK (manufacturer of many
military solid rockets and explosives from hand held to ICBM/SLBM, now
Northrop Grumman). Ares I, Liberty and OmegA, and the SLS boosters, are just
more of the same.

1 - [https://en.wikipedia.org/wiki/Solid-
propellant_rocket#Perfor...](https://en.wikipedia.org/wiki/Solid-
propellant_rocket#Performance)

~~~
rbanffy
Indeed. The SRBs are a stupid idea. Someone must build a decent rocket factory
in the same congressional district so that NASA can make safe rockets.

It makes sense if refueling is just filling them with fuel in an automated way
(think a giant extrusion device printing layers of fuel inside a rotating
empty rocket). That way, the cost of flying a rocket becomes downtime+fuel.

------
kk58
On studying the joint cost and schedule analysis document highlighted in the
article. It is evident why NASA has problems. The estimation of cost and
schedule delay distributions is subjective and not Bayesian. This leads to
outcomes severely influenced by optimism bias.i work on similar problems for
an energy company and it's clear that Bayesian methods with broad based
machine learning models can rectify a lot of these issues

------
sytelus
TLDR; _The GAO identified the culprits as “risky management decisions,
unforeseen technical challenges—some avoidable and some not—and workmanship
errors.”_

Looks like nothing we didn't knew. NASA wants very high success rates on
missions which leads to spending enormous sums to drive down probability of
failures. On the other hand, Musk can blow up rockets dozens of times and keep
moving at a fraction of the cost. The second biggest factor is simply the
unknowns. It's not that we have launched giant telescopes in space dozen times
and have became very good at predicting costs.

------
rurban
One should add that this didn't happen with the old NASA in Alabama under the
germans. This only started with LBJ ditching the Apollo team, the liquid
rockets, and splitting up NASA into Houston and California under US
management. It went downhill since then.

The NASA never really was about space travel, because then they would have
kept the Apollo tech and liquid rockets. But military needed only solid fuel
rockets so they switched and developed the shuttle. And now they have no
proper tech anymore at all.

~~~
bryananderson
This is a wholly inaccurate picture of NASA’s organization then and now. The
centers in Alabama, California, and Houston were all subsidiaries of NASA
headquarters in Washington, and remain so today. Control was never held by the
Alabama center, nor was it transferred to any of the other field centers. Von
Braun was the director of Marshall (the Alabama center) throughout the Space
Race until well after LBJ.

~~~
Herodotus38
Thanks, I thought the OP's post was odd, as I'd never heard of Alabama being
the center, and I've been more interested in spaceflight for the last 4 years.
I don't have enough historical knowledge in this area to definitively rule out
what was posted, and it seemed plausible that there was tension between solid
and liquid development. I am currently readin Von Braun's 'Space Frontiers'
and enjoy it. Any recommendations for a next book about the history of US
rocketry?

~~~
bryananderson
Andrew Chaikin’s ‘A Man on the Moon’ is a wonderful history of the Space Race.

~~~
Herodotus38
Thank you

