
Project delays: why good software estimates are impossible - SarasaNews
http://chrismm.com/blog/project-delays-why-software-estimates/
======
cpitman
If I am running a business that is looking at what projects to fund for the
next year, and one of those projects is a software project with an
_unknowable_ time to completion vs (for example) opening a new store, with a
well known time to complete, why would I _ever_ pick the software project? It
has a completely unknown ROI! If you cannot even give me probabilities, the
lottery is a more guaranteed investment!

Software estimation is required for real business to ever consider software
projects. The key is to embrace uncertainty, and instead of estimating that
task X will take exactly Y days, estimate a range of possibilities. I like the
three point system, estimating a best case, worst case, and most likely case.
It explicitly asks you to think about the long tail for when things go wrong.
Then use statistical methods to combine estimates and create estimates for
multiple confidence levels.

This is all covered in a great book, "Software Estimation: Demystifying the
Black Art"
([https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...](https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDMPOVQ&preview=newtab&linkCode=kpe&ref_=cm_sw_r_kb_dp_kE5KxbAPDNJB5)).
It's not that long a read, and it goes over the problems covered by this blog
post but doesn't throw it's hands up in the air and say estimation is
impossible.

~~~
markbnj
>> If I am running a business that is looking at what projects to fund for the
next year, and one of those projects is a software project with an unknowable
time to completion vs (for example) opening a new store, with a well known
time to complete, why would I ever pick the software project?

Maybe because you're trying to innovate? Will opening a new store dramatically
change your company's market outlook? Will it open up entirely new markets for
products you don't yet have? Will it solve new problems for people?

Milton Hershey offered his first chocolate bar in 1900. It took him a further
five years to perfect the manufacturing processes that produced the Hershey's
Kiss. None of those processes, and none of the machinery to execute them
existed when he started. He had to invent them.

Today, 110 or so years later, the company produces around 60 million of them a
day which it ships all over the world. Probably he should have just opened
another store.

No offense, but people who think the way you do about risk are exactly the
reason large companies lose their ability to innovate at all.

~~~
cpitman
I'm not really making the argument your reacting to. Software projects _can be
estimated_ , so it is possible to understand what their ROI is. I'm saying
that, because software _can_ be estimated using rigorous technique instead of
hunches, that we can continue to justify developing software.

Even if what you are doing is truly research that is completely open ended,
there still have to be controls placed on it. Budget/grants/runway will run
out at some point.

~~~
maxaf
I'm not a believer in what you're saying, but I dearly want to believe, in no
small part because there are significant organizational rewards to be reaped
for believing in estimates and acting the part.

What are these rigorous techniques you speak of? I seriously, honestly want to
know.

~~~
cpitman
The book I linked up above is one of the best I've read on software
estimation. It's short, it covers exactly the problems we run into everyday
(managers take estimates to mean commitments, optimism on most estimates,
constant pressure to reduce estimates because they cannot be defended, etc).
And it covers several different estimation methods that you can choose from
based on circumstances, the pros and cons of each, etc.

It's "Software Estimation: Demystifying the Black Art" by Steve McConnell
([https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...](https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDMPOVQ&preview=newtab&linkCode=kpe&ref_=cm_sw_r_kb_dp_kE5KxbAPDNJB5)).
Same author as "Code Complete" and "Rapid Development", which are also great
books but much longer and showing their age a little more.

~~~
maxaf
Thanks! I'm going to give it a read.

------
maxaf
I find naive and pretentious the suggestion that the average programmer's work
is in any way akin to Einstein's search for the truth about our universe. My
criticism might seem nitpicky until it's time to measure this article's
message against the possible reaction of an engineering or business manager.
Launching into a systematic dismantling of the pervasive estimates culture is
best done with humble, verifiable assertions and helpful suggestions in tow.
Attempting to recruit the long-dead Einstein is unhelpful and hand-wavy.

There must be a better way to package the no-estimates message, and it's not
this.

~~~
ktRolster
_I find naive and pretentious the suggestion that the average programmer 's
work is in any way akin to Einstein's search for the truth about our
universe._

Indeed, most projects are remarkably (and sadly) similar to other projects
we've done before.

~~~
markbnj
It doesn't much matter how "similar" they are. Unless you can copy and paste
the code you're building new stuff and running the risk of encountering all
the issues inherent in building new stuff.

------
Klockan
Software estimates will be optimistic as long as optimism is rewarded.

~~~
amai
I would even say: Software estimates will be optimistic as long as wrong
estimates aren't punished.

------
lucindo
I like this Quora answer on the subject: [https://www.quora.com/Why-are-
software-development-task-esti...](https://www.quora.com/Why-are-software-
development-task-estimations-regularly-off-by-a-factor-of-2-3/answer/Michael-
Wolfe)

~~~
AdieuToLogic
That Quora answer is brilliant! Thank you for linking it here.

------
vermooten
“What? Two days?”, “But our previous developer could do that in a couple of
hours! No way it will take you that long.”

Answer: Well get the previous developer back to do it then. Otherwise STFU,
and let me get started.

~~~
bunderbunder
If I had $1 for every time I saw one teammate get roped into spending a week
building some goofy little low value feature because another teammate - never
a guy with any skin in that particular game game - said, "I could have that
done in two hours" during a meeting. . .

~~~
dpark
You have bad managers if they let that crap fly. The appropriate response to
that kind of thing is "then you should do it, because you clearly have the
domain expertise to do the work efficiently".

------
wellpast
> The best you can do is estimate based on historical metrics which ... is not
> worth very much.

I've had significant success in using historical metrics to accurately
estimate software projects.

It takes diligence and professional experience to learn to do this, but it is
possible in my experience.

Unfortunately I find this philistine sentiment quite frequently in the
software world: "I've had difficulty doing X. I am smart. I am senior. I am a
professional. Therefore X must be impossible." Isn't it possible you haven't
been stubborn/hard-working enough in your own personal growth?

~~~
avasylev
Could you share more details on how you used historic estimates? You basically
described me. I've never found historic estimates useful in practice, of
course I haven't tried to hard, but over the years can't think even single
case I'd find useful to use some old estimate.

~~~
wazoox
That's quite simple, really. If you've carefully tracked your past projects,
by comparing tasks from the planned project to those you've done of similar
complexity you can make more realistic estimations.

It works best on a long-running project. When you plan a new feature, you can
gauge really accurately at what speed your current team developed a feature of
comparable complexity, so as you go forward you get better and better
estimates.

Joel Spolsky explained it there:
[http://www.joelonsoftware.com/items/2007/10/26.html](http://www.joelonsoftware.com/items/2007/10/26.html)

------
doozy
Not impossible, its akin to having a P vs NP problem.

Properly estimating a software project takes just as long as developing the
software itself.

When people want estimates they somehow assume they can be given on the spot,
or at most with a few hours of work. Nothing further from the truth.

A good software estimate is a project in its own right. It takes time, a
deadline and a budget to break apart a software project, design a solution,
identify its subtasks, identify its critical path, research alternatives,
evaluate frameworks, tentatively assign tasks to teammates, write down an
estimation report, discuss it with the customer, etc.

And then you end billing the customer for 120 hours of work to conclude the
project can be done in five days, give or take a week.

------
_Codemonkeyism
1\. People need to look into PST from Simon Wardley.

2\. Some of my clients do excellent work with estimations delivering on time
and on budget and satisfied customer (In the Wardley Six Sigma zone - low
uncertainty)

3\. Estimation does not work in the Genesis or Product zones (high
uncertainty)

We need to grow up beyond the "software estimation not possible".

~~~
wpietri
In Wardley's model, almost no software development should take place in the
"six sigma" zone. Consider figure 1 here:

[http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-
sig...](http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sigma.html)

It says "Six Sigma/Outsource" for a reason. The product is "ordered, known,
measured, standard, obvious, low margin". If at all possible, this should be
an off-the-shelf choice. Previously, that would have meant packaged software;
now it mostly means getting something as a library or service.

Six Sigma is fine for manufacturing low-margin items, because you are trying
to get high repeatability in the world of atoms. If you have one widget and
need to make one million more, you use Six Sigma. But with bits, we already
have near-perfect repeatability. If I have one integer and need a million more
integers, the computer will do that at a reliability level well beyond six
sigmas.

If software is being made in the Six Sigma/Outsource zone, it mostly means
that one is reinventing the wheel. Sometimes that's necessary for historical
reasons. E.g., if I'm working in a new language but need a library that
doesn't exist yet, sometimes I really can just take the old spec and implement
it anew. Or if I want developers to use my cloud service, maybe I need to just
reimplement chunks of the AWS API, because Amazon won't give me that code.

But the great majority of software development should take place in the
higher-novelty zones, where reliable project-scale estimation is somewhere
between impossible and nonsensical.

(As an aside, casting people you disagree with as children is insulting.)

~~~
_Codemonkeyism
Yes you're right in principle and I agree.

Though some of my clients create software for others duplicating and modifying
existing products with high certainty.

~~~
wpietri
If they're duplicating, I agree high certainty can be had.

But once modifications happen, I think you're headed back to the other end of
the Wardley spectrum. Do the modifications meet user goals? Do they meet
business goals?

In those circumstances, the only way you can return to high certainty is by
creating very firm specs and resisting change orders. From the perspective of
a contract development shop, I'm sure that's fine. But for the people actually
paying for the software, that's a process smell.

The commercial and practical validity of modifications can only be fully
evaluated by shipping. But large specs require long release cycles. This
creates a long period where learning is deferred. Estimates only become
manageable because the project is saving up trouble for later. A period of
artificially high predictability will be followed by a period of artificially
low predictability.

And even in the pure duplication case, what comes after that? I can't think of
many successful pieces of software that stopped at 1.0. Once the duplicate is
released, most businesses will need to respond to their customers, doing novel
and innovative things specifically for them. So at best, you're back in the
fast-cycle end of the spectrum. But at worst, you have executives who are
attached to specs and estimates and long timelines, which are the wrong tools
for innovation. You have development processes that match. And you probably
have a code base that's not set up for flexibility.

Given that, even if I were, say, making that AWS competitor, I'd still use all
my tools from the fast-cycle innovation end of the spectrum. Because even a
pure duplicate assumes that a) there is nothing left to learn, and that b) the
context of sale/use is not changing. Estimation is always a bet that you won't
learn anything important between the start and the end. Given how smart my
colleagues are, that's a bet I'm rarely willing to make.

------
unoti
This isn't a problem unique to software engineering. All kinds of things in
business require estimating the unknowable. Investing in any business
enterprise requires forecasting what the return on investment is and how long
it'll take before you see those returns. These issues exist whether you're
starting a hot dog stand, building an apartment complex, bringing a software
product to market, or building an off shore oil rig.

Business people have put a great deal of worthwhile thought into how to manage
these risks in ways that make sense for the company. For some fresh, eye
opening ideas on how to do project management for software and other
businesses, check out the book _The Critical Chain_. The short section at the
end on metrics for investment is worth the price of admission alone.

------
spotman
The difference between a consultant with lots of real world experience and
not.

Yes there is slippage, but you learn to plan for it. In the past week I had an
employee have a baby and an employee have a parent pass away and another one
have a horrible knee injury and be unable to work for a couple days, meanwhile
we lost no time on a project by we are up some of our buffer.

If you don't plan to meet your deadlines even when faced with feature and
scope creep ( which managing is another discussion entirely ) or things 100%
out of your control there is other people that can and so your not going to be
picked for projects that require real time mission critical demands.

And yea, if you figure that out, you get to charge more, and sleep less.

------
ryanmarsh
Software isn't construction it's design.

If you start treating estimates for software as estimates for design work your
life will suck less.

The anxiety and stress around making and meeting software estimates is all
rooted in a misperception of what software is.

"Real businesses" budget for design work too, they just have different
expectations for it than, say, a new office building.

------
FollowSteph3
How long would it take to walk from San Francisco to Los Angeles. Even that
almost impossible to estimate and not be off be several multiples, if you're
lucky!

[http://www.michaelrwolfe.com/2013/10/19/50/](http://www.michaelrwolfe.com/2013/10/19/50/)

~~~
ktRolster
The guy in your link did lousy routefinding, and estimated that he could walk
_40 miles a day._ Anyone with experience backpacking can tell you that's a
ridiculous pace (for a group, one mile an hour is a good pace). Plenty of
people plan out long hikes with accurate estimations.

~~~
FollowSteph3
But remember you're doing something novel so you've never hiked before, and so
you can only estimate how much you can do each day. You may have done other
travels but never walked that distance. Even if you half the speed you'll
still be way off. And this is just 400 miles, what about a walk that across
the whole US? Can you predict that Susie on your team will fall ill for a few
week. How many meetings will management need? What about feature changes? How
much time do do you have to estimate? So you have months to plan out your
walk? Most projects need estimates within minutes to hours. And so on...

~~~
ktRolster
_But remember you 're doing something novel so you've never hiked before_

You're not a novice programmer, you've done it before. And if you are a novice
hiker, you shouldn't be doing it alone.

~~~
FollowSteph3
You've built a custom system with the exact same business requirements that
connect to the exact same different sub systems, api, etc? It may be the same
programming language and general framework but that's only a small part of
most projects ;)

~~~
ktRolster
_You 've built a custom system with the exact same business requirements that
connect to the exact same different sub systems, api, etc?_

Similar enough that I can make good estimates. You can too, with experience.

------
ktRolster
Think of estimation as an outer bound (with a confidence level). You are
saying "I am 98% sure the project will be completed by time X." The more
unknowns you have, the farther out you should push the deadline.

Give yourself extra time: no one will complain if you finish early.

------
paulsutter
Evolution. 100,000 years ago, there were humans who could accurately estimate
the difficulty of, say, catching a deer. And since they knew it was too much
trouble, they just stayed home in the cave.

Meanwhile some crazy optimist went out and to catch a deer. It was many times
more difficult than expected, but they returned to the cave with the deer and
eventually reproduced more successfully than the accurate estimator.

Thus our ancestors were the crazy optimists.

~~~
pliny
Or the estimators only hunted deer when they were well positioned to do so
(e.g. few predators of humans and many deer around), and the optimists were
eaten by bears.

------
kyled
Meh, i wouldn't say impossible. I worked with amazing people. We were able to
hit our goals consistently. Key is to have a great team who knows what they
are doing and are honest. Measure by complexity, not time.

Unless you worked on a few products, your going to be bad at estimating larger
pieces. A lot of software is very similar.

------
tomlu
I find this a bit defeatist. Estimating software projects is definitely
possible, especially for less novel projects, and high-effort estimates will
have higher fidelity than low-effort estimates. Few projects are truly novel.

You can't make business decisions without having _some_ idea of estimates, so
I find that arguments like this are used to weasel out of making any sort of
commitment. Go ahead, add a big buffer, try to take uncertainty into account,
pad, pad, pad - at least after all that we'll have _some_ idea.

On the other hand a very real problem I've often faced is when the target date
is an input into the estimation process. I.e. we want to ship by May 2017, go
estimate until the estimation fits this target date. Of course, after
estimation the target date can be an input into requirements scoping, after
which you can re-estimate.

------
alex-yo
Heh, I recall when a funny manager asked my team about an estimation. We
agreed on a year (exactly something like 12-14 months). Then he went to his
managers, sold the project with 6 months deadline, and was quite happy.

The outcome was simple: his estimation was good, the team was lazy.

------
Nokinside
Simple equation that is just as good as any:

p : number of people

c : number of changes into specifications (= learning during implementation)

f : number of frameworks, libraries, tools.

x : number of new people, frameworks, libraries, tools

e : initial time estimate in weeks

correct estimate: E = e + e(0.1p + 0.2c + 0.3f + 1.1^x)

~~~
maxxxxx
My equation for the effort until the project is really finished:

E=e*5

Almost always works.

------
bboreham
The fundamental activity of software development is _learning_.

Although it looks to an outsider like the main thing is typing stuff in (or
pasting it in :-), the thing that takes all the time is learning what you
should type, or learning why the code you already typed in isn't doing what
you expected.

If you've done that exact project before then sure, you can estimate how long
it will take to do it again. But that isn't often the case - usually we're
trying to do something new, or better, or on a different platform. Always
learning.

------
evoltix
Another thing to note here that I have experienced is when management has an
incompetent mentality that, for a given project, that all tasks for this
project are created equal. For example, they expect that if task X took 1 hour
that surely task Y will also take 1 hour despite the fact that task X and task
Y vary greatly in complexity.

~~~
tluyben2
Explaining the difference in complexity helps, but the problem is that often
the programmer doesn't really know in detail besides a feeling that comes
mostly from many years of experience. And the best thing is if they then get
'their 10x code slinger' who does it indeed very fast, sends the code back to
you and you are stuck with something akin to a broken toilet with diarrhea
coming out the sides.

~~~
evoltix
True. But it does no good if management is dismissive. But that's another
problem outside of the topic of estimating.

> And the best thing is if they then get 'their 10x code slinger' who does it
> indeed very fast, sends the code back to you and you are stuck with
> something akin to a broken toilet with diarrhea coming out the sides

A grim picture to think about but I find this to be true a lot of the time.

------
perspectivep
Even if you get better at estimating tasks, it's extremely hard to account for
bugs that aren't discovered until later.

I usually have a pretty accurate estimate for small tasks but never go back
and add the time spent on fixing recessions caused by that task. Or the time
spent on extra smoke testing due to missing automation.

~~~
mentat
That why you use procedures that find bugs early. This has been solve (in
various ways) since the 70s yet people are still writing about how it's "not
possible".

~~~
perspectivep
We do. Unit tests, integration tests, manual smoke tests, code reviews of
every commit, design reviews, self hosting...

Sometimes your test matrix is too big. But I agree: sometimes you have to move
slower in order to go faster overall and have better estimates. Estimates have
to take into account testing and regression testing, which most people don't
so it can make you look slow if you start doing that.

~~~
ktRolster
If you're having trouble, this book can help you:
[https://www.amazon.com/Zero-Bugs-Program-Faster-
Thompson/dp/...](https://www.amazon.com/Zero-Bugs-Program-Faster-
Thompson/dp/0996193308)

~~~
jonex
You should probably mention that it's your own book you are plugging here.

~~~
ktRolster
ok, next time I will. Edit window has ended.

------
slavik81
The problems he talks about have noticeable effects on task completion time
and estimate error. In my experience, estimation error is not normally
distributed. For most tasks, estimation error is small, but perhaps one in ten
estimates are off by an order of magnitude.

It's usually a single bug or feature which blows the iteration. Though, by
consuming so much time, it may cause other tasks to spill too.

If you aim to estimate most tasks accurately, the overall project will
probably be underestimated. Outliers consume a significant portion of the
overall project time.

------
segmondy
I must urge any serious developer who has read that article to immediately
forget about it!

Let's begin with the first phrase, "Software estimate". An estimate is an
approximation. An approximation is an educated guess. So if someone said to
you, "Please make an educated guess on when the software you are working on
will be completed" To say it would be impossible would be the most absurd
thing!

To make an educated guess, You would have to break down the tasks into
manageable chunks that you can clearly reason about. Once you have your chunks
of task, you will have knows, unknowns, some with risks or not, some that are
complex or easy, etc. The problem with estimates is the unknown. If you are
constructing software for an entity, then you must not approach it as an R&D,
but rather, as a possible Research THEN Development. All your unknowns should
be figured out at the research phase, then you estimate the development, else
you should fight to remove those unknown tasks from the project.

Think about it, do you get asked for "Software research estimate" or "Software
development estimate"? It's the latter, but most developers make the mistake
of offering their estimate across both R&D. The really beneficial thing to
developers about estimates if they do take it serious, is that it forces you
to stop, think, design, and think again before coding.

The biggest concerns for developers is that estimates TURN into deadlines.
This is a legitimate concern, but nevertheless this doesn't invalidate your
estimate, which is nothing but an approximation. The other concern is that
feature/scope creep into the project really puts a wide gap between the
estimate and completion date. This again is true, remember your estimate was
for the original requirements. So what are you to do as a developer? You must
teach your manager if they don't know or whomever you are working for the
difference between estimates and deadlines. You must also learn to keep
feature creeps at bay or estimate those new features anytime they are added
and adjust your estimate accordingly.

If you are a non manager or business. Please do think about this from their
point of view. Projects have costs, which we can quickly approximate via cost
of people working on it multiplied by the duration of the project. Part of
running a business is managing your cash flow, knowing if you should spend or
not. Without having an estimate of a duration of the project. How can
businesses make decisions on which projects to fund? If you have a startup and
$100,000 in cash. No income yet, 5 programmers that you pay $5,000 a month.
Would you let them work on a project with no completion date, 1 yr estimate or
2 months estimate?

------
jacques_chester
By definition any perfect knowledge of the future is impossible. Regardless of
why.

That's a trite observation. What _matters_ is than an estimate can be made
better than not estimating at all.

It's funny how all these problems come up in other disciplines, but they don't
throw their hands in the air and quit.

They accept that the Nirvana Fallacy is not an acceptable reason to _not_
estimate and they go ahead and do it anyway.

------
yason
Estimating software schedules is easy. Just gauge a ballpark figure of what it
might take if everything went nice and smooth, you had no distractions nor
surprises, and then multiply by pi.

~~~
dozzie
Estimating is always easy. Estimating something somewhat _accurately_ is
difficult. And no, your formula is not a solution either.

------
alexnewman
I've never had a big problem with it. Even with multi month estmates

------
graycat
As has long been accepted, can get good estimates of time and cost for
software project A if it is quite similar to and done by the same team as
successful projects B, C, D, E, F, ....

IMHO, for software project A, break the work down into relatively small pieces
and attack those. Make each piece small enough so that if that the work on
that piece takes too long or even just fails, then the loss there is not too
big.

Some of the most important pieces are about the _experience_ of the team --
have they successfully done just such a thing before?

So, for such _experience_ , start with the _skills_ and, in particular, the
documentation. So, start with the skills and experience with the operating
system, the text editor, the e-mail program, the file system, the command
lines, and the programming languages. Then move on to other related software,
e.g., TCP/IP, HTML, CSS, JavaScript, SQL, AJAX, Web site security, collection
classes, APIs, etc. So, for each of these, get the documentation, the _skills_
, the _experience_ , etc. E.g., have people write test programs that
illustrate and test the tools, and some of the more important limitations on
the tools, being learned.

Then start in, say, outline form, the _waterfall_ development process by
coming up with a _requirements document_ , that is, what the heck the software
is to do.

On writing this document, e.g., what level of detail to include, get some good
consulting help.

Then move on to the _architecture_ document. This document is an example of
the standard project approach of _divide and conquer_. So, that division
results in _pieces_ , e.g., _components_ of some kind. So, get started on some
of the components. And have write some code to test those. Do these pieces one
at a time so that a failure is not to expensive.

As long as the project is not struggling at some obstacle, is making good
progress, each of the small steps looks okay, then continue on.

Soon begin to get some data on how fast the work is going, at least for each
of the stages.

When are well into writing the code for version 1.0, then soon should have
some okay estimates of the time to complete version 1.0.

Here see part of the truth: Much of the work is getting relevant _skills_ and
_experience_. Next much of the work, especially the risky part, is getting a
clean description of what the heck the system is to do. With those parts done
well, we should be talking mostly a nice road downhill from there to a good
finish line.

Gee, in the middle ages in Rome there was a big project to move a huge stone
obelisk some yards to one side. Big obelisk. Big project. Lots of timbers,
ropes, horses, workers. It was successful.

Impressive until ask how the heck the obelisk got there in the first place?
Sure, ballpark 1000 years before some of Caligula's slaves went to the upper
Nile, cut the obelisk from solid stone, floated it down the Nile, across the
Mediterranean, and to Rome, and erected it.

So, Caligula's slaves did some _project planning_ , for a project they were
likely mostly doing for the first time. And they got it done.

And now we are struggling with some software project for some Web based order
entry and inventory system or some such? Ah, come on, guys!

------
dpweb
When in doubt, add time

