
Why are software development estimates regularly off by a factor of 2-3? - cviedmai
http://michaelrwolfe.com/2013/10/19/50/
======
crusso
So the article tries to describe the hidden complexity of software with a
hiking analogy but then it leaves a huge part missing. Let's extend the
analogy somewhat... this isn't your first hiking trip. In fact, you've been on
dozens and dozens of hiking trips. You're an experienced hiker. Yet, why do
you continue to give meaningless as-the-crow-flies estimates of how long it
will take you to get to your destination? The reason is that if you miss your
estimate, it probably won't matter. Your sherpas will be there at every stop
with your food, water, and tent. You have nowhere to be any time soon, so
estimating your time just isn't important enough to worry about.

Developer estimates are regularly off because they seldom impact the developer
directly. Experienced development managers will pad the hell out of the
developers who give them the worst estimates. Most developers will explain all
the complexity of what threw their estimate off without acknowledging the huge
mistake of not anticipating extra complexity in the first place.

My estimates in my early career were no better than anyone else's, ie way off
-- especially for more complex projects. I'd explain what happened to my
managers and soldier on. The very next task that came up, I'd give my manager
some best-case estimate of how long something would take and the cycle would
begin again.

That all changed once I started to do consulting for myself using "not-to-
exceed" pricing. The first multi-month project I did killed me. My effective
hourly rate went down to sub McDonalds levels and took much longer to deliver
than I had expected. After that project, I did a post-mortem on the project to
figure out where I went wrong. I came up with several spreadsheet templates
and checklists to run through before giving any more estimates.

Mostly I just concerned myself with getting a hell of a lot better at
estimating project duration and difficulty. Like most things, when you really
pay attention to it and practice it, you get better at it.

~~~
kronholm
Been going at it for a couple years now with consulting and freelancing. I've
finally gotten good at hitting my estimates. What worked for me was to keep
strict time of everything (shoutout to toggl.com, love it!), so I could learn
from my mistakes.

The trouble is now though, my competition seems to be underbidding me, but in
reality they're providing those ~33% estimates they will never realistically
keep, while I'm at ~100% estimates. Not really sure how to relay that to
clients. One of the many reasons people like me need a salesperson in front, I
guess.

~~~
RyanZAG
You're probably losing more business than you think - customers have no idea
who is accurate or who is just faster, etc. They will generally go with the
lowest quote regardless of other factors unless they've been burned before
multiple times. Most people haven't.

I don't have a solution here though other than noticing that underbidding and
then getting skilled at convincing clients to do paid extensions later
actually appears to make the most money at the cost of your ethics. I'd avoid
that approach, but it does seem to work for a lot of companies.

My personal approach is to quote for very bare projects with only the bare
essentials (eg, poor UI design, minimum possible feature for the client to see
what they're asking for, etc). This can usually be done a lot cheaper than
most people think as 90% of the work is in the last 20% of the features. Then
once the client has something, you can give them a quote to touch up the parts
they need. Basically you split the project up into many small projects each
with their own quote which helps you to estimate tasks as they appear and
helps your client to minimize costs by leaving off features that are more
expensive than they initially appear.

~~~
crusso
I get contract work through referrals. I'm not interested in being perceived
as a "low cost developer".

I would just as soon not keep customers only interested in the lowest price.
They're the ones that typically will be the biggest headache.

~~~
phamilton
The other side of that coin is that often the clients I get via referrals have
no reasonable alternatives. They often have done all business through
referrals and so the alternative to me is the open market which is
intimidating and has a significant barrier to entry. The end result is that we
build a solid and stable client relationship that generally only gets upended
if/when money runs out.

It's a great position to be in, provided you have the throughput to expand to
other clients and aren't dependent upon a single client for income.

------
ebiester
It's simple. People don't like the honest answer.

As a developer, I have pressure to keep my estimate down. I can give you a
guaranteed date with few problems, _but_ the project is going to end up more
time-consuming than what another developer is going to estimate.

The good news is that I will usually significantly beat my estimates under
this system. However, I will then be accused of padding my estimates and will
be given more aggressive targets, some of which I will miss.

The best way to combat this is to give me a team that has worked together on
multiple projects within the domain, and account for changes and ambiguities
as part of the process.

------
dlitz
The problem is that people try to eliminate uncertainty instead of trying to
manage it effectively. If you're trying to make estimates with more than
order-of-magnitude precision, then you're probably doing something wrong.

If we drove cars like we develop software, we would pre-calculate a list of
instructions, set a timer, and then execute those instructions without regard
to what's actually happening on the road. With practice, you could probably
get better at making those lists of instructions, but why bother? The general
approach is just ridiculous.

Decades of experience have shown that there is enough inherent uncertainty in
software development that trying to plan everything from the start just
doesn't have a high success rate. Even if you know everything that has to be
done _today_ to get the job done, your environment will change, and your
stakeholders will change their minds. What good is software that perfectly
solves the problem you had two years ago?

Once you dig through all of the buzzwords, "Agile software development" is
simply about applying closed-loop control theory to software development: You
structure your project to provide high visibility and to allow frequent
changes in direction, then you iterate and make adjustments until the result
is satisfactory.

Everything else generates unnecessary risks that somebody ends up paying for.

------
chris_mahan
Because management looks at you funny when you say something simple will take
4 months. Then they lie and say that it couldn't possibly take that long. And
the reason they lie is because there's no negative to them tying to you. Their
job is to get you to go faster, and they are allowed to lie as part of their
job, if it makes you go faster. The reality is that it doesn't make people go
faster. It makes people leave faster. It makes people give up. Never give an
estimate to management. Estimating is their job, as they control the inputs to
production. If you give an estimate, they will pressure you to meet it, no
matter how unreasonable it later turns out to be. That also makes people want
to leave.

~~~
w0rd-driven
Management giving estimates? No thanks. I'd rather them take my input and do
their job of understanding trends. When I say something takes a long time and
it takes longer, I'm not bullshitting. When I think something takes a long
time and doesn't, its generally because shortcuts are rarely understood up
front. Will I find the same shortest path next time? If I'm doing the same
exact work maybe.

I do work in a place where we are a software company but very few people, even
some of the devs, understand this culture. I don't say something is complex
just for the hell of it. I want to give good estimates but at the end of the
day that is all it is, a somewhat educated guess _before_ I've gotten in the
weeds. I don't need people that absolutely don't understand this concept to
mandate a timeline that is unrealistic. It does absolutely no one any favors.
_That_ is why I leave. I don't need to promote unrealistic expectations down
the line to customers who think everything is extremely simple. Especially
when all we do is create custom software where almost 0% is turn key or off
the shelf. Its just insulting to continue that nonsense and the quicker I can
facilitate a reality check for all parties, the better and more trust is given
to my judgment (and theirs if they actually take the time to learn)

~~~
wpietri
> Management giving estimates? No thanks.

This is actually achievable in a sane way.

I have done it using the XP planning practices. Basically, you make the suit
break everything down into relatively small lumps and place them in priority
order. Every week, the team completes a few lumps. Before you do them,
engineers grade their relative complexity in arbitrary units. (The smallest
substantial thing you do is 1 point; something twice as big is 2 points, and
so on.) Every week, you count up the points completed. That's your "velocity".

From there, you let managers do all the estimating they want to. If they want
the complexity of a unit of work measured, they ask the engineers. If they
want to know when X will be done, they look at the team's recent velocity,
what's in the queue before X, and do some basic math.

The nice part about this is the mental judo involved. Whenever they want to
know when something will be done, it's their problem to trade features against
time. Working like this, it's not geeks vs suits; the suits channel their
schedule pressure into productive work: grooming the backlog.

~~~
w0rd-driven
I misspoke. What you are describing comes from developer feedback but
indirectly in a sense. This, I think, is a nice standard of measurement. Its
when people solicit absolutely no feedback either in the form of past projects
where hours are measured somewhat or what they just "feel" something should
take. I'm all for metrics based estimation because that's how most developers
would likely estimate. Its when management seems to pull things out of their
ass to get a prospective client I have a problem with. I understand when we
need money but I can also trace some of our worst clients to some of the most
unrealistic estimates we've ever given. They're almost a 1:1 direct
correlation and its like no one sees how much of a drain they can be all
around.

~~~
wpietri
I have been there, and feel your pain. It has taken me a long time to learn to
say, "Oh, you told somebody that? Well, then _you_ have a problem, don't you?"
Of course, there are some companies that are so pathological that this stuff
just won't get better: there's a broken feedback loop between the promises and
the consequences. Ugh.

------
wpietri
I love this post; it conveys the feel of the experience so well.

One of my big aha moments about estimation was a bit in McConnell's _Rapid
Development_. He pointed out that most estimates get made with executives
pressuring for short numbers. When you iterate a few times with that, you end
up with the smallest number that developers can't absolutely prove is
impossible.

If you draw out a bell curve of probable completion dates, this is basically
the same as picking one far to the left, so you have a 95-99% chance of being
late. But somehow, executives are still surprised when their totally biased
approach yields the numbers they liked, not the numbers they needed. All the
estimation effort was pure waste; our time would have been just as well spent
making cotton-candy raincoats.

I now have done a few projects with little to no estimation, and it has gone
much better. I have a hard time now seeing why we ever bothered.

~~~
ScottBurson
I once read an argument that went something like this: if you're actually
going to go to the trouble to build something, its expected return to the
business should be so large that how long it takes (within reason) shouldn't
matter. Conversely, if a project is only economic to undertake if it can be
done within a certain amount of time, it shouldn't be done at all.

I find that I often don't have a good feel for how long something is going to
take until I'm about 1/4 the way through the project. If I absolutely had to
produce accurate, high-confidence estimates, I would just do 1/4 of the work
first, then estimate the rest.

As to _why_ estimating is so hard, here's how I like to think about it. Tasks
form a tree: each project has a number of steps, each of which itself breaks
down into sub-steps, and so on. We tend to estimate by thinking about the
tasks near the root of the tree; but the actual time required is proportional
to the number of _leaves_ \-- tasks we can implement without further
breakdown. Without planning the entire project out in detail it's hard to know
what that number is. But to plan to that level of detail requires making many
design decisions; and researching and making those decisions _is much of the
work of the project_.

The bottom line is, good estimates are expensive to produce, and my experience
aligns with yours: it's rarely worth the trouble.

~~~
wpietri
Agreed. Certainly, estimates of cost shouldn't be more precise than estimates
of value, because ROI is the number that matters. And business is a black
enough art that estimates of value are very rarely more precise than a factor
of 4x.

What I think people try to get from estimates is a sense of control. But I
think there are better ways of giving people not just a sense of control, but
actual control.

------
rdtsc
Good points in the article.

You can think of it this way. A coast line is a fractal line. As you zoom in
its length gets bigger and bigger seemingly. From the top level on the map it
looks fairly straight.

Related to estimates. Programmers have and operate with idealized mental
models. A FIFO queue works kind of like this ("/closes eyes and sees a line of
people lining up at the store"), a tree traversal looks like this ("/closes
eyes sees a red cursor move down a tree drawn with blue nodes on an white
board"). Same with estimates. How long will this take? ("/closes eyes and
imagines an idealized sequence of events that will lead from now till project
completion -- use this library, develop that API, test a little bit here,
done... 2 weeks").

The problem is, zooming in is hard. "Oh this library we are about to use
actually doesn't implement these corner cases". So now we are spending 3 days
patching it. Oh, while working on this project, a critical support ticket
comes from a customer. And then testing reveals a fundamental flaw in our API
design. Re-write a huge chunk and repeat.

Some developers and managers just use heuristics. They do the mental estimate
then x2 or x3.

^ All the above is true if the developer is everyone is honest and trusts each
other. As other posts wrote, that is not always the case.

Then it is a more complicated game. The main question to ask then is "What is
the punishment for overestimating vs the punishment for underestimating?".

Have people been fired for underestimating? No, ok underestimate. Have people
been fired for missing deadlines? Ok then overestimate.

You can even play this game with yourself. What kind of deadlines do you set
for yourself and how to you handle setting a too short or too long of a
deadline? Do you get better with time at setting deadlines? If no, why not?

~~~
mentos
I think the fractal analogy nails it.

Haven't thought about it before but I realize I too use a heuristic of x2 or
x3 when estimating my time. Nothing would stop me from getting a pen and paper
out and stepping through the 'trail' in my head in pseudocode beforehand, but
for most of my projects I'd benefit from just writing the real code from the
start and apply the time I saved to the project.

------
nahname
Imagine you are going on a hike to the top of a mountain. Countless paths
exist between you and your destination. Most of these are dead ends. Every
dead end forces you to backtrack, sometimes all the way to the beginning.

Software is a lot like that.

~~~
tsotha
Then whoever was planning the hike decides to go to a different peak when
you're halfway up the mountain.

~~~
adambenayoun
99% of the time you'll die before reaching the peak due to lack of water or
food.

------
pitchups
This post was originally an answer on Quora to the question posed in the title
[1]. The question has so far received 222 answers - some of them arguably
better, though not as popular as this one.

[1] [http://www.quora.com/Engineering-Management/Why-are-
software...](http://www.quora.com/Engineering-Management/Why-are-software-
development-task-estimations-regularly-off-by-a-factor-of-2-3)

~~~
mikegioia
This blog article is more popular than the quora post because this blog
article doesn't require people to sign up to read it.

~~~
greenyoda
There was a trick posted recently to HN for reading Quora articles without
signing up:

Tell HN: Add ?share=1 to Quora URLs to display content without login

[https://news.ycombinator.com/item?id=6300856](https://news.ycombinator.com/item?id=6300856)

~~~
maxk42
I still don't want to support any site that uses this business model. You want
me to use your site? Make it easy for me to come by for the information I
need.

------
tmoertel
Software-development estimates are regularly off by a large margin for two
reasons. First, the problem is inherently hard. Second, social and business
pressures bias the estimates downward.

Why is the problem inherently hard? To see why, let's imagine that the All-
Knowing Fairy Godmother of Software Estimation descends from the heavens and
lends to us her magic estimating function _F_ that, applied to any software
project _S_ , will tell us exactly how much time and money our preferred
software team will consume to implement _S_. Don't worry about how _F_ works,
just believe that it does. (Okay, okay. Let's just say that _F_ peers into a
parallel universe in which our team has _already_ implemented a software
system line-for-line identical to _S_ , and it just observes how much time and
money were actually consumed in that universe. Anyway...)

Now that we have the magic _F_ , our problem is easy, right? Nope. The real
problem is that we don't know what _S_ is. All we actually know is that our
_S_ , whatever it ends up being, must satisfy some set of fuzzy constraints
_C_ (commonly called "software requirements").

In truth, there are a countless number of possible values for _S_. In other
words, _S_ is a random variable, a mapping that ascribes a probability to each
of the possible software systems that we could build to satisfy _C_.
Therefore, our best estimate, even with the perfect estimating function _F_ ,
is itself a random variable. In other words, it's not an estimate but a
distribution of estimates.

See how fun this is getting?

But, wait, it gets funner. That's because people in the business world don't
want a distribution of possibilities. They want a budget, a date on the
calendar. So we must squeeze point estimates out of the true distribution.
(And, remember, we don't even know what the true distribution is!)

That's where the second reason kicks in. Let's just think about the
distribution of possible budgets for our distribution of possible values of
_S_. Because _S_ can range from "the simplest thing that could possibly
satisfy _C_ " to "the most insanely complex thing that a frighteningly gifted
salesperson for an enterprise consulting firm could get our CIO to throw money
at," that distribution is going to be w-i-d-e. From _X_ to 10 _X_ wide.

So if we're the folks tasked with coming up with those point estimates, we
could plausibly estimate _X_ on the low end, or 10 _X_ on the high end, or
anything in between. Guess which estimates are going to get us the most push-
back from the higher-ups?

Now, we're swell guys and all, and we want to do a good job with our
estimates. No question. But, still... We're human. If we think that giving a
higher-end estimate is going to make us unpopular with the higher-ups, maybe
we'll estimate a little lower. And when we get push-back on _that_ estimate,
maybe we'll "take another look at the numbers" to see if we "missed any
opportunities."

And that's the old one-two. We start with a hard, fuzzy problem and then add
to it the pressure to deliver only feel-good solutions. The result: estimates
that are often way low.

~~~
wfunction
For what it's worth, I haven't seen hardware engineers complaining about this
issue as much as software engineers, which I find baffling. I think if you're
going to justify why software development is such a "hard" problem, you should
mention what makes it so different from other fields.

~~~
tmoertel
Here's the thing. Software is profitable at much lower quality levels than the
hardware market will tolerate. As a result, most hardware is designed through
a process that could accurately be called engineering, even though most
software is not. So hardware engineers get to make estimates with the benefit
of having fairly reliable engineering outputs to use as estimating inputs.
Further, the business side of hardware is more culturally attuned to the
realities of engineering and is therefore less likely to challenge estimates
coming out of mature engineering processes.

In short, on both sides of the old one-two punch, the hardware guys have got
advantages.

~~~
wisty
So the hardware estimate is more "how long will it take to debug the
prototype, and bring it to production"?

~~~
ep103
More like,

1) When someone buys custom built hardware, they know what they want, and the
specifications for it. When someone buys software, they want "something that
does xyz, is just like facebook and paypal, but also has our logo." And then
it turns out that the problem they're trying to solve is completely unrelated
to any of those things. But you can just rewrite it to do those things too,
right?

2) If there's a problem with the hardware, the customer that paid the money
for custom chips will notice, and will have the technical skill to lay the
blame on the vendor accurately. This means hardware manufacturers need to
deliver quality, and since point #1 guarantees they actually have a
specification, they can therefore engineer to it. In software, often the
client takes your application, puts it on machines from the mid 90s that it
won't run on, gives it to minimum wage employees to use (so management may not
really care if it works), and then ask if its too late for the product to also
be an app on their iphone. When they can't do anything with it 3 years later
because it was written by people who have never looked at a computer before,
the customer will just say "software always has bugs / always going out of
date, find a new consulting company to write us a new one", as if its normal
to have to build a completely new system to do the same thing over and over
again every three years.

------
damon_c
[http://coding.abel.nu/2012/06/programmer-time-translation-
ta...](http://coding.abel.nu/2012/06/programmer-time-translation-table/)

I occasionally just paste this link with no explaination at the end of emails
containing time estimates.

------
swanson
Most people would agree that practice makes perfect, right? I think a big
issue with software estimation is that it is hard to get practice. I'd done
full project estimates for 4-6 projects in 3 years of working professionally.
Compared to the amount of practice I have writing code, this is nothing. How
can I ever hope to get good at something if I only do it once every 3-6 months
and it might take 2 years to get feedback on if my final number was close to
the actual cost? And unlike other skills, I can't practice on my own or take a
course (I've never seen an open source project that did round-trip estimates,
maybe that's something to try out...)

Another issue is when you have an "indirect estimate" \- basically I will
estimate the work, but someone else is the one that ends up doing the project.
If you aren't careful to consider who will be doing the work, you might
estimate too low (if you are an expert and the work is done by a bunch of new
hires).

And none of this even touches on misinterpreting client demands, scope creep,
dev team turn-over, or often neglected timesinks like documentation and
meetings.

------
jl6
Kept waiting for the punchline where the friend has moved to Las Vegas and a
total replan is required.

~~~
pc86
"Oh it's just one state over, I didn't think it mattered. See you tomorrow!"

------
reticulated
The single biggest mistake in the example is giving an estimate for the whole
project without knowing team velocity. He tried to plan everything up-front
and committed to a completion date, strongly setting delivery expectations.

An Agile approach to this walk would have managed the expected target date
much better and improved team morale by not overstretching each day
("sprint").

Lots of people think they know (and implement) Agile. I did, until a course I
took about a year ago where I learnt skipping seemingly unimportant bits
(measuring velocity, sprint retrospectives, etc.) bring the whole process
crashing down.

~~~
cs02rm0
_The single biggest mistake in the example is giving an estimate for the whole
project without knowing team velocity._

I'm inclined to agree. Often though a client will want an estimate (to a
ridiculous accuracy) before agreeing to the work. Unless you churn out web
page templates or something equally repetitive it's hard to have that relevant
velocity.

I still come up against client managers that don't "believe" in Agile
approaches too.

~~~
reticulated
Oh we still get regular "lively discussions" about exactly what features are
going to be in a product with a release date 6 months away. I can completely
understand though; it's very difficult to leave a conversation being told "we
don't know yet, it's likely to be X & Y (but we're not 100% sure) and possibly
Z too".

------
pjmorris
I just had an estimate/actual conversation that went like this: I think it
will take me X time. Ok, but I want it done in .5 X time. Ok, I'll try to get
it done in .5 X time. I finish it in X time.

Is it a missed estimate? That depends on who answers.

~~~
alextingle
> Ok, I'll try to get it done in .5 X time.

Never, ever say this.

Personally, I always give my estimates as a range. If you break down your
project into small tasks, and estimate each task as an optimistic/pessimistic
range, then you can get a pretty good idea of the expected completion date,
along with the deviations from that which you might reasonably expect.

However good your estimate, committing to ONE number is madness - it hides the
complexity of the situation, making it impossible for you or your manager to
see the real picture.

------
pasbesoin
Years ago, I was introduced to the (simplistic) dual concepts of profit
centers and cost centers.

To many in Management and accounting (and with varying degrees of pointy-
haired-ness), profit centers "make money" while cost centers
"spend/burn/"waste" money.

The sun shines on the former, while the inquisition perpetually stalks the
latter.

Systems and systems software are often seen as a cost center. There is
constant pressure to "control" these costs and to do more with less.

People are required to be over-optimistic in their communication while at the
same time overly pessimistic ("frugal") in their resource allocation. Note the
use of the word "pessimism". It is not so much a matter of "optimisim"
thinking that "we can get it done with less"; rather, it is the pessimism of
"we can't afford that".

Much of my professional life has been spent in big, "conservative" (cough),
risk-averse organizations. I'm sure things are to some degree different
elsewhere -- some _degree_.

Nonetheless, it's worth keeping in mind this (simplistic, and to varying
degrees counter-productive) concept of "profit centers" and "cost centers".
And if you are working for the latter, expect to be squeezed into ever more
unrealistic positions.

It's something the seasoned professional stays attuned to. There will always
be some squeezing; when it starts to worsen and become excessive, it's time to
go. Your personal solution, as a "simple" employee, is to find another
organization -- the sooner, the better.

------
avenger123
I have found that providing estimates within a range is the most reasonable
approach.

The range is usually between "this is the likely amount of time with
everything going as it should" and "this is the worst case scenario time with
all that can go wrong".

It's a pretty broad definition between the two but I find it gives clients a
sense of the scope of the project. The minimum time does include some padding
to include all the usual likely overruns (testing, requirements gathering,
etc.) but not too much.

The worst case time better really be worst case as I provide clients the
expectation that it should not take longer than this.

I find that usually I will end up somewhere in the middle of the estimate and
the client is still happy since they didn't hit the worst case scenario.

I find that this usually tends to be relatively effective even if as a rule of
thumb the worst case time is 3-4x the minimum time.

Of course, there is still a lot of analysis done to come up with this range
and there is a lot of discussion with the client to help them understand the
details on why and how it could go.

At the end of the day, communication is king. I find that educating clients
about the complexities of the process always helps to gain trust.

------
logicallee
Forgive my username, there is nothing logical about this.

It's because estimates are really just a test of your imagination and skill.
They're a "different game" from delivering something. Here are some examples:

How long does it take you to add 1 + 1 together? The only correct amount of
time is "0 - no amount of time." But clearly that's not true, is it?

How long would it take you to multiply 5 by 7? Well, zero. No amount of time -
you know it 'by heart'. It doesn't take you any amount of time to multiply 5
by seven. Well, clearly, that's not true is it?

How long would it take you to test whether 31337 is prime or not, by any
means? Like, 1 second because Wolfram Alpha can tell me I bet. But clearly it
wouldn't take me ONE second, would it?

How long would it take you to write a function to test whether an integer less
than 2^32 is prime or not, without having to be super-efficient about it? Well
like 10 seconds. But, clearly that's not true, is it?

And so on. How long does it take to put a twitter bootstrap page up? Like, 5
minutes.

How long does it take to...

You should be able to build an app that doesn't crash and does what you're
trying to code in like a day.

You should be able to build something that is software you can launch with in
like a week, as an MVP.

You should be able to replace someone else's mature product with a
reimplementation in like a month.

You should be able to build a scalable business that is ready for a series A
in like a year.

These things are not meant to be accurate, so much as show that you have
imagination.

------
jconley
This McConnell book[1] is a must read on the subject. Generally developers
just fail at estimating at a fine grained enough level and forget about all
the ancillary tasks that take up a majority of a project's time.

[1]
[http://www.amazon.com/gp/aw/d/0735605351](http://www.amazon.com/gp/aw/d/0735605351)

------
ChuckMcM
I enjoyed reading this the first time around. I purposely illustrates the
complexity of software and perhaps inadvertently illustrates the challenge of
getting schedules from inexperienced teams. In rebuttal, had they walked from
San Francisco to LA along any freeway they would have been done much closer to
their predicted date. Which they would have known had they been experienced in
walking back and forth to LA from the Bay Area.

The "big" message is that if you have _never done_ something before, you have
no idea how long it will take, even if you can describe what you are going to
do pretty easily. Returning to the hike metaphor, this is why you need to
listen to the cranky old guy saying "walk along the roads, not the beach"
instead of the inexperienced folks saying "Hey its just walking, and walking
on the _beach_ rocks! Walking along a road is boring and lame!"

------
ssiddharth
I admit it's funny but if I showed this to a non-programmer friend, he'd be
asking why I didn't research my route better or start on an unknown trail with
minimal planning.

~~~
avmich
Because the research will take a significant part of the project time. Once
the thorough research is done - third-party libraries are verified (that they
do what's needed, and not too buggy), prototypes are written (to confirm
libraries and internal dependencies work as expected), proof-of-concepts are
run (to actually see that it's possible for a reasonable effort), time is
measured (to confirm the way to go), interfaces are clarified (as part of the
research, specifically, in the architecture department), results are estimated
(basing on all results above), specifications are formalized (and agreed upon)
- there is an awful lot less of what to do.

~~~
Philadelphia
Maybe part of the problem is that development organizations don't tend to
charge for pre-estimate planning.

------
codex
To put it simply: software estimates are hard because nobody has made that
particular bit software before. Why? Because software doesn't rust. There is
little need to exactly duplicate software which has already been written.
Thus, most software is charting unknown territory.

------
praptak
Expect to be told, halfway there, that the party has moved to Tijuana. After
all you agreed to hike to the party and it would be unreasonable to expect of
the host not to move the party when the weather changes.

------
smokey_the_bear
We sometimes make white label geo apps for other companies. We've repeatedly
had problems where our estimates were off by quite bit because we got data in
incredibly bad formats from the customers. And not just once, every time
they'd send us updated data it would be in a different format.

The first time or two it made us hate the customer, and feel like it wasn't
our fault. But then we started writing it into the bid, and telling the
customer they could save money if they delivered conforming data. Hopefully
we'll be happier from here on out.

------
stevewilhelm
To extend the analogy further: add the fact that you need to meet up with
several other groups that are providing essential supplies and they are hiking
from several different locations in the Central valley.

Some of the groups are showing up on time and are waiting for you, some are
late, all won't have all the supplies you need.

And to make it more realistic: half way through the hike, your friends call to
say the lunch has been moved to San Diego and is now a formal dinner party at
which you will need to wear tuxedos. And can you bring the Champagne as well?

------
systematical
After getting a number of my estimates wrong recently I was thinking about
this question. While walking to work I determined a few things.

It's hard to estimate something that is complex. If I am very confident the
job can be completed in under an hour my assessment is very close to 100%.
This is because something simple does not have a lot of variables and
interacts with very few things. Things like fixing a display bug, a simple
calculation, adding a print now button are all easy. Once you go beyond this
it gets more difficult to determine how long something will take.

Business wants the estimate done quickly. Many businesses want estimates done
fast. An accurate estimate for a complex job cannot be given fast. There has
to be several rounds of back and fourth. Use-case scenarios need to be
determined, which will open up even more questions. The underlying
architecture to accomplish the feature also has to be scoped which will take
time and usually we don't get it right the first time we think about it

Some more reasons are, businesses want the job done quicker than is realistic.
Businesses don't know everything they want. They think they need stuff that
they never end up using. Haven't thoroughly thought out the feature or problem
themselves and that uncertainty gets passed into the development phase more
often than not. Business is not solely to blame though. Sometimes developers
think a particular portion of their code will be simple when its not or
difficult when it isn't.

Basically the whole process is riddled with human error on both sides.

------
Zenst
People are generaly optimists and with that anything in the future will be
biased towards ideals, even planning for exceptions will bias towards a ideal.

The other factor is feature creep, if you find a project so well defined and
set in stone then it would be fair to say it will have a better chance of
finishing on time.

That is just the software side, then there is the hardware to run it and that
opens up a whole new aspect of poetntual delays that will get leveridged at
the project and when people say project behind involving software people will
just focus on the software aspect and all other factors are buried.

Bugs and issues come from many factors and even suppliers of hardware can have
bugs which can have a impact, design could have bugs and many aspects can have
issues that impact the end result.

But it is impossible and silly to plan for every situation as whatever you
plan for there will be an exception and you get a deminishing return, hence
easier to have a area of time labeled contingency as a catch all. Best example
of such exceptions would be say an earthquake killing your entire development
team, that can and has happened and at the same time is not something you plan
for. Partialy why we have insurance. Though you will find it very hard to
impossible to get insurace again software development delays beyond life/death
of parties involved, though that will not chance the delay into a non-delay,
mearly compensate.

With all this Engineers have for many lifetimes gone with the engineering
factor or whatever you think it will take, muliply that by 2-3 to counter
optimisim. But that does not help feture creep and anything added to the
design after code has started is creep.

------
pjungwir
I loved the article, but I was hoping we'd get a reference to the Coastline
Paradox[1]. Basically, since a coast is a fractal, its distance is infinite.
The "zoom" that caused the very first delay could keep going, and going, and
going.

Anyway, nice article!

[1]
[http://en.wikipedia.org/wiki/Coastline_paradox](http://en.wikipedia.org/wiki/Coastline_paradox)

------
fayyazkl
That's a very interesting sort of analogy. Among most of the discussed issues,

a) For one, giving enough time to come up with a project estimate is often
rare. Usually people try to ask for those upfront.

b) Usually we try to add a general buffer for any unseen issues. However, the
deeper you go and list tasks against time, the closer to realistic is the
total.

c) Often it is just a customer giving a requirement needed at date x, or a
management guy setting goal for a date y. When all you can do is to cut
features but have to deliver by a given date.

d) No matter what we say, but at the back of our minds, we are not enough
prepared to defend a timeline when giving it in the first place. Implementing
feature x needs a week? gosh boss would say it should take a day. Latter it
turns out the assumption was based upon a library which for just internal
implementation reason (that you only discover upon learning it enough through
actual code) is completely useless and it should actually be planned for 3
weeks at least.

------
sechastain
The short answer is optimism and poor assumptions. The hiking analogy works
well to illustrate this answer.

Everyone who has been a part of a software project knows about poor
assumptions. From the start, you don't have well-defined requirements. You
don't have a complete design. You don't understand the consequences of
decisions you have made or will make. Any estimate is doomed to fail in the
face of all these poor assumptions.

Staggering a software project into smaller chunks goes a long way to
controlling these poor assumptions and the impact to schedule and, more
importantly, customer expectations.

The principles of the agile manifesto go a long way for mitigating the too
common pie-in-the-sky wishful thinking that dominates software engineering.
They work even better when the customer realizes they are a part of the system
under development and its success - if they can keep that balanced with giving
the development team the space to do what it is they do.

------
eftpotrm
Well, yes, but also (stretching the analogy) how many times have we been asked
for a schedule to an unnamed destination via unspecified waypoints and then
told it's too long and we need to make it shorter?

Devs make mistakes, sure, but as often as not in my experience it's absurdly
vague end user requirements married to unrealistic expectations.

------
sytelus
Dumbing down analogies aside, you can look at software development as a graph
of connected tasks with each task having some variance around when it would
complete. This variance accumulates as taskw are completed and you move on to
next nodes in the graph. When you complete the graph, you have large
probability distribution about average ETA.

Most decision makers have to choose some number for ETA however because world
does not understand probability distributions or variances. Most likely this
has to be lower end of the number to get project sponsored or fend off the
competition.

This is not unique to software development. Any projects that requires large
connected graph of small task nodes would show same characteristics. For
example, fighter jets, Boeing Dreamliner, space station etc etc.

------
judk
Because the estimates are always BS, but teams get 2-3x slack before they are
force to declare victory and launch whatever they have. The more over budget
they get, the more they cut features (including QA and security details)

------
forgottenpaswrd
I really love this article.

In practice a good strategy for companies is reduce unknown to a minimum, the
Steve Jobs strategy, just focusing on very few products, with each having the
less complexity possible(simplify and simplify).

Only focusing on very few products you could make perfection possible,
estimates will be off, they always are when there is unknown, but at least
completion will be finite (you will complete it!, which is number one issue
with research work for example).

Sounds simple, but is really hard, leaving creative people on their own is
dangerous for this reason, they could start new projects, not finishing
anything.

~~~
greenyoda
Reducing the complexity of products is an enticing ideal, but in reality, some
products are inherently complex. Let's take one of Steve Jobs' products, iOS,
as an example. No matter how much you try to simplify it, it's going to be a
huge piece of software (I'm guessing hundreds of developers and supporting
staff). Apple's success has depended on keeping customers engaged by regularly
coming out with improved versions of products, so they need to estimate fairly
accurately what the release dates are going to be: being six months late could
be disastrous for marketing efforts and the company's stock price (e.g., you
want to get the next version of the iPhone in the stores by Christmas). So you
can be sure that there are people at Apple whose job it is to make very
careful estimates of how long it will take to develop stuff.

------
dj-wonk
Summary: The fractal dimension of a software project is roughly 2 or 3. :)
[https://en.wikipedia.org/wiki/Fractal_dimension](https://en.wikipedia.org/wiki/Fractal_dimension)

Now, if you think about that metaphor for a while you may find value in
seeing: (a) the metaphor doesn't really fit intuitively because software
projects are often charting new ground, not zooming in on a predefined complex
problem (b) the metaphor is not too bad of an intuitive explanation -- you
learn more about what is really needed as you get closer to the details.

------
11thEarlOfMar
If you can suspend the obvious question of why are you hiking from San
Francisco to LA in the first place, the general analogy holds quite well: With
software development, you can not know how long it will take until you know
exactly what you are building. You can not know exactly what you are building
until you have fully described it. The act of describing it fully, precisely
and with sufficient quality, is the act of coding.

Hence the completion estimate has to be continuously refined as the details of
implementation are continuously refined.

Just like the slow zoom-in on the hike.

------
cuillevel3
I think the calculation usually goes like that:

    
    
      * Take a day to come up with "estimates" for the project,
        can't bill that time of course.
      * Sell fantasy gant chart to the customer.
      * Take 10-20% of the projects time away for "project management".
      * Actually develop the software.
      * Fix "bugs" until customer goes silent.
    

If you want good estimates you need to take estimates and project management
more serious and get away from that waterfall methodology.

------
aufreak3
Kahnemann and Tversky call it the "planning fallacy".[1] the interesting bit
is that the predictions are pessimistic when you get "non-involved" parties to
estimate time to completion of the same task. Perhaps using a figure between
self estimates and third party estimates may work?

[1]
[http://en.wikipedia.org/wiki/Planning_fallacy](http://en.wikipedia.org/wiki/Planning_fallacy)

------
nathan_long
In this analogy, you have a single destination; anything short of that is
useless.

What about the agile idea that you deliver something incrementally useful with
every sprint?

Obviously, some projects can't be done that way. But if they can, it would
mean 1) estimation can be done on a 2-week scale instead of maybe a 6-month
one, and 2) they can start deriving benefit/making profit from what you've
built while you build the next thing.

------
gesman
Allowing for end-customer's bad habits, such as delayed payments, real-time
changing of specs and other unknowns that are fact of life for developer.

------
DigitalJack
[http://en.m.wikipedia.org/wiki/Planning_fallacy](http://en.m.wikipedia.org/wiki/Planning_fallacy)

------
shitgoose
If only they used one of those frameworks, that lift you off the ground and
glide you towards the destination effortlessly, graciously pulled by flying
pigs, there would be no delay. They would even have time for daily scrums.
Always pick the right technology stack, guys!

------
cavilling_elite
The joke between us engineers at work is to estimate the project time, then
multiply by pi.

~~~
antonwinter
You sir are the inspiration for my new workflow. pi based software dev

------
csbowe
For me, a lot of underestimation is about motivation. I work best with looming
deadlines. I _feel_ like I will be motivated to work harder with a deadline
that's less realistic, even if that' not always true.

------
programminggeek
One reason is as an industry we are terrible at planning. We do a very
unthorough job of planning details or requirements. Then we allow for changes
and new features mid project with no respect for budget or time.

------
judk
That's a great story for why the first project is slow. But next time, you
say, OK, we can hike 10 miles a day, so it's 60 days, and it _still_ comes out
2-3x of budget. Why?

------
anupshinde
Okay, I agreed (had no other choice) with management to not have less than 20%
variance in my estimates.(read: negotiated-and-reduced estimates) as a part of
our performance metrics.

~~~
dreamfactory
Reduce your batch sizes to reduce uncertainty.

------
relet
For the same reason other estimates are off by a factor of 2-5, unless
contractual penalties are agreed on. The cheapest contractor gets the job, and
can adjust the estimates later.

------
clord
I try to do my estimates (team size, man-days) in Big-O notation. Haven't
nailed how to do that yet, but it helps people understand that estimates are
non-linear functions.

------
antonwinter
when asked to estimate, i ask if the biz if they can also give me an estimate
for meetings, UAT, changes, hiring resources, unscheduled downtime and
scheduled downtime and historic data comparing previous project estimates to
actuality. Its the only way to get closer to accurate.

The developers estimates generally are best case scenarios without any
external or motivational factors included.

------
coldtea
Here's another question: "Why are software feature requirement estimates
regularly off and changing while in progress?"

------
analog31
Imagine estimating the time to complete a hike, and then being told that a
bunch of children will be coming along.

------
brosco45
Because software estimate is impossible, if it's possible, then you are re-
inventing the wheel.

------
lignuist
Another reason is, that there are the first 90 percent and then there are the
other 90 percent.

------
puppetmaster3
As opposed to what? Marketing estimates? Scrum and such can be used by
Marketing and others.

------
mpg33
I just don't know why people won't arbitrarily add 2-3 more time on the
estimate...

~~~
jwpeddle
Hofstadter's Law: It always takes longer than you expect, even when you take
into account Hofstadter's Law.

~~~
dj-wonk
According to DJ Wonk's Law (shamelessly self-named), when discussing software
estimation, it is inevitable (only a matter of time) that someone will mention
Hofstadter's Law.

------
sz4kerto
Because software developers, like most people, don't like conflicts.

~~~
mkramlich
also because we don't have Time Machines

------
Shtirlic
All estimates should be multiplied by Pi.

------
i_am_dead
Only 2-3? Please tell me your secret!

------
habosa
It's the halting problem.

------
grownseed
Here's my experience with this problem. First, a number of people go meet the
clients, often several times, to see what the client needs. Generally however,
none of the people involved with actually building the product are ever
involved in the meeting (because, from the mouth of the Boss, "operations are
too expensive"), and in the unlikely event that they are, they're essentially
shut down because "nobody really cares about the technical details at this
point"...

This is then followed by extremely loosely defined requirements being passed
down to the technical people in hope of getting an estimate back. Technical
people, especially experienced ones, know how this goes. First, you know you
don't have half the details you should have, even though you've most likely
asked for them numerous times. But you know, "all I need is an estimate" says
the sales guy (who in this case also happens to share the board, made solely
of other sales related people). You already know you can't win here, so you
give a ballpark estimate considerably above what you imagine should be the
real value, hoping it'll cover your ass, but highlighting the fact that until
more details are given, this is still a ballpark figure.

It doesn't matter. The sales guy takes your estimate, shaves a third off of
it, throws in a couple of bonus features in there (which are most likely of no
use to the client), and makes it a hard estimate. When you point that out,
you're told not to worry and that it's not your problem (spoiler alert: it
very much is). So you write your specs down, or rather, you slap a few bullet
point lists together with whatever little time you're given (you're
expensive). This is given to the client as what both sides should abide by,
despite you repeatedly saying that it's a bad idea.

From that point, you realize that the time you've been given to achieve the
work is, roll drums, not nearly enough. You can already see poorly defined
features growing arms and legs: clients, and I can't really blame them, will
use any vague requirements to their advantage. You also realize that in order
to do all the work and still meet the deadline (you still most likely won't),
you have to half-ass a bunch of the work. You can already imagine the support
issues pouring in once the project is in production, and you see all those
legacy projects constantly getting in the way of your new, poorly estimated
work, thus slowing it down even further. You will also get blamed for all
these issues that are cropping up from past projects, and this ties to the
next point.

The best one of all, that thing that wasn't your problem, suddenly is. You've
somehow become responsible for the poor estimates you clearly said weren't to
be relied on. So you know, you better make an effort and fix that by, say,
working extra hours here and there, like, every single day. You burn out and
grow frustrated, which slows the work down even further. Even better is that
your crazy work hours become an expectation, if you happen to finish a project
on time this way, it'll be used as "See, no problem here, you got it done!",
pat on the back and condescending compliments ensue.

So you realize you're never given time to do decent work, that at the end of
the day you're better off not meeting deadlines, and that whatever happens,
you're always on the losing side.

I'm not saying this is the case in all companies, but I've also seen this far
too often. On the very few occasions I had the chance to be the main point of
contact with clients, projects were on time, clients were satisfied, and of
course, some sales guy still slapped his name somewhere on there and got his
bonus, but at least I got to do good work on reasonable deadlines.

I loathe most sales and marketing people, I often find they're a completely
unnecessary layer in product and value creation. It's not that their position
couldn't be of value, but I don't believe it can be if the money incentive is
always more prevalent than the idea of good work done or happy people. As
Henry Ford said, _" A business that makes nothing but money is a poor
business"_.

Note: I've resigned from this job to work in research.

------
zupa-hu
thanks, was a great read!

------
paulhauggis
My estimates are always off because management decides to constantly change
requirements or didn't have all of the requirements in the first place.

------
anotherfadjs
Because no one wants to admit they still have to learn more about the system
they are going to build.

~~~
noarchy
There's something to this, no doubt. I recently took part in a project where a
firm was contracted due to their "expertise", which they fulfilled by hiring a
team of contractors, most of whom were woefully unprepared, in terms of
knowing the system on which they worked.

------
mkramlich
why? focus very closely on the following word:

estimate

------
lahwf
Egos.

