
Software estimation considered harmful? (2011) - luu
http://gigamonkeys.wordpress.com/2007/04/26/estimation-considered-harmful/
======
prof_hobart
Unless I missed it in there, there is another reason for estimating work -
deciding whether it's likely to be worth the effort to build it.

It's rare that a development organisation has run out of things to build.
There's almost always going to be several ideas competing to be the next thing
to start, and there's got to be some way of deciding which you should go for.
One obvious way is weighing up the likely cost v benefit of each idea, both of
which require you to make some form of estimate.

Of course those estimates will almost certainly be wrong, but at least
starting with a view of whether you think it's going to take a week or 9
months to add a mobile optimised version of your website, or whatever feature
you're considering next, is going to be more helpful in making that call than
starting with absolutely no idea at all of the effort it's likely to take.

~~~
slgeorge
Totally agree. Whether you're creating a new product feature, or doing a
specific piece of work for a customer you need an estimate to define the cost,
resource and time-line. Without it you can't understand whether the work is
worth doing.

For managing cost and resource the estimating process acts as a way of
discovering the complexity in the problem. Yes, it will probably be wrong to
some degree, but there are ways of handling that (ie probability) on the
effort analysis side. The other thing that helps is to identify the edge of
what you can see, if something truly is totally unknown then breaking the work
into blocks or discovery stages makes sense.

At a practical level time-line is the one that's the least useful to estimate
up front, and seems the least amenable to assessment.

Without the ability to make some sort of estimate you cannot assess whether
the work is likely to be worth the investment.

~~~
humanrebar
> Without the ability to make some sort of estimate you cannot assess whether
> the work is likely to be worth the investment.

I completely agree. But I'll add that without some notion of risk, the
estimate is itself risky. In other words, when building a business plan around
software estimates, getting a one number as an estimate (30 days) is a bit
crazy. It would be much more useful to attach a range or confidence to the
estimate

What happens now is that Project Manager asks Team Lead for an estimate. Team
Lead pads the estimate a little and says 40 days. If Project Manager expects a
20% ROI, there's a significant risk that the project will still end up in the
red. Or the team ends up crunching like crazy to "catch up", which is
basically the same thing since your spending other things (quality, employee
health and goodwill, etc.) instead of budget. Whatever the exposure, there is
significant risk that is not clear in the business plan, which is a failure of
management.

In contrast, maybe Team Lead could give a 50% chance that Project Jabberwocky
will be done in 30 days and a 95% chance we'll be done in 120 days.

Wait, Project Manager says, that's a wide range of values to plan around! He's
right, but the emotional reaction is due to him being educated about risk, not
due to the Team Lead dragging his feet or something.

So what is a Project Manager to do with that estimate? Take the risk into
account. Plan around 95% estimates (keeping in mind that 1 in 20 projects will
still miss the mark) or make sure the risk is mitigated in other ways: lots of
little projects, relatively low cost projects, projects with huge expected
ROI, etc.

~~~
cpitman
The book mentioned in this article, 'Software Estimation: Demystifying the
Black Art', actually goes into a lot of depth about this. For example, what is
statistically likely to happen if you only use single point estimates when
they are aggregated together, and how to correctly handle range estimates.

------
taeric
Surprisingly good read. I will be looking more into the study showing that
folks with no estimation had a higher productivity. I can say from experience
that the Herculean task of estimating my own work adds to my stress
significantly. To the point that I have no doubt that it hinders me.

Oddly, just having a deadline is nowhere near as destructive. So, coordinating
with folks is a lot easier if they tell me when they'll need something, rather
than asking when I can have it for them.

~~~
reedlaw
The worst case scenario is when there are only vague requirements or too many
half-baked ideas. And somehow you're supposed to say when those poorly-defined
features can be done. As if the completion date is not closely tied to the
full definition of the finished feature. This is why I almost prefer the
waterfall approach when hard estimates are required.

------
koonsolo
The need to estimate highly depends on whether you are developing a project or
a product. On a project, the customer always wants to know _what_ will be done
at which _date_ , and how much that would _cost_.

A product however, can be released "when it's done", so there is no real need
to estimate. The additional benefit is also that it can be released when the
developers feel that the product is ready for release, unlike a project, where
there is a lot of pressure to deliver things.

My personal view (and experience) is that developing without estimates saves
time and creates a higher quality product. But other (smart) developers that I
know like to work within estimates because they think it pushes everyone to
deliver things.

We probably are both right, because there is yet another factor that comes
into play when choosing to estimate or not: motivation of the developers.
Highly motivated developers who love their job, rather work in their code
instead of goofing off on the internet. But when the project/product is not
that special, and not everyone feels passionate about it, estimating and
follow-up is probably preferred to put a little pressure on getting things
done.

~~~
UK-AL
Estimates should never be used to apply pressure(people take shortcuts), they
should always try to reflect reality. In products estimates are used to work
out which features provide the most value, in the shortest amount of time.

~~~
koonsolo
> In products estimates are used to work out which features provide the most
> value, in the shortest amount of time.

With products, estimating the value of a feature is even harder than
estimating the implementation time/cost. And calculating the future revenue of
your product is also harder than calculating the future implementation cost.

You might argue that your product must be finished before you run out of money
(which you can calculate with estimates), but MVP's don't rely on estimates
that much and you can test your product/market fit at the same time.

In theory, product profitability and selecting features seems highly depend on
estimates, but in practice I always saw that gut feeling was good enough, for
both features and how long a development will take. You can't drop the must-
have features, and nice to haves can be implemented after your first release.

------
bagels
"a Wesabe developer who consistently gets excited about developing things that
Marc refuses to allow into the product should probably make sure their resume
is up to date"

If you cannot read the mind of our CEO who changes the direction of the
company every two weeks, you are fired?

~~~
jorgecurio

        If you cannot read the mind of our CEO who changes the 
        direction of the company every two weeks, you are fired?
    

I've experienced this.

Pretty much. Whats even worse is if it's some guy between you and the CEO, who
can then pretty much travel back in time and paint you in whatever picture he
sees fit to put his interests above yours as a lower level developer.

------
benjiweber
I think it's more that estimates don't work well with projects. If you know
you're going to be building a, b, and c as part of the project then there's
not a huge benefit in estimating them.

I also think projects don't often work well as a way of building software.
There's often uncertain deliverable a long way off into the future, and a
guess at whether it will actually generate any value.

I don't just think that estimating the cost of features along a project is a
bad idea, I think trying to come up with features a long way into the future
is a bad idea. Things you learn along the way are likely to require you to
scrap them or make significant changes, unless you're ether prescient or
learning nothing.

Without projects estimates become more useful again. Focus on the most
valuable things you can be building in the next couple of weeks. Estimating
the top n that seem the most valuable will tell you a) if it's achievable to
complete them in 2 weeks, b) give you an idea of the value:cost ratio to help
prioritisation.

~~~
wlievens
So... agile?

~~~
benjiweber
Well yes. I suppose the problem is people think they are being agile when they
actually have a 6 month backlog of user stories and won't release anything for
months.

In that scenario estimating things becomes very tedious and not particularly
useful.

~~~
wlievens
But what if you actually have a 6 month backlog of real user stories? As in,
you know these requirements pretty well, for instance because you're re-
implementing a legacy system?

~~~
dreamfactory2
What's the difference then between the new system and the legacy one?

~~~
wlievens
In this hypothetical scenario? Any number of nonfunctional requirements.

~~~
dreamfactory2
So no added business value can be conveyed in stories?

~~~
wlievens
Sure it can. User story: "The software is orders of magnitude too slow.
Rewrite it from the ground up using scalable technologies."

~~~
dreamfactory2
Point being that it isn't a rewrite but a new piece of software.

~~~
wlievens
If you're implementing, say, a general-purpose accounting system, from
scratch, then I think your requirements and feature set is well known beyond
the "a few weeks" horizon, and planning that feature set months in the future
is not unreasonable.

~~~
dreamfactory2
Actually wouldn't agree with that example. Accounting systems are very diverse
e.g. Sage, Wave, Freeagent, SAP, Excel+Access.

------
Expez
I think part of the explanation for why the software products with estimates
performed poorer in terms of productivity can be explained by parkinson's
law[1].

Now, this phenomenon isn't always deleterious imo. I've noticed in myself that
my work 'expands' in two ways: if I'm working on something dreadfully boring,
then it expands through time wasted on hacker news or time spent on side
projects. If on the other hand I'm doing meaningful work, then it's quite
often that the work expands when I take on refactorings and minor additions
that will benefit long-term productivity.

If the work is indeed expanding in a meaningful way, then even though the
measured productivity might not seem 'optimal' it doesn't necessarily indicate
a big problem either.

[1]
[http://en.wikipedia.org/wiki/Parkinson's_law](http://en.wikipedia.org/wiki/Parkinson's_law)

------
islon
My big point is:

\- If you don't know what you are doing or you are doing for the first time,
there's no point in estimate as your estimations will be off anyway.

\- If you know what you are doing and already did it before, there's no point
in estimate as you already know how much time it'll take.

~~~
timv
_If you know what you are doing and already did it before, [...] you already
know how much time it 'll take._

So you've estimated it then?

If the argument is that we spend too much time producing detailed estimates
that don't actually reflect reality, then I agree.

If you're suggesting that a sensible way to run a business is to say "I have
absolutely no idea how long this will take, and how much it will cost, but
let's do it anyway" then that's crazy.

Even the team as Wesabe (in the article) are estimating - _He does encourage
his developers to spend most of their time working on things that they can
finish quickly_ and _When they need to, Wesabe developers will tackle bigger
projects_. You can't distinguish "finish quickly" from "bigger projects"
without estimating. It may be the quickest easiest form of estimating "big or
small", but it's still an estimate.

The reality of the argument is that a lot of organisations can make do with a
lot less formality and rigour in the estimation process. Moving from "vaguely
approximate" to "slightly more accurate" is an expensive process with all
sorts of cultural side effects that too many people assume is "worth it"
without any real justification.

 _But_ we all estimate, because it's pretty rare that you can justify starting
a piece of work that might take a day, or might take 10 years - I'll let you
know when it's done.

~~~
jacques_chester
I think that part of the problem is that when people say "estimate" in our
profession, they're thinking of capital-E-Estimates. With a whole bunch of
theatre and disruption and danger and political nonsense.

It's not the informed prediction of the value of some unobservable variable
that upsets us. It's the larger picture around it.

------
dreamfactory2
Of course the higher value a thing has, the less you need to quantify that it
has high value. Conversely, the lower your margin, the more you need to
control it to within an inch of its life. You could even say that if it needs
estimating it maybe isn't worth doing.

~~~
jacques_chester
Where I work we have weekly sessions to plan the week's work. We ask our
customer to put stories in order of value to them. It might look like this:

    
    
        A
        B
        C
        D
        ---- week marker ----
    

Then we perform a quick estimate for each of the stories. We can talk them
over quickly, ask clarifying questions and so on. Because of research on the
unpacking effect, I like to make sure to say aloud the sort of subtasks I
think will be involved, but sometimes we don't.

We quickly do a "shoot out" and then simultaneously reveal our point estimates
for the story we're estimating. On divergence we explain our reasoning.
Sometimes it takes a few rounds to reach a consensus.

So after this process, it might look like this:

    
    
        A    4 points
        ---- week marker ----
        B    2 points
        C    1 point
        D    1 point
    

Because we've got velocity of 4 points.

At this point the customer can change the order of stories. They might decide
that having stories B, C and D is actually better than having story A by
itself.

Just trading purely on value _or_ on estimated effort/complexity doesn't give
you the full picture. It's the ratio of the two per-story and _between_
stories that helps a customer make the best-informed decision.

~~~
dreamfactory2
Agree that you want to get a view on whether a task is easy or not - but this
is rarely understood until you have started. And frankly most advocates of
estimation aren't talking about T-shirt sizes but some kind of currency they
can tot up to measure 'productivity'. The danger with trying to predict it is
that you are getting deep into a specific implementation at that point and it
is exerting a pull on the project aims. I think that can conflict with
business value as the driver for doing anything. What's more important for
eliminating waste (and death march ftm) is a set of alternatives (outcomes and
implementations) and a framework which makes it cheap to back up and switch
routes when you find something isn't turning out to be easy thing you
imagined.

~~~
jacques_chester
We deliberately focus on simple estimates of a unitless measure (story
points), rather than imposing a unit that will cause distortions in our
thinking. The value of those points is specific per-project and it emerges
from actually doing the project.

I agree that allowing change is important. Reduce the cost and risk of
changes, and you will make changes as and when required. You'll also become
less preoccupied with perfect planning, because it won't be necessary to
manage risk.

There's two loops here. One leads to small, light, manoeuvrable projects. The
other leads to building bigger plans because the risk is higher, which means
that a bigger plan is needed, which raises the risk, so the plan is more
detailed so that ...

~~~
dreamfactory2
Impact mapping gives you that big plan without having to detail implementation
and keeps it manoeuvrable. Kanban lets you switch the backlog around through
the day and doesn't lock the project into sprints (which are mini waterfalls
imo). Estimation is entirely optional here. (The main benefit I've seen from
estimation is in forcing a dev team to collectively confront what's involved
in implementing a feature - as an input not an output.)

~~~
jacques_chester
I've found that Kanban leads to coordination overhead because you always have
too much work in flight in too many stages. I prefer story-by-story
development, which is the direction to which most Kanban implementations wind
up evolving. One small thing, to completion, at a time, per pair.

~~~
dreamfactory2
WIP limits should iron this out

------
blublub
The author questions the concept of an estimate in Software Development, and
proposes four sub estimates. I'm not sure that replacing one estimate with
four will do the trick...

------
jacques_chester
One of the things I sometimes say to our clients at my workplace is that
estimation can be done two ways. The traditional way is summative. Take all
the things you want to do, estimate them (which is where McConnell's book
really shines), then produce a range that expresses your likely outcomes.

The other way is emergently. Build a pile of things you want to do, put them
in rough order of value, then estimate a few things at a time. Then start
working. Only after a few weeks will you be able to begin to predict outcomes.

This is old news. I'm talking about velocity. Which is actually the perfect
word, because summative estimations are about a total distance travelled;
velocity is about the speed and direction you're currently moving in. Velocity
is the differential of the project's outcomes.

Velocity is a more powerful metric because it includes its own mechanism for
updating estimates. A classic "estimate smell" is that the first estimate made
is the _last_ estimate ever made. The closely related smell is "estimates that
fit the plan", aka "estimation by fiat".

Velocity, however, emerges from _observation_. No additional action is
required to update summative estimates. And what comes for free is typically,
in software methodology, all that gets done.

Another way in which moving to velocity as your estimating technique is that
you change what is fixed and what is fluid in discussions with clients.
Traditional project management takes the work breakdown as a fixed outcome and
then looks to optimise around that. Time, schedule and quality are then the
variables. Velocity allows you to vary scope and work as well. You can then
hold quality, time and schedule constant instead.

It's hard to convey how big of a deal this is in practice. It seems like a
minor alteration. But taking _seriously_ that the speed of light in a vacuum
is fixed (per Maxwell's equations) means that spacetime varies. Which makes
... a big difference to physics.

I have seen this change in thinking bring high-risk projects to a successful
conclusion under tight deadlines, with high friction, with high technical
demands, in short timeframes. We had the data to show where the project was
going, and that allowed us to focus on ensuring we were delivering in order of
value. It also allowed us to discuss with the client what was possible and
what was not possible.

Under traditional schemes we'd have had a fixed pile of features and the
project would've been a "failure". Yet it did everything that was needed on a
particular day, with very generous margins of safety.

------
blowski
OFF TOPIC RANT

Just have to register my frustration with the cliched "[x] considered harmful"
headline in the IT world. It devalues the proposition instead of lending it
authority.

EDIT: "Betteridge's law of headlines" also applies here ( _Any headline which
ends in a question mark can be answered by the word no._ ) Since it's an
opinion piece, state your opinion in the headline, and stick with it. The
question mark suggests even the author's not sure he agrees with the premise,
which makes me think the article is not worth reading.

~~~
julie1
If you don't rant how do you learn from your mistakes?

It makes you reconsider what went wrong like The Mythical Man's Month (tools
considered harmful aka silver bullets) by F Brooks the project manager that
nearly killed IBM and OS360 is such an example.

Coding is about creativity and innovation. Not about respecting the authority.
It is a hard mix of conservatism (standing on the giants' shoulders) and like
the other giants ... to rant.

At one point the best developers new ideas are rants ex robe pike vs Unix
(plan9, Go) (all his rants against unix (see cat) are a very good source of
knowledge for unix devs)

And rants are always instructive when done by great person.

* [http://harmful.cat-v.org/software/](http://harmful.cat-v.org/software/) (I learned a lot with [http://harmful.cat-v.org/software/symlinks](http://harmful.cat-v.org/software/symlinks) [http://harmful.cat-v.org/software/dynamic-linking/](http://harmful.cat-v.org/software/dynamic-linking/) ) * computer for cynics on youtube (a rant against computer industry in general)

“It’s a curious thing about our industry: not only do we not learn from our
mistakes, we also don’t learn from our successes.” — Keith Braithwaite

~~~
blowski
I meant that my comment was an off-topic rant.

I have no problem with the article itself (I didn't even read it), nor the
right of anybody to rant about anything. It's the cliched headline on which I
was commenting.

