
Software Estimation Games - cpeterso
http://www.thomsett.com.au/library/item/estimation-games
======
nlawalker
Just like how most people who ask for estimates are actually asking for
commitments, most articles that state they are about estimates are actually
about commitments, including this one.

Estimates and commitments are opposites - with a commitment, the _giver_ of
the commitment is under pressure to deliver. With an estimate (an _actual_
estimate, not "I'm using the word 'estimate' but I really want a commitment"),
the _receiver_ is under pressure to use that estimate to control the project
to meet its targets. That's all an estimate is good for, so there is no other
reason to ask for it... which is why hardly anyone does.

By definition, you can't negotiate an estimate. The subject of a negotiation
is always a commitment, so if you know that the number you're going to bring
to the table is going to be negotiated, make sure it's a commitment you bring,
not your personal estimate. The time-honored tradition of "doubling and adding
some" is simply the process of converting an estimate into a commitment before
bringing it to the table.

------
DanielBMarkham
"...It is our belief that over the 30 plus years of commercial computing has
developed a series of sophisticated political games that have become a
replacement for estimation as a formal process..."

No. You are confusing the natural inclination for people to negotiate with
some kind of slippage into "politics" (The word "politics" is most often used
to describe "system of people stuff that I do not understand and that does not
make sense to me")

For thousands of years, people have had a natural way of dealing with
commitments where incomplete information is present. It's called
"negotiation", and it's done by various techniques and tactics such that when
all is done both sides are happy. We've also had ways of numerically
determining tangible things. It's called math or science.

These are completely different things. What I see all the time is scientific,
technical people flummoxed at how business-minded people are treating them.
And the reverse is equally true. The reason is that both sides are working
inside a completely different universe.

I like this article because it shows how multiparty negotiations can lead to
disaster, but I do not like it because it confuses the true nature what's
going on.

~~~
sdenton4
Politics can be understood in a narrow sense (members of political parties
performing kabuki dances suggesting a form of governance), or in a broader
sense, as what happens when multiple humans try to make a decision.

I tend to think of politics in the broader sense, and suspect that the author
of the piece does as well. It's the process of negotiation between people with
different spheres of influence, when a single decision has to be made. Client
negotiations are absolutely political.

------
vermooten
I'm a development manager and so am obviously asked to give estimates all the
time. I found many years ago that no-one reads the qualifications that go with
estimates, they just want a date or a number of weeks' effort and assume that
it's a commitment.

I now play The Quality Game, which works every time. An example from a few
weeks ago:

I gave an estimate for a piece of work which has plenty of contingency. For
any number of reasons, when I gave the date, I expected a sharp intake of
breath from the client --- so I quickly added: "The reason for this is that
it's a complex piece of work - I've allocated 2 developers and one DBA for 6
weeks, and then one tester for 4 weeks. It's complex and important, and there
is no way that the team wants to compromise on quality. That's why it's taking
so long. Obviously, if we can do it faster, we will. How does that sound?" No
problem.

I've played the Quality Game when telling clients that they need to upgrade to
a later version, despite their having been told they wouldn't have to.
Upgrading takes time and effort. Again, when quality is at the center of what
you do, the logic is unassailable. ("WE could do it without you upgrading - of
course - but our strong recommendation is that you upgrade. We're not saying
this to piss you off, we're saying it because we want you to have the best
quality product possible.").

Dev teams and clients never want to compromise on quality. Sales people don't
care, they have to hit their figures, but it's 2 against 1. We win.

Thing is: it's not a game, it's just the truth.

(Great article BTW, will share with colleagues!)

------
georgeecollins
"...It is our belief that over the 30 plus years of commercial computing has
developed a series of sophisticated political games that have become a
replacement for estimation as a formal process..."

If there was a planning process that was consistently reliable for new,
complex software I think people would use it. And I think the adoption of
agile methodologies in software development is an example of people trying to
implement an improved planning process. Still, there is a lot of uncertainty.

So people negotiate, incentivize, and test conviction.. I guess that is what
the author means by politics. Those same games happen in most complcated
transactions and relationships. I think some programmers think this stuff
happens to them because they are deeply misunderstood. But it happens to
building contractors, real estate agents, lawyers, etc. too.

~~~
asgard1024
> If there was a planning process that was consistently reliable for new,
> complex software I think people would use it. And I think the adoption of
> agile methodologies in software development is an example of people trying
> to implement an improved planning process.

There are such processes. But they require understanding of the problem, which
takes time. I am not sure "agile" is really attempt at an improved planning
process, it's at best not doing planning and just marching onto the problem,
seeing what happens at regular intervals in the hope of better understanding
it, and at worst, things like "planning poker" where you put a bunch of people
into a room and let everybody make a (possibly educated but usually not very
much) guess from the top of their heads.

------
geebee
There have been a number of posts recently about software estimates, including
an emerging hashtag #noestimates.

I'd say that developers do pretty clearly prefer working in an environment
where they aren't required to provide estimates and meet arbitrary deadlines
(nobody likes hard deadlines either, but there's a difference between a window
to launch a rocket to study halley's comet and a manager who figured that
saying "march" rather than "may" might get the team to work a little harder).

This does't mean that developers don't like to work hard or produce good work,
I think they just greatly prefer to take on a task, work diligently at it, and
show progress rather than be pressed for estimates based on limited knowledge
that quickly morph into hard deadlines before the problem is adequately
understood.

Yeah, I'm stating the obvious. But thing is, it isn't a complete pipe dream. I
do know developers who have quite a bit of autonomy. While the do have to
answer for what they've been up to, they develop software and show the value
of that software. They don't take specs, give estimates which become
deadlines, and get reviewed based on how well they met those deadlines.

It's all about aspirations. I have a few for myself. I'd like to get to the
point in my career where my interview focuses more on my existing work and
contributions (github, blogs, speaking at conferences) rather than my
performance on finding cycles in linked lists at the whiteboard. I'd like to
work in a quiet office (happily shared) with a door that closes rather than a
loud, open office with back visibility. And yeah, now I can add in that I'd
like an environment where I can work steadily on a challenging problem and be
evaluated by what I have accomplished.

Just to be clear, nobody owes me a job on my terms, I have to earn it. But,
ahem, the loudest people in our industry seem to be the ones who think the
world owes them a talented developer on their terms. If you're having trouble
attracting and/or retaining talent, you might want to consider how developers
like to work...

------
pnathan
I had many long discussions with a previous boss about estimates. What it
boiled down to is this:

* He is on the hook for certain things to be done, and is being asked when they will be done.

* I don't have enough data to tell him in such a way that the organization can plan around it. Anything I tell him is purest moonshine and unicorn hair. Sometimes it relates to reality.

* If I give an estimate, I am essentially passing dishonesty through the organization. I am lying to him.

This did not go over well. We compromised: I gave a order of magnitude WAG.
Which, at the least, gives meaningful information and probably isn't wrong.

------
pixelcort
When asked for an estimate, one good prerequisite is: "with what confidence
level?"

Assuming estimating 100 tasks, how many can we safely go beyond the estimate?
20%? 1%? 0.001%

Then give the estimate based on some probability curve considering the known
unknowns.

~~~
tonyedgecombe
I don't think many of my customers know what a confidence interval is.

------
tempodox
It is agreeable to be reminded of the fact that psychology influences us and
our craft disciplines. We're so occupied with math, algorithms and automation
that this may slip our minds.

There is a popular psychology book, “Games People Play” by Eric Berne that
shows and explains many of those game mechanics. Everybody uses them, not just
PMs and bit crunchers.

~~~
drhayes9
This book! A great introduction to transactional analysis and a very useful
tool for disarming and defusing interpersonal strife.

This book helped me figure out that, in personal relationships especially, I
could raise the discussion by one level and talk about the game I was playing
in an effort to figure out why and get somewhere useful, emotionally. As in,
"I think I'm just playing 'Why Don't You - Yes, But'. I need you to just
listen to me while I vent."

Can't recommend enough.

EDIT: I should say that the edition I read seemed a bit sexist? I just ordered
the new edition so we'll see if any of that text changed.

------
rainmaking
This is a nice taxonomy of estimation games, but stopping the games won't
address the root problem: Software development is complex enough to be
fundamentally unpredictable. The games are just about hedging against people
not wanting to hear that.

~~~
jt2190

      > Software development is complex enough to be 
      > fundamentally unpredictable.
    

This is false.

The weather is an extremely complex system, yet we use weather forecasts all
the time. What's different between weather and software is that everyone knows
that weather is extremely complex, and we account for that when we think about
the weather in the future. ("Maybe I'll carry an umbrella today.")

The equities market is an extremely complex system, yet we use market
forecasts all the time. What's different between equities market and software
is that everyone knows that equities market is extremely complex, and we
account for that when we think about the price of an equity in the future.
("Maybe I'll diversify my investments.")

Forecasting the completion date of a software project is no more complex[1]
than predicting the weather or the future price of an equity. In no way is it
"fundamentally unpredictable." What's different is that people don't regard
project estimates as, well, _estimates_.

[1] It's probably a lot simpler.

~~~
john_b
> _" What's different between weather and software is that everyone knows that
> weather is extremely complex, and we account for that when we think about
> the weather in the future."_

Weather is governed by impersonal physical forces which can be described by
mathematics. Software is built by groups of people who most defitely cannot be
modeled as mathematical entities. Weather is a bad analogy here.

> _" The equities market is an extremely complex system, yet we use market
> forecasts all the time. What's different between equities market and
> software is that everyone knows that equities market is extremely complex,
> and we account for that when we think about the price of an equity in the
> future. ("Maybe I'll diversify my investments.")"_

Those market foreasts are often _wrong_. Even a company's own forecasts are
frequently wrong, despite having the optimal perspective from which to make
those forecasts.

Furthermore, diversifying investments is easy to do because the price
mechanism makes those investments fungible (in a liquid market, anyway). This
is not the case with software. Each piece of software is unique. You may
"diversify" your software by investing in multiple projects, but ultimately
you have a business need for a piece of software which does task X.
"Diversifying" by funding software that does tasks Y and Z will not help you
if the software "investment" in task X fails. "Diversifying" in three software
projects, all intended to do task X, will roughly triple your costs for doing
X. Again, equities are a bad analogy.

Software development is a process which can't be easily summarized by
analogies to existing systems. In fact, the term "software development" can
describe many different systems.

The real reason schedule estimation in software development is so hard is
because, regardless of which system of software development is used, the
problem of simultaneously optimizing to maximise a desired _quality_ while
minimizing the _cost /schedule_ is inherently hard in a mathematical sense.
Changes which promote one oppose the other, and the sensitivity of a software
development system to unknown and unpredictable future changes cannot,
obviously, be predicted in advance.

Your point about hedging one's bets is valid, but there are costs to doing so
which are not as simple as carrying an umbrella or splitting up a pile of
money across multiple equities.

~~~
jt2190

      > Software is built by groups of people who most defitely 
      > cannot be modeled as mathematical entities.
    

Putting aside whether mathematics is up to modeling human behavior, why does a
_project schedule estimate_ need to be modeled using _people_?

    
    
      > Weather is a bad analogy here.
    

I was using weather as an example of complex things that can be estimated, not
as an analogy for software development itself. Sorry if I wasn't clear about
this.

    
    
      > Those market forecasts are often wrong.
    

Useful forecasts/estimates are seldom binary, and "wrong" versus "right" is a
bad way to evaluate their usefulness. Estimates have both precision and
accuracy. For example if you want perfect precision (as in the wrong/right
example), you may in turn get a very low accuracy, i.e. the estimate will
often be incorrect. If, on the other hand, you can live with less precision,
you can often get higher accuracy.

    
    
      > The real reason schedule estimation in software 
      > development is so hard is because, regardless of which 
      > system of software development is used, the problem of 
      > simultaneously optimizing to maximise a desired quality 
      > while minimizing the cost/schedule is inherently hard in 
      > a mathematical sense. Changes which promote one oppose 
      > the other, and the sensitivity of a software development 
      > system to unknown and unpredictable future changes 
      > cannot, obviously, be predicted in advance.
    

Again, this is true only if you need very high precision and very high
accuracy. If you can live with lower accuracy and precision, estimation
becomes quicker, easier and cheaper.

So I stand by my original assertion that software estimation is not
_impossible_.

(edit: To remain consistent, I should say that software schedules are
certainly predictable, just not _perfectly_ predictable. Most teams can get by
just fine with _good enough_ predictions.)

------
bottled_poe
Once upon a time, I offered cost estimates on a per-feature basis. It didn't
take a lot of time for this strategy to bite me. Over the last few years I
have established more stable contracts and now I tend to give estimates for
larger blocks of work. My estimates per feature are still often wrong, but
this risk is mitigated by spreading the estimated effort across a large set of
features.

------
ThrustVectoring
A game I play that is not mentioned in the article: give an estimate in the
form of "I will not be done with this before %time." I'll often add in
something about giving a better time estimate for the entire thing when that
period of time elapses.

I usually build that time estimate by imagining the whole process going
smoothly. The vast majority of surprises are negative, and of unknown size, so
however long the project takes, it's going to be something bigger than the
everything-goes-right scenario.

------
workbin
I usually provide an estimate of the time it would take me to come up for an
estimate of the whole project.

Pushing for a 2 level estimation is definitely not easy, and if the person on
the other side of the negotiation refuses this, then I ask him/her to stop
playing the "Price is right" and spit out his time requirement already.

------
michaelochurch
This is an excellent article. It's a great synopsis of what's really behind
those estimates, whether in naked form or couched in that "story points"
bullshit of the "Agile" cult. I fucking hate all of it.

Here's my take. Programmers don't underestimate because they're afraid of
getting fired or yelled at. In this market, a good developer is unlikely to be
fired for honestly estimating. Yelled at, yes. Fired in worse economic
circumstances (e.g. 2003)? Possibly. Fired now? Doubtful. Rather, it's about a
desire not to be jerked around. Implicit in the demand for an estimate is the
threat to jerk the employee around-- either imposing stupid new
micromanagement ("Agile") or changing project priorities. No one wants to work
for the boss who jerks her around and never lets her finish anything, because
soon enough she gets to 3 years and while she's fixed bugs and added a few
features, she doesn't have the coherent project that would make the case for
promotion or transfer to her target group... because her boss kept jerking her
away from projects whenever his hindbrain misfired, he interpreted it as the
"this is taking too damn long" impulse, he got impatient, and pulled support.
It's a fight-or-flight reaction in the boss when that happens, and managers
tend to favor flight because fighters tend to get themselves fired. If you
work for a hair trigger "flighter", you can very easily get to 3 or even 10
years without ever actually finishing a project. And flighters _love_
estimates.

A good middle manager isn't one who exacts and reports reliable estimates. A
good manager goes out, wines and dines the executives, and gets sufficient
credibility for his people that no one ever has to give an estimate.

Anyway, programmers give misleadingly optimistic estimates-- not dishonest,
just aggressively optimistic-- because they want to be left alone so they can
actually produce something and move along to a better project. This is what
"Agile" doesn't get. The thing that motivates us is delivering a major project
that gives us enough credibility that we never have to make estimates again;
not delivering small cantrips that anyone could do, but in record time. I
think that "Agile" provides some structure for junior programmers, but it
doesn't provide an exit. Under the Agile Ideology, even the senior programmers
in R&D and architectural rules have to structure their work in terms of these
dumbass "iterations". Implicit to Agile is the absence of personal progress
for an engineer: no matter what you do, you'll always be working on feature-
level "user stories" instead of real projects where you call the shots, and
you'll always be subordinate to the business.

I've been doing this for almost 10 years and I'm good at it, so what being
asked for an estimate communicates to me is that _whatever I 'm doing isn't
really important_. I will do what is necessary to meet a real, hard deadline
(they're rare, but they exist) such as a bid deadline for a government agency
(where people can go to jail for corruption if they grant an exception to
someone who's a day late) but the silly deadlines that business usually
creates are just resource limits, and (sorry but) I'm too good to work on
something so unimportant that I'm not even trusted with a month or few of my
own time on it. If you won't accept delivery on my time, then it's not
important enough to be doing in the first place, because I will work very hard
to make sure it's done reasonably quickly _and_ well, and if you don't trust
me on that much, then you should just fire me.

In other words, the difference between "4 weeks" and "2 weeks" is going to
mean that I either don't get to do it or have to relegate it to extra hours,
then (unless there is a hard deadline, and not some silly target like "Q3
KPIs") it's just not fucking important enough for someone at my level of skill
to be doing. So give it to some junior who'll view absolutely anything as a
learning experience, and leave me out of it.

~~~
davegauer
I assume you were down-voted for going on a tear about Agile methods. I can't
speak to that since I've never been in an Agile shop (read about it plenty).

But I think what you have written is applicable to most developers in general,
particularly this gem, which I loved:

    
    
        "The thing that motivates us is delivering a
        major project that gives us enough credibility 
        that we never have to make estimates again..."
    

I also agree that the article was wonderful. I laughed out loud several times
because, like all of the best comedic material, it is firmly grounded in the
painful truth.

In the end, developers really do just want to build great stuff and I think
most of us _are_ acutely aware of the realities of costs and time in real
business. I sympathize with management's desire for estimates (and even the
various methods to somehow get a reign on development). But the unfortunate
truth is that the attempts often only serve to irritate and chain down people
whose only desire was to work like hell making quality software in the first
place.

