
Ask HN: How do you deal with or improve software development estimates? - yawz
Estimating software development work is hard. There are typically many reasons for that, but this is something that we, software development professionals, have to live with. But thankfully there are ways to improve this process (e.g. breaking the project down into smaller pieces, staged delivery, conducting pre-mortems, regular status checks, etc.)<p>What are some techniques&#x2F;approaches you&#x27;ve successfully used to improve software development estimates?
Any recommended reading on the subject?
======
mikro2nd
No, estimating software work is _impossible_. I refuse to do it. If pressed
insistently I will "estimate" four hours.

I've encountered situations where I was told, "That's unacceptable." My
response was, "OK, two weeks." Need I say that that was also "unacceptable".
My point is that any "estimate" is nothing more than a guess, and irrelevant
in any case. A given feature/bugfix/whatever is gonna take as long as its
gonna take.

Estimation is a shibboleth.

~~~
gadders
I hope you never get any work done on your house.

"I've given you the architects plans for my kitchen. How long do you think it
will take?"

"It will take as long as it's gonna take. Please sign this blank cheque."

~~~
mikro2nd
There's a world of difference... People have been working on houses for... a
couple of thousand years?... there's not much that's unknown about house
building. Software building? Not so much. It's almost always all unknowns. New
libraries, new programming languages, new requirements never before dreamed
of, new everything. Your parallel with housebuilding is a false one.

~~~
gadders
If you can't give at least give a order of magnitude estimate, or an estimate
with documented assumptions, you don't know your job well enough.

------
AnimalMuppet
Pay attention to how long things actually take. Then, when you're estimating,
you can say "This is about like X, and X took a month, so I'll estimate this
as taking a month." That gives you a pretty decent estimate - as long as
everybody remembers that it's an estimate.

That assumes that you can find a good previous example. If you can't, you
can't use this technique.

One place that gets me is porting software. How long is it going to take?
Depends on how many issues we find. How many will that be? _I have no idea._
Neither does anybody else. We'll know we're done when we're done.

------
afarrell
I have been frustrated by the lack of resources on how to estimate software
tasks for the past 8 years and had essentially come to agree with micro2nd
that estimation was impossible. I was, however, searching for some method
which would turn it from [impossible task that I had no idea how to do or how
to learn how to do or how to ask how to do] into merely difficult work.

However, this past summer, I got a new manager who has had a goal of
introducing estimation into the team and I've been working with him. The
biggest thing I've learned so far is that if you're estimating by asking "how
long will this take?" then you're asking the wrong question. To estimate, ask
instead "What are the mile-markers?"

The work of answering this is to go through the research process of
determining what the task involves. This takes significant time and so it is
still important to stand firm and resist the urge to give a number when
someone asks you during a meeting if you haven't prepared. It also is still
fallible and I'm honestly not all that practiced at it yet. However, at the
end of it, you'll have mile-markers which provide the three pieces of real
value from estimates:

1) A rough intuition for the level of complexity of the task.

2) An opportunity to notice when you can cut down the scope of something or
when you should prepare for something by doing a small refactor.

3) A way to judge your velocity as you are doing the task.

The third one is particularly important and can make guessing a number an
actually-somewhat-useful activity. Why? You can guess numbers for the
individual mile-markers and then, if after a while of working notice you are
falling behind or moving faster, you can communicate with stakeholders.

\----

When I've asked for advice on estimation, I've often been told "just gut-check
yourself and multiply by 3. The important thing is to communicate when you
think you might not hit the estimate." But if right after producing a single
number and saying it, the alarm starts blaring in your head saying "I'm not
going to be able to make that estimate." well... you'd feel silly "It will
take two weeks. Oh I think its going to take longer than two weeks." and so
you never have a point where it makes sense to raise a flag. But if you have
mile-markers and a regular habit of checking yourself against them, then there
_is_ a defined point where you can raise an alarm.

------
tomohawk
Practice, and follow a process that enables engineers to internalize the
design as well as communicate it.

For example FDD encourages creating an agile model that the team agrees to,
and then creating a 3 level WBS for the next 3 months, with estimates.

Give it a go, and then when you realize your team didn't do well the 1st time,
call it and then try again. By the 3rd time you'll have built enough skill to
start getting within a reasonable ballpark.

------
amirathi
Earlier this year, I wrote these tips to improve estimation:
[https://blog.amirathi.com/2018/02/05/science-of-software-
est...](https://blog.amirathi.com/2018/02/05/science-of-software-estimation/)

I will summarize it here:

\- It's not just about code. New developers typically don't account time for
testing, documentation, data migrations, backwards compatibility work and such

\- Spend a few hours to discover unknowns. Unknown unknowns can rattle the
estimation big time whereas known unknowns can be factored into estimation
with some upfront research.

\- Have a deeper conversation with product owners. Often you realize that some
nice to have use case is taking 80% of the time and can be decoupled from the
core feature. Win-Win.

------
dr01d
Get 3 engineers to do estimation on the same work. Not only helps to get
better estimations, but also helps to understand people's strengths and
weaknesses on tasks as well as inform more people of a project specifics. Also
PERT: [https://www.techrepublic.com/blog/it-consultant/use-pert-
tec...](https://www.techrepublic.com/blog/it-consultant/use-pert-technique-
for-more-accurate-estimates/)

------
GFischer
It depends a LOT on what you're estimating.

Estimating a CRUD line of business app is very much doable. On the other hand,
estimating an innovative product using unproven tech or novel methods requires
a proof of concept before even knowing feasibility.

I worked with major consultancies, and the most serious gave you order-of-
magnitude estimates, with iterating improvements on the estimation itself
until you reached a +/\- 15% estimation that was the best they could do.

Very similar to: [https://www.construx.com/software-thought-
leadership/books/t...](https://www.construx.com/software-thought-
leadership/books/the-cone-of-uncertainty/)

In those cases, it was a very reasonable estimate because they'd done very
similar things before.

One of those consultancies used an tool and they had measured their teams'
velocity and were very confident on their estimates.

For one of my own projects, we had to estimate a rewrite of an existing app
(with slight improvements), which is as easy as it can get in regards to
estimation. We used several approaches, the one that ended up being closest to
reality was Steve McConnell's tool Construx Estimate (no longer available but
you can find a mirror here:
[https://web.archive.org/web/20170920093850/https://www.const...](https://web.archive.org/web/20170920093850/https://www.construx.com/Thought_Leadership/Books/Survival_Guide/Resources_By_Chapter/Construx_Estimate/)
)

------
agarbayo
The first thing to keep in mind is that reasonable accuracy might never be
achieved because of the nature of the problem. Social causes and bad process
contribute to bad estimates. It’s not only the difficulty of guessing the
unknown. Biases, guesses taken as deadlines and stakeholders trying to
influence the estimation results contribute to worsening the problem.

This article [https://www.infoq.com/articles/software-development-
effort-e...](https://www.infoq.com/articles/software-development-effort-
estimation) written by a researcher at Simula Research Laboratory is one of
the best summaries on the estate of software estimation.

Simula has excellent research on estimation. I found particularly interesting
the different biases that might affect us: the scale used, how the question is
asked (introduces an anchor to your estimate), the size of the task estimated
previously, etc.

However, we can improve on gut feeling estimates and training on a personal
level can reduce the uncertainty. A few highlights:

\- On a team level, planning poker is known to improve estimates.

\- Checklists (common forgotten activities) and historical data (% of time
spent on unplanned or project management) tailored to your current project
improve estimates.

\- Avoid early estimation based on incomplete data. Avoid exposure to
irrelevant, misleading estimation information.

\- Break specs in tasks and estimate those tasks. Keep historical data on your
estimates and final times and review them periodically. Automated time
tracking tools for developers like
[https://www.codermirror.com/](https://www.codermirror.com/) can help in
removing the burden of tracking coding times.

------
afarrell
One thing just occurred to me: the most successful estimation exercise I've
gone through so far actually involved me pair-estimating a project with my
manager. I think this is generalizable: If you aren't sure how to do an
estimate, the steps are:

0) Stand up straight and chamber your courage in order to be honest responding
to the person who just asked you for an estimate.

1) Say: "I don't know how to produce estimates. I'm going to get a more senior
engineer to pair with me on breaking down a project and producing an estimate
for you."

2) Email your team lead about who would be a good person to sit down with you
and go through the process of estimating the task.

Tell her that you are going to do #3 Say you are looking to both answers the
question you were asked and to talk you through the thought process so that
you can learn to be good at producing useful estimates. Book like 30 minutes
in the senior engineer's calendar.

3) Write your own draft where you first take the definition-of-done and make
sure it is clear, then you turn the task into bullet points of what _you_ can
think of are the subcomponents of it. Don't produce times yet.

4) When you have the meeting with the senior engineer, be clear that you want
to learn the thought process. If they say "I just go with my gut and multiply
by 3" then say something about how their years of experience with estimates
allows their intuition to be highly trained, but you need a more rigorous
process to follow until your gut is similarly well-trained. (tip: read
Thinking Fast and Slow) Ask if they can use their experience to help you think
through your breakdown, spot missing subtasks, and put numbers on things.

I've not actually tried steps #1 and #2, but I have done steps #3 then #4
together and they worked well.

=======

If you think you'll get pushback on the idea of taking some time to go away
and do the estimate, imagine yourself calling up an auto mechanic and saying
"Hey, my 2010 Peugeot Ion is making a noise like a coffee grinder when I
accelerate. Can you give me a quote for how much that costs to fix?" Then
imagine refusing to let them take a look at it before they give you a quote.

------
Jtsummers
1\. For estimates to become effective (in software) you need to estimate a lot
of tasks' sizes/complexities and then record how long they took.

2\. You need familiarity with the _type_ of system you're working on (whether
it's new development or sustainment doesn't matter, you need to know the
system). If the work is new, you will need to work on it for a period of time
to learn how to estimate it.

3\. Initially, your estimated sizes and times will be way off. Accept this.

Record the estimated task size, record the final time taken (at least to the
number of days needed). Run this through a stats package: You should see
clusters, each size category should take around the same amount of time. If
you see a size "small" or "1" taking as long as a "large" or "100", work on
your size estimates.

Once you get better at estimating the size, you'll be able to (more easily)
estimate the time (within some confidence interval).

I discourage estimating time directly ("Here's the task, I think it's 10 hours
of work.") because you really don't know it. And everyone in my office that
does the above and insists on estimating time (versus size/complexity) ends up
using a multiplier against their time estimate after a couple years! That is,
their time estimates become sizes after all. They keep calling it "100 hours"
but then double it or halve it based on the statistical analysis above. Better
to just call it a "10" and have the stats package spit out 200 hours or 50
hours or whatever is appropriate, eliminate the confusion.

============

For estimating longer term projects (not just tasks), you can do the above if
you've worked with similar systems (or the system being maintained) for each
of the tasks/features/bug fixes. String them together using a PERT chart
(illustrating dependencies between these tasks and the path to your
objective). This will give you a reasonable estimate for the overall project
time.

============

Finally, because you're using stats and know your standard deviation, you
don't give a precise target (100 hours). You give a range (100 hours +/\- 10
hours). The more you (or your team) does the same kinds of work or work on the
same systems, the tighter your bounds should become on your estimates.

------
sloaken
Lie:

Tell management it will take twice as long as you think.

Tell your staff they have half the time you feel they need.

~~~
BillBohan
I agree with afarrell that lying is not advisable. Lying is a very bad thing.
Convincing someone else to do a bad thing is even worse.

The method I used when I was Software Development Manager was to ask one of
the programmers how long he thought it should take (or estimate it myself).
Then I told management that it would take 3 times that long. We usually could
make that deadline but it was often close and sometimes a little late. I have
since learned that it will take PI times as long as you think.

Telling the programmers that they have half as long does no good. They will
not write code any faster. It may even distress them and a distressed
programmer produces less code causing it to take longer.

FYI

Consider carefully what I say; my first computer program compiled and ran in
the summer of 1967. I have typed in a program while both Bill Gates and Paul
Allen watched over my shoulder. I'm retired now but still work daily on a
subgame for minetest.

