
Software Development Estimates: Where Do I Start? - clarkm
http://diegobasch.com/software-development-estimates-where-do-i-start
======
biot
An analogy I like to use: "Here's a book of sudoku / crossword puzzles. How
long would it take you to solve all of them?"

Estimating this is similar to estimating a software system. You've seen
puzzles like them before, but you don't know the difficulty level of each or
which ones have tricky clues that cause you to rack your brain trying to find
the solution. It also nicely illustrates the 10x effect: some puzzle solvers
breeze through them while others take forever.

If you could break a software system down into a rigorous formal specification
that could be precisely estimated, it would be possible to use that
specification to build it automatically. Then that shifts the estimation up
the ladder of abstraction to one where you estimate the time to create the
rigorous formal specification.

~~~
jacques_chester
Suppose you're asked to estimate a single sudoku problem. That's basically
impossible, because n=1. In small samples from a large population, variation
is enormous.

But what if I ask you to estimate the time for 100 sudoku problems? For 1000?
What if, after 2 years of solving sudoku problems full time I ask you for your
estimate? What if I develop simple statistical models based on the number of
squares that are already filled in?

Already it has become possible to _improve_ your estimate, to make it more
_accurate_. The value of an estimate is balanced on the line between the cost
of its uncertainty and the cost of its development. Over time, for any class
of problem, the cost of developing estimates falls.

~~~
zeteo
> What if I develop simple statistical models based on the number of squares
> that are already filled in?

What if you were unable to? NP-hard problems are weird like that. What if the
underlying statistical distribution had random variance or some other strange
features that make traditional statistical models useless?

Maybe the issue is that we're trying to use intuitions developed with everyday
problems that are essentially linear ("mow the lawn", "drive to the store") in
places where they're not appropriate. Unlike usual everyday tasks, NP-hard
problems (such as Sudoku or coding) have no known "royal road" \- sometimes
you just have to try all possible solutions.

~~~
jacques_chester
NP-hard problems often have "good enough" solutions that don't take heat-death
time to solve. Consider the TSP. You can wait until the last proton decays and
tie up all matter and energy in the universe.

Or you can use an ant colony optimiser and accept that you will probably only
come to within a few percent of the best possible solution. Oh noes!

And so it is with estimation. We can build an ever-more-elaborate estimate
that will consecutively shave fractions of a percentage point of precision and
accuracy. Or we can accept that _they are estimates_ , that they are meant to
be imprecise and that accuracy is a range, not a number.

------
swanson
Most people would agree that practice makes perfect, right? I think a big
issue with software estimation is that it is hard to get practice. I'd done
full project estimates for 4-6 projects in 3 years of working professionally.
Compared to the amount of practice I have writing code, this is nothing. How
can I ever hope to get good at something if I only do it once every 3-6 months
and it might take 2 years to get feedback on if my final number was close to
the actual cost? And unlike other skills, I can't practice on my own or take a
course (I've never seen an open source project that did round-trip estimates,
maybe that's something to try out...)

Another issue is when you have an "indirect estimate" \- basically I will
estimate the work, but someone else is the one that ends up doing the project.
If you aren't careful to consider who will be doing the work, you might
estimate too low (if you are an expert and the work is done by a bunch of new
hires).

And none of this even touches on misinterpreting client demands, scope creep,
dev team turn-over, or often neglected timesinks like documentation and
meetings.

~~~
jacques_chester
Perhaps you could try scaling estimation down, instead of up.

For example, in the SEI's Personal Software Process you estimate very
frequently. Like agile, it works by attacking big problems in lots of small
pieces. So instead of just doing the 4-6 whole-of-project estimates, you can
also do estimates for stories, estimates for modules and so on.

The other thing to consider is re-estimating as you go. The Cone of
Uncertainty demonstrates that as a project progresses, uncertainty about the
problem and solution domains diminishes and so estimates can be made with
tighter ranges.

------
wpietri
My main trick these days for this problem is to discourage estimation by
pointing out the costs of it and putting the burden on the requesters.

One great way to do that is to release early and often, allowing stakeholders
to change plans in response to what they've learned. Instead of doing the work
of (re-)estimating a bunch of stuff every week, their focus on what's actually
going on lets them stop obsessing about arbitrary dates and semi-fictional
plans.

People very rarely need estimates. They want a sense of control. They want to
manage risk. They want to convince people that money is being well spent. They
want to avoid looking like fools. If you solve those problems without using
estimates, people generally stop caring.

~~~
swanson
I agree with that estimates are largely about a [false] sense of control, but
this is a tough sell when dealing any company with budgets and stakeholders
that have to go ask their boss for a finite number of dollars to trade for
software.

Often times, this is a one-time ask. You get money allocated once, so you
better get enough to build the whole thing. You can't go back and say, "hey
boss! look what we got with the first $10k, can I have $10k more?"

If you are a startup or dealing with a 4-figure project from a local business,
then sure, you can probably skip estimates.

~~~
wpietri
Sure, but that's an artifact of traditional budgeting. In the Lean world (as
in Lean Manufacturing), traditional budgeting is considered a dangerous and
misleading activity.

Another way to work it is to have a backlog of valuable things to do that get
handed out to teams. When somebody needs something to do, you pick the
highest-ROI project from the backlog and give it to them.

In that context, detailed specs and careful estimates generally turn out to be
wasteful. If you're trying to pick projects by expected ROI, then making the
software estimate more precise than the (generally handwavey) business value
estimate is pointless.

~~~
facorreia
> you pick the highest-ROI project from the backlog

How do you calculate the return on investment without estimating the
investment?

~~~
Anchor
You don't have to know the ROI value of each project, just compare their
relative sizes to select the best from the set of projects with the highest
profit ranking.

This assumes, though, that the best of the projects is worth building in the
first place.

~~~
jacques_chester
How do you know their relative size? Isn't that an estimate?

------
annnnd
Not sure where I first heard this (and it applies more to single developers),
but I find this very true: when estimating the time needed to perform
something, try imagining how much time ANOTHER person would need. In general
we underestimate the amount of work when thinking in first person; we are much
more accurate when assessing the ability of others.

------
brudgers
Knowing a bit about estimating construction, and a bit more about the
estimating of design which is more akin to software, there's more to the
ability to execute according to a time table than the author gives
credit...and the fact that the process can be separated between design and
construction gives a clue.

Mature processes for delivering construction start with a budget and something
called a program - an architectural program being a design and implementation
independent description of the project's components and the relationships
among and between those components.

Then there is the matter of age. Architects aren't worth a shit until they hit
about sixty. Those running big designs have decades of experience. In the US
the median age for initial licensure is 33. It's not twenty-something's
freshly out of college, or even parents with grade schoolers organizing the
process.

Then, in the US, there tend to be standard contracts which describe industry
standard milestones and acknowledge that nobody really knows how things will
change over what is often a multi-year cycle. Buildings are delivered
reasonably on time and within budget because the process of contracting for
the work doesn't require reinvention - even the plumber's subcontract is a
standard form and tied to CSI format specifications which are tied to industry
standards and to ANSI materials standards and the building codes.

Nobody roles their own using the coolest new fad. It's Java, not Haskell. If
you read Hamurabi, you'll see the source of those traditions.

~~~
wpietri
The software process can not actually be separated into design and
construction. Software is a 100% design activity. Construction is what our
compilers and installers do. Projects that appear otherwise have created
artificial barriers to prevent certain kinds of design at certain times.

You're also missing that repeatability in software is a sign of waste. If
developers are doing things that are truly well-understood and predictable,
then they're doing something wrong. They should download a library or extract
a framework, automate the predictable parts, and shift their attention to
something worthwhile.

~~~
brudgers
As an architect, my business is 100% based on estimating my design time.
Within that design time I also estimate construction time.

What allows me to do both is not only my personal 20+ years of relevant
experience, but also the collective experience of the industry and design
methodologies that reflect that experience. I.e. architectural designs are
delivered in moderately well understood stages of increasing concreteness and
there is an established language for communicating designs to clients (plans,
renderings, project manuals).

Furthermore, repeating the same operations in the design of a building is no
less a waste than in the design of software. The same may be said for actually
constructing a building. Programmers have no monopoly on the ideal qualities
of laziness, impatience and hubris. It's just through experience that I have
learned that at some point the nail gun has to come out if the plywood is to
go up.

Or the pencil gets put to paper if there's going to be a design - and note
that it is not uncommon for good designers in the software world to start on
the drawing board not the computer.

And this, I think gets at some of why people have so much trouble estimating
software. They start designing the building having already decided on its
structural system and hvac layout and light switch locations.

Choosing Django over Rails over ASP.NET before diving into the problem means
that the tool has to be made to fit the job rather than delaying that
decision.

Now, I'm not suggesting that this is always the case. But to put the
difference in collective and individual experience in perspective, one of my
mentors [a formal mentor as part of the formal process of becoming licensed as
an architect] was Ronn Ginn, an architect known locally but not somebody
anyone has heard of. Anyway, Ronn is still practicing. He started practicing
before there was FORTRAN.

Think about how many Ask HN's there have been based on the idea that 28 is too
old.

Frank Gehry, who you probably have heard of, is about the same age. His first
project to receive any sort of attention came when he was 42 in 1971. He was
nearly 70 when Bilbao gave him international fame.

------
philbarr
One thing my Dad always told me about software estimates: "Take the estimate,
double it, and increase the time unit by one. So if they tell you it's a
couple of days effort, that's really four weeks..."

~~~
yannis
Use the pi rule. Multiply the estimated duration by pi. Works for many other
things too, besides software.

~~~
arethuza
Multiplying by a value between 3 to 4 seems pretty common - the explanation
I've heard being:

\- One to work out what you should be building ("I have to tel you what I
want? I might as well build it myself")

\- One to build it

\- One to get in working in the required way (allowing for "That may be what I
asked for, but it's not what I want")

And a bit more for contingency...

NB The above approach is based on an assumption that the real difficulties
come from working out what to build - technical risks are generally easier to
address (prototypes etc.).

------
DanielRibeiro
A gret book on that is _Agile Estimating and Planning_ by Mike Cohn[1].

Having that said, there has been a lot of controversy on the value of
estimates recently:

 _Estimation is Evil_ [2]

 _Purpose of Estimation_ [3]

I believe their TL;DR comes from the second one:

 _For me, estimation is valuable when it helps you make a significant
decision_

[1] [http://www.amazon.com/Agile-Estimating-Planning-Mike-
Cohn/dp...](http://www.amazon.com/Agile-Estimating-Planning-Mike-
Cohn/dp/0131479415)

[2] [http://pragprog.com/magazines/2013-02/estimation-is-
evil](http://pragprog.com/magazines/2013-02/estimation-is-evil)

[3]
[http://martinfowler.com/bliki/PurposeOfEstimation.html](http://martinfowler.com/bliki/PurposeOfEstimation.html)

------
jacques_chester
> _The gist of why estimates are hard: every new piece of software is a
> machine that has never been built before._

I am tired of the argument that we unique snowflakes amongst all professions
and that consequently we deserve special treatment. We aren't. _All_
professions deal with uncertainty and most of them deal with it
deliberately[1]. Throwing your hands in the air because a perfect prediction
is impossible is just silly; an estimate is _by definition_ an _uncertain_
statement of an _unknown_ variable.

Estimating software can be difficult because there are many points at which
complexity can be multiplied (McConnell's example of the requirement "validate
phone numbers" shows a span of minutes to months)[2]. But when you see very
wide ranges on an estimate, it's a signal that the problem is poorly
understood.

Outside of _genuinely novel research_ , improving estimation accuracy is
possible and valuable. And how often do we invent publishable new algorithms
for graph traversal?

I have skin in the estimation game, as I am currently developing a tool for
performing estimates[3]. I've also been researching the topic of estimations
generally.

I'm mostly amazed at how people in my profession try it once, get an
inaccurate result, and then decide -- "That's it! It's impossible! I tried it
one time and it didn't work!". If you go into a gymnastics club and you can't
do a backflip, that doesn't mean backflips are impossible in principle. It
means _you_ can't do it. Estimation is a skill too.

[1] See Petroski's _To Engineer is Human_ , I reviewed it here:
[http://chester.id.au/2013/07/07/review-to-engineer-is-
human-...](http://chester.id.au/2013/07/07/review-to-engineer-is-human-the-
role-of-failure-in-successful-design/)

[2] In _Software Estimation: Demystifying the Black Art_.

[3] [http://confidest.com/](http://confidest.com/)

~~~
wpietri
_> I am tired of the argument that we unique snowflakes amongst all
professions_

Aren't we, though? Zero marginal cost of production is at least very rare. It
basically means that to the extent that our projects aren't novel, then we're
wasting time duplicating something that ideally would have been copied from
elsewhere.

And that not even getting into the very high rate of change in tools,
techniques, and materials (e.g.,
[http://www.jcmit.com/mem2013.htm](http://www.jcmit.com/mem2013.htm)), which,
if not completely unique, is certainly unusually high.

 _> But when you see very wide ranges on an estimate, it's a signal that the
problem is poorly understood._

That is occasionally a sign of idiocy. But at least on the projects I see,
it's much more often a sign that it's an interesting problem. Indeed, if
you're delivering software in a competitive market, it's guaranteed the
problem will remain poorly understood, because your competitors will be doing
their best to create changes in the landscape.

~~~
jacques_chester
> _Aren 't we, though? Zero marginal cost of production is at least very
> rare._

Every profession can come up with a comparable point of uniqueness. Perhaps I
should have said "special snowflakes" instead of "unique snowflakes".

> _It basically means that to the extent that our projects aren 't novel, then
> we're wasting time duplicating something that ideally would have been copied
> from elsewhere._

This is a good argument. It reminds me of DeMarco's argument that as time goes
on, software engineering becomes less and less about process, because anything
that is repeatable will be automated. All that's left is the hard bits that
can't be automated.

There is, however, an enormous amount of duplication in our industry. And even
when we are adapting an existing system, we're still only rarely performing
acts of genuine novelty. Carpenters produce many items, none of them quite
identical, but they aren't inventing new methods of carpentry or entirely
different kinds of furniture on every job.

> _But at least on the projects I see, it 's much more often a sign that it's
> an interesting problem._

I envy you.

~~~
wpietri
_> Every profession can come up with a comparable point of uniqueness._

Not in ways that have such dramatic effects on predictability of costs.

 _> There is, however, an enormous amount of duplication in our industry._

Sure, but if we're going to put our attention somewhere, I'd rather we work on
eliminating that waste, rather than standardizing the waste so that we can
better estimate the cost of doing work we didn't really need to do.

~~~
jacques_chester
Your last remark is true, but a bit falsely dichotomous. The choice isn't
between perfect estimates and zero wastage. In practice all projects lie on a
continuum between totally novel and pointless duplication. Within that
continuum we can draft estimates of varying accuracy and I don't see a reason
why we shouldn't.

~~~
wpietri
The amount of time we get is fixed. Estimation is, from the Lean persepective,
muda (normally translated as waste). If I can be spending that time instead on
an activity that reduces the waste of duplication, I'd rather do that.

I should be clear that I'm not totally opposed to doing estimates. I've done a
lot of them, I'm good at it, and there are circumstances where I would do it
again. Some waste is temporarily necessary.

What I'm pushing back against here is the common assumption (which maybe you
don't have) that estimates are generally a good thing to do, or, often, the
only way to do things.

~~~
jacques_chester
> _If I can be spending that time instead on an activity that reduces the
> waste of duplication, I 'd rather do that._

Estimates can prevent waste by giving us a better of idea where to redirect
effort. Time is, as you say, doled out at a constant rate.

The highest ROI possible for an estimate is to prevent wasteful effort from
proceeding.

Timeboxed systems still perform estimation, you just hold different variables
constant. In flow-based systems you can perform estimates by taking an
integral of the current output.

My point is that estimation is _always_ present, even when we go out of our
way to say that it isn't. They lie on a continuum from gut feel to
supercomputers and conference papers. Estimates are always present because
humans must always reason under conditions of uncertainty; the future is
strictly unknowable.

Part of the confusion here is that I think there are two or three definitions
of estimate in this thread. I am talking both about the universal case and the
methodical estimate-with-an-E case. The universal case is that all decisions
leading to action or inaction involve estimation. The methodical case is that
estimation for software development is possible and tractable outside of
genuine research.

------
jacques_chester
My favourite book on software estimation is McConnell's _Software Estimation:
Demystifying the Black Art_.

As usual he takes a vast body of literature and boils it down into a chatty,
usable book. The tables and checklists are worth the sticker price on their
own.

~~~
do-it-good
"As usual" for what - a book, the author, a book series called _Demystifying
the Black Art_ ?

~~~
jacques_chester
As is usual for McConnell. All his books are excellent.

------
zackbloom
My key advice is to not pay attention to the minutia of the task, if you do
you'll always underestimate. A better method is to forget the specifics and
ask yourself 'If a buddy told me he was doing this, how long would I guess it
would take?'. We have more experience hearing about projects and how long they
took then we do really estimating the duration of creative tasks by reduction.

~~~
louthy
This is finger-in-the-air approach, which seems doomed to fail. For a start,
it relies on your buddy having exactly the same career experience as you, and
any new project being an almost exact match to previous projects you have
worked on.

It doesn't take into account the unknowns, and doesn't break the task down
into components that you could at least attempt to quantify.

Personally I go through a short mental breakdown of the project, trying to
partition it in my mind into smaller jobs. Based on my years of experience I
will then have a gut feeling for certain aspects of the project.

If any job is more than 3 days long, then I try to break it down further.

If I can't mentally break it down, then I need to spend some time researching
why that is (investigate the unknowns).

Once everything has been broken down, and I have reduced the unknowns as much
as possible I add contingency. Anything that I feel has the potential to be
problematic I will add more contingency to, or add additional research time.

------
bcantrill
Completely agreed -- and I hit on the same themes on a talk on software
engineering management (and its pathologies) at Surge on Friday[1]. The date
fetishization so common in software engineering management is a direct result
of non-technical management, who do not understand that every piece of
software is solving a heretofore unsolved problem (even if a teensy tiny one)
-- and that the unknowns often cause software to take much longer to build
than one would anticipate. This gives a kind of CAP analogue for software:
schedule, quality, features -- pick only two.

[1]
[http://www.slideshare.net/bcantrill/surge2013](http://www.slideshare.net/bcantrill/surge2013)

------
jwilliams
There is a way to get a perfect estimate - and that is to build it and see how
long it takes.

Everything else is using an abridged model is short-cut the process. However,
I think this is an important point. Estimating is somewhat akin to actually
building the end-product.

The reason I find this important is that the most common spiral of death I see
is re-estimation. You take on something unrealistic, plough on regardless,
realise too late, then you re-estimate. The re-estimation causes a project
stall and takes time. The end result being you have less time.. Few months
later you're in the same boat again (repeat).. Big organisations are really
prone to this.

~~~
jacques_chester
> _There is a way to get a perfect estimate - and that is to build it and see
> how long it takes._

The concept of a "perfect estimate" is tautological, isn't it? Estimates are
by their definition uncertain statements.

~~~
wpietri
I believe that's his point.

~~~
jacques_chester
Then the larger point is that if estimates are always imperfect, we need to
stop obsessing about their always being imperfect.

~~~
jwilliams
Yeah, more or less :) I think admitting/deciding upfront what you want to be
imperfect is a good start.

------
xarien
"The gist of why estimates are hard: every new piece of software is a machine
that has never been built before."

This statement is simply not true. Every new piece of software can be broken
down to pieces that are very similar to other software that had been done
before. This is actually the secret to providing a good estimate.

What generally screws up estimates are complications that arise. Much of it
actually has to do with pre-existing code as opposed to new code. If you were
to build something from scratch and had a team of experienced engineers, I'd
bet that you'd get a pretty damn accurate estimate.

~~~
dreamfactory
Disagree entirely. If "Every new piece of software can be broken down to
pieces that are very similar to other software that had been done before", you
are talking about assembling library components, not programming (though as
you say even that often comes with its own unforeseeable difficulties).

This is the fundamental paradox of software development estimation (and
separate to the uncertainty of external forces): when you try to estimate
development you are estimating design of something that has never been made
before. If you aren't, you should probably be buying a product instead of
building. It's a fool's errand basically.

------
DanielBMarkham
"...The gist of why estimates are hard: every new piece of software is a
machine that has never been built before..."

Yes, and that's why a lightweight, repeatable estimation process beats every
other way of doing it. As you continue to estimate, you create and refine a
mental model of the project's complexity. For some projects, you're able to
create a mental model that has high fidelity quite easily. For others, it
takes a bit of work. The entire article here was an exposition on this fact.

Some folks figure that out and want to just give up. Estimation is impossible!
Other folks, however, are going to figure it out if it kills them. What
happens in these cases is they start to list every possible variable that
could be involved in such a model in every scenario, and then create one uber,
ultimate, super model that works in all situations.

Over time and through lots of trial and error, both approaches have been found
to be bullshit. Instead of giving up, or creating more and more complex models
that take more and more time to work, with each project you're better off
starting with the most ludicrously simple model you can and then adding
complexity as needed. The trick is incremental complexity, repetition, and
convergence. If you have that nailed, the details of your actual model, oddly
enough, do not matter that much.

Obligatory link to previous comment:
[https://news.ycombinator.com/item?id=6389227](https://news.ycombinator.com/item?id=6389227)

~~~
dreamfactory
Interestingly, Thoughtworks have observed that over a project they get the
same consistency simply counting number of tasks as counting effort estimates
for those tasks. The key for project management is therefore to focus on
maintaining a backlog of manageable tasks and throughput, not in estimating
them individually.

~~~
DanielBMarkham
We're finding this out in a lot of places. The key issue is variability.
Larger sample sizes usually lead to less variability, which means flow-based
systems will hold up better.

If you've broken your work out into tasks, instead of stories, presumably you
have a lot of them, and they're mostly the same size over a large set. So
sure, should work fine.

If, however, you have a small number of highly-variable chunks of work, then
flow-based systems fail. It all depends on the nature of the item pool.

~~~
jacques_chester
Summation can _apparently_ reduce error without, actually, reducing error. I
wrote an article about different ways of measuring estimate accuracy[1] that
explains the difference.

It's a question for each business as to whether that matters.

[1] [http://confidest.com/articles/how-accurate-was-that-
estimate...](http://confidest.com/articles/how-accurate-was-that-estimate/)

------
ollysb
I think the problem is that developers(including myself) tend to make
estimates assuming that they're not going to find anything that makes them say
WTF! Code is never perfect and it's extremely difficult to know how much time
it's going to take to dissect and restructure code to accommodate your new
feature.

~~~
jacques_chester
This is called optimism bias -- when estimating, we forget the things that can
go wrong.

------
alok-g
>> "The gist of why estimates are hard: every new piece of software is a
machine that has never been built before."

What is not explicitly said here is that a good developer needs to know what
would be a "new" piece of software. He should be at least vaguely aware of
what has been built already, what is within the reach of the state of the art,
etc. And just like in the stock market, it is impossibly hard to know all the
needed information to come up with wise judgement, complicating the estimation
process even more. Yet, this is an essential piece in estimation anyways. Just
for example, for some projects you may simply declare them to be beyond the
state of the art.

------
rogerbinns
I address the problem by giving a range. Giving a single number is the real
problem. For example I might estimate something as two weeks plus/minus six
weeks, or a month plus/minus a week. That range is because of the usual
factors (scope creep, things that have never been combined before, testing,
meetings, holidays etc).

Sometimes the questioner picks up on the two weeks minus six weeks is a
negative number. I then explain that existing functionality could be used,
goals could be achieved in other ways, or maybe it just isn't really needed.

------
gte910h
[http://www.amazon.com/Software-Estimation-Demystifying-
Pract...](http://www.amazon.com/Software-Estimation-Demystifying-Practices-
Microsoft/dp/0735605351) is fantastic for giving many approaches to the topic,
and counseling for non-technical stakeholders where it isn't a negotiation,
but math.

------
rrhyne
Always provide two quotes, the "shoot the moon" and the budget quote. The
client will always cherry pick segments from the more expensive quote to
arrive right at their budget. You can still loose to the next guy, but if you
are in the ballpark, it won't be because you came in (a little) too high with
this approach.

------
jasonhanley
Simple 5-step methodology: [http://blog.pmrobot.com/2011/09/5-steps-to-
estimate-software...](http://blog.pmrobot.com/2011/09/5-steps-to-estimate-
software.html)

------
known
Good judgement comes from experience. Experience comes from bad judgement.

------
jlebrech
Minimum of 2 weeks for anything you think might take more than 4 days, if it's
a day then double it. etc.

------
molson8472
I really like this statement: "Every new piece of software is a machine that
has never been built before. The process of describing how the machine works
is the same as building the machine."

That's probably the best way of relating the problem to non-engineers that
I've heard.

------
grantph
I ALWAYS quote fixed price projects regardless of scope. Some software
engineers would say that's highly risky. For me, it tends to be highly
profitable because my estimates are usually accurate. If anything, I've learnt
to over estimate and impress clients by delivering on promises.

I am amazed by the number of people who try to portray computer science as a
pseudo science like voodoo, where estimates are impossible because the
problems are unknown. Just because information is abstract doesn't make it any
more difficult to estimate than something physical. Computer science IS a
science and software engineering IS engineering. Good engineering involves
breaking problems down into small manageable pieces that are easily understood
then planning how to implement them. This practice is called design. On more
complex projects it's architecture - a sexier form of design.

Let's get back to basics. Who has heard of "the software development life
cycle"? Everyone, I hope. It's modeled off something from the early 19th
century called the product development life cycle. It needs to be mentioned
because IT people like to think they're special and mysteriously invented
SDLC. Feasibility, analysis, design, development, testing, deployment, etc.
Agile methods are simply a minimal version of this repeated fast and
frequently. Release the minimum viable product then iterate. Unfortunately, so
called "hacking culture" has led people to jump straight to development
without considering analysis or design. Real hackers design their attacks
first. Successful hacks are well thought out and executed in small brilliant
steps.

The analogies of sudoku in defining a problem and construction in defining a
project are perfect, so I'll borrow those.

If someone presents me with a 1000 sudoku puzzles of varying difficulty, the
first step in estimating is to DESIGN a solution. Allocate time to categorize
each puzzle by complexity. Then estimate times based on complexity based on
past experience. Anyone who says they can't do this because they have no
experience with the problem should be immediately fired. You've obviously
misrepresented your experience and/or skills as an engineer of sudoku problem
solving.

If a problem is truly unique and novel (and very few are nowadays), then
simply factor in time during the analysis & design to perform a few
experiments that will more accurately help with estimation. Or training. OR
HIRE SOMEONE WHO HAS DONE IT BEFORE to help with design and estimations. Then
find the most effective resource to implement (solve) each puzzle.

As for construction, any idiot can slap a few lines down on a piece of paper
and call it a design for a house or a building. That's the way some people are
building software. Real architects and engineers exist because construction
can be broken down into hundreds of thousands of pieces and estimated
accurately. They know exactly how many bolts it will take and what types of
contingencies to allow for. Software, although abstract, is no different. The
software industry has been around for 40+ years and it's modeled off past
industries like construction and manufacturing. It can be accurately designed
and estimated.

Once a clear design exists, estimation is easy.

My preferred estimation method is function point analysis. It's simple. Break
each problem down into the smallest manageable piece. Any piece that takes
longer than a couple of hours is too large and should be broken down further.
That's an excellent rule of thumb. Anything longer than a couple of hours also
suggests the design was poor.

Of course, in some organizations this fails due to a lack of clear process. An
analyst will be tasked at collecting business requirements to write a
specification. That is often full of nonsense written by someone with no
design experience. That specification is given directly to a developer for
estimation and development. Where was the design? The nonsense works it's way
into the product. It becomes an estimation nightmare. Projects and budgets run
over or worse, fail.

How do I know this works?

When I break a job down for a client into small manageable pieces I know the
entire scope and can estimate accurately.

Clients love it because they can see every single piece of the puzzle in the
design. The quantity and times for each piece look reasonable when broken
down. The clients begin to appreciate the scope but ultimately don't care how
it's built or what technology is used. They just want to know how much and
when? If there is a deadline? Either add more resources or cut functionality.
Let them make that choice. It's not my concern.

The cream, and why this industry is so much better than construction and
manufacturing, is to look at all the pieces and see the re-usable patterns.
Design optimization. Copy, paste and re-use as much as possible. Clients don't
care about re-usability. I charge top of market rate and still manage to be
cheaper than competitors because my estimates were realistic (as opposed to
their crystal ball methodologies). Re-usability and working smarter then gives
me 4-5x that hourly rate. Shh! Don't tell the clients :)

If you can't estimate software accurately, you're clearly in the wrong
business!

~~~
reustle
> If you can't estimate software accurately, you're clearly in the wrong
> business!

That was a terrible closing statement to a pretty helpful writeup. Software
estimation isn't something you're going to be good at off the bat. So with
your logic, nobody new should be getting into any software development were
estimations are requested (most situations)?

------
forgotAgain
Does the customer understand what they are asking for?

Can they afford it?

------
airlinenut
Some additional notes/suggestions on estimation on Stackexchange:
[http://freelancing.stackexchange.com/q/495/67](http://freelancing.stackexchange.com/q/495/67)

