Hacker News new | past | comments | ask | show | jobs | submit login
Why Are Projects Always Behind Schedule? (priceonomics.com)
178 points by sidcool on Dec 19, 2015 | hide | past | favorite | 108 comments

There is actually a nugget of thought in the article beyond the explanations you always hear (and which make up all the comments in this thread as I'm writing this): Even if you base your project schedules on accurate estimates of the median time required for each step, you will still be behind schedule for most projects.

The reason is actually quite simple: when things go well, they can only go so well, but when they go bad, they can go really bad.

Or, in a more quantitative way: while each step of the project will be equally likely to take longer or shorter than median, the steps that take longer can take much longer, while the steps that go faster only have a limited potential for balancing out the delays.

According to this reasoning you should set targets based on expected time, not "median" time. You're saying the tail of "completion time" is long, and so the median is a poor estimator for expected time. But if you actually knew the distribution of times for task completion (which is implied if you know the 'median time'), you could simply use the expected value of this distribution and your projects would then tend to complete on time.

I don't think this is the correct explanation. I think it's far more likely that (1) people don't know the true distribution of task times, so estimates are just crap guesses based on hubris, what managers want to hear, etc., and (2) scope creep.

Another reason is a kind of selection bias: an organisation wants something built, gets offers from various software companies, and picks the cheapest.

That is, if you are doing the project, your company's initial estimate was the one that had the highest chance of being too low.

That's not only that, the field is full of bullshit. Advertising feature as available when they have not even been discussed seriously is the usual in the companies I have worked for the last 10 years. Our website and sales all go advertising "We support X. We are ready for regulation Y", while we are actually waiting for a paying customer to buy it to even look into it.

And that's all companies. You can't really be honest as all your competitors are similarly bullshitting. ( I used to be upset about that thinking I was always working for the black sheep, then over the years, you always end up working with your competitor one way or another and you find out it is the same everywhere )

There is also the deadline game. The client will push for earlier and earlier release date. The provider will accept because the provider knows that the client will not be able to test the product. I used to be upset to deliver code that would not even compile. Then over the years, we have had client not ready to test for several years. An extreme case, is a client that took a package I developed 5 years after delivering the working version of it.

That's the biggest problem I have had with Agile. Very often companies are not ready to support the lack of bullshit even internally - no more schrodinger status, no creative budget allocation - developer appear to be slower and cost more.

> An extreme case, is a client that took a package I developed 5 years after delivering the working version of it.

You completely lost me there, what happened exactly?

A client wanted to run our application in windows at the latest, a week before I joined the 2 men project. 2 months (?) later we delivered a "patch" (the first version didn't even work at all) of the initial solution using the same unix scripts running in interix.

Seems like they were not in any hurry after all. Some 5 years later, they contacted my company again asking how to install it in their test environment. They were apparently not happy that after 5 years, the solution was still a bodge solution using interix rather than a proper port in windows. I don't know what happened from there, I had moved on to another project right after that delivery, could not remember anything and to be honest only painful memories could come back from that shitty codebase. ( the company was making something like 5K a year gross from that application )

I think he means (and that is also my experience in running services companies) that even if you deliver the software to your client, the client does not have the time scheduled internally to actually test and report issues. So you delivered all the client wanted and they will not even look at it because the internal manager who issues the project and his team are too busy.

We delivered a project a few weeks ago and heard nothing; I heard only yesterday that the manager went on holiday and will be back 3rd week of january. And when he comes back his inbox will be full so I do not expect any testing till the 2nd week of feb...

Exactly. All this article is really saying is "most of the time, adding together median expected times underestimates the sum of mean expected times". Well then, easy fix: add together the mean expected times off the bat.

Bingo. When we think about the steps intuitively, we tend to gravitate toward the median. "How long does this usually take?" is asking about a median in surprisingly precise terms.

Instead we have to understand the distribution, and the mean as well, and add up how long it takes on average, rather than just in the most numerous case.

Further, and perhaps more importantly, by understanding each distribution, you have far greater knowledge about what the expected time will be, and where the delays are most probable; and you can begin to start analyzing where those delays might be coming from in the interactions of all the parts of the system. This is straight-up quality talk right out of W. Edwards Deming, and it's how you start to improve how you produce in general. Good stuff.

The problem with high estimates, if accepted by the stakeholders, is the risk of over-engineering. When given lots of time, development teams often will find ways to use that time which don't add to the value of the features being delivered: over-testing, making things "re-usable", a chance to try something new. And the project is late anyway.

Maybe optimistic estimates are a motivating factor for teams. The desire to finish faster, to be more efficient than you were before, a commitment with a challenge.

In my experience, it takes a strong product owner and a mature development team to meet deadlines. Shipping on-time is always a game of tradeoffs. Accuracy in estimates is usually the result of doing things in a known, measurable way. And by keeping the stakeholders close, you can make critical decisions together to keep a project on track.

And account for correlations. It would be interesting to see if multi step projects with overruns tend to overrun in more steps then expected. I could imagine a stakeholder management heavy project to be more likely to overrun in every step. In other words, even by summing the distributions you might be underestimating the expected time of completion.

There's more to it than your easy fix, the article alludes to it but does a poor job of covering it: if you know the distributions of the intermediate steps, but you only sum the means of the steps, then you are throwing away data that could help you calculate a better estimate.

Oh of course! More data (sanely applied) is almost always going to result in more accuracy. I just found it very strange that the core assumption of the article -- that the default setting is to add median times of subtasks -- was never questioned. Why was that the chosen method to begin with? Most time estimating tools (Pivotal Tracker, etc.) work with averages, not medians, for precisely this reason.

I think what he/she's saying is you can't predict how badly something can go (since it can go infinitely badly), you can only predict how well it can go (because it actually touches the finite minimum). Which passes the logical sniff test and is in fact the correct explanation. At least it has been for absolutely every project I've ever been on.

You could feasibly be on a project that could last until the heat death of the universe.

if it's a long tailed distribution knowing the expected value isn't going to tell you anything you can count on

If you're working with censored data, it's possible to know the median but not the mean of your data.


This is very true. The best way I have found to combat it is by splitting up the tasks into many smaller pieces. If you have a task that is scheduled to take a month, then it can easily go out to 10 months in the worst case. If you have a task that is scheduled to take a day, it can easily go out to 10 days in the worst case. This allows you to make corrections to your schedule.

There are 2 other important issues. First, you need to update your schedule based on the amount of time tasks are actually taking. If you only have 10 tasks (each taking a month), the error bars can be quite significant. But if you have 200 tasks (each taking a day), the mean completion time will have quite small error bars. So if you keep a rolling average (say over the last 30 tasks), you can have a fairly good estimate for completion time (agile developers will notice that this is "velocity").

The second very important issue is to be completely anal about your definition of "done" and about making sure that the completed tasks actually meet that definition. Your mean completion time will only be useful if you are measuring the time to completion accurately (obviously).

This effect is so powerful that I recommend "same sizing" tasks and planning everything to have a completion time of somewhere between 1 and 2 days.

There is actually one last thing you need to do. Requirements discovery never happens completely before you start development. As you write the code, you discover new things to do. It is obviously important to modify your plan to accommodate that new information. If you don't, you will end up building something that nobody needs. However, the amount of new work seems to be predictable. I made graphs of new work added to projects over a couple of years and it appears that the growth of new requirements is very similar to some of the defect discovery models (for example Littlewood). Just making a graph of new stories/tasks added over time will give you a decent idea, but I have found that a rule of thumb of adding 30% or so (over the whole project) for new requirements seems to work well.

There is some stuff that can easily take almost no time or can end up taking months. Two items that are commonly like this on software projects are fixing bugs (that can either turn out to be really, really hard or not bugs at all) or upgrading package dependencies (can require no changes whatsoever or can require a significant rearchitecture).

I think scheduling these issues would benefit more from splitting up risk and estimated duration and estimating both separately. Perhaps by drawing a mini probability distribution for each task.

Breaking down the task only helps if, in some way, analysing the constituent pieces actually helps you better estimate the risk or brings unconsidered requirements to light. Sometimes it doesn't. If you do too much of it it can also end up being a contributing factor to the delays (analysing every last detail).

There's another hidden aspect: Be aware of dependencies.

When you have a series of dependencies (A -> B -> C) stacked against each other, once A is late, B and C are almost guaranteed to be late. If B is late too, C suffers even more and has little chance of being on time.

If you can lay out the tasks so that dependencies have slack between them so that lateness can be absorbed without shifting later tasks, you are more likely chance to hit deadlines and potentially complete the project on time. Or in more formal terms, track the Critical Path.

In practical terms, I would see this all the time commuting home in DC. I walked to the train to the bus to home. If I timed it perfectly, my commute was ~25 minutes. But if I couldn't cross the street in time, I'd catch a later train which made for a later bus. Or a slightly later train would be a much later bus. When those variances stacked up, it could take an hour.

You can also try to eliminate dependencies as well as track them.

This is partly why in software it's so critical to loosely couple software. If you're working on a big old ball of mud there are so many dependencies that even the smallest task ends up taking forever.

Fortunately SCRUM provides you with a block of time which you can use to decouple software independently of working on features or bugs, so this necessary work always gets done. Ha.

Projects timelines are like naked shorts in the stock market. You have a capped upside but unlimited downside - things can never be done faster than instantly but they can take literally forever (i.e. just never get finished).

You are being a bit loose with the term 'unlimited' but I guess that's marketing for you. You stand to lose the difference between your strike and zero. Last time I looked that's not 'unlimited'.

I'm not really sure what you mean. If you naked short without covering with options then the price can pretty much go up forever and you cover the difference. I don't understand where zero comes into play.

Sorry, my mistake. I was thinking the option on the back of it. Yes, you are correct, for simple stocks, if the stock price rises forever you theoretically have the possibility of infinite losses. Practically I'd not use infinite, but that has busted many people.

I thought 'strike' was an options thing rather than a short thing?

In addition to that, project's stages ahead-of-schedule deliveries are typically wasted, but behind-of-schedule deliveries are accumulated.

As an example: Arriving to the airport one hour in advance doesn't allow you to actually board a flight an hour earlier. But get to the airport one hour late — and you're flying tomorrow (or whenever the next flight is scheduled).

So it's like shorting a stock... Limited upside, unlimited downside.

Interesting observation, I'd never thought of project planning that way.

Yup. If you look at the factor your estimates are off, they'll follow a lognormal distribution.

And it's natural, if you think about it, that any schedule which doesn't include slack between tasks will almost certainly run long. (Even assuming you've got a mean of 0 -- as many tasks go short as go long -- you're not going to get lucky enough that the tasks that go short do it 'just right' to make up for the tasks that go long.)

So maybe include some slack.

Still, maybe you want to know how often will your schedule run long. So do some measurements of historical data to find out how your task estimates work out in practice.

Plug those in and do a few simulations, and viola, you can get a precise number of how you're likely to do. Plus handy tool to evaluate schedule quality.

I'll take me 2 weeks to use this API to make this application. If I work diligently it'll be done on time. But....oh no, the API can't do everything I need. Now its going to take 6 months.

The article suggests that failures in an n-step process are distributed as a Binomial distribution, which doesn't have the kind of long tail you describe. I hazard that you're right and they're imprecise. It would be great if they measured the shape of the distribution of completion times, scaling for project length, and then suggested a theory behind it.

> they can only go so well, but when they go bad, they can go really bad.

Well, they can, but usually the acceptable explanation for a project going really well is it actually was less work than estimated and the consequence is fewer resources next time. So, there is virtually no motivation left to drive a project into that area.

Ah I like this explanation a lot. It's almost as if project task timing distributions are negatively skewed.

This is a great writeup on a simple bit of math that violates common sense expectations. Consider you need to go on a 100 mile trip and want to average 100 miles per hour. Conditions are bad and for the first 50 miles you only average 50 miles per hour. How fast do you need to go to average 100 mph for the whole trip? Clearly 150 mph, right? Except of course you are too late and would have to go infinitely fast as you need to complete the trip in one hour and have already taken an hour.

From the article: "But while there’s a lower bound to how “under” the median a step can be – a step can’t take negative time – there’s virtually no upper limit to how much over the median time a project can take."

So when one step takes a little longer than expected most people fall into the 150 mph trap and get frustrated when they don't meet the false expectation.

Steve McConnell has this example in sin #5 of his 10 deadly sins of estimation...


Great example!

Time estimations is an industrial way of thinking applied to a post-industrial world.

In the post industrial world time isn't the problem but rather project definition and scoping.

In the industrial world the problem was already solved (machine was built, market often established and output depended on a few factors that could be adjusted. Need more output add more of X)

In the post industrial world every project is about problem solving and scoping.

To put it into comparison.

If we apply post-industrial reality to an industrial world. It means that each time a product needed to be done, if not the factory, then the machines would have to be developed.

It will take many many years before time estimation will die, but it will happen.

That's a good point, but it has two issues you should consider.

First, the industrial world also had significant issues with project definition and understanding. The problems were not already solved, and the process, inputs, outputs and the whole system was constantly changing. Every project was already about problem solving and scoping. So there's a bit of rose-colored glasses toward the past here.

Second, the idea that time estimation is no longer applicable is probably off. There are two separate problems: problem definition, and problem solving, and estimation is extremely useful in the latter. Problem definition is a different problem that still needs much focus, but it doesn't preclude the need for better understanding of time to coordinate other processes and dependencies. You might be saying that those dependencies aren't as important as we think, and I tend to agree, but that's a different argument.

Generally speaking, the idea that knowledge and skills from the industrial era are no longer applicable is untrue. There is a huge body of knowledge about how products are made and built that have 99% applicability to software and technology in the post-industrial world. This is because the problems are the same: management of people, leadership, understanding interactions within complex systems, understanding statistics (the importance of which this article proves profoundly), and improving the spread of knowledge. This is the way that Toyota began operating in the post-WWII era, the way W. Edwards Deming modeled companies, and the way that the current Lean movement guides you to improve almost any business. It's highly relevant.

The main point we should take away, is that time and estimates are not constraints on a system; but rather outputs that are predictable and follow statistical patterns. We can use those outputs to make better decisions, especially if we understand the whole process of production in a systemic way.

Estimation need not die. It's a tool for good in the hands of a systems thinker.

It is my experience that there is absolutely not predictability in learning from the past unless you are doing exactly the same. For something like an agency that are hired to come up with something else thats normally not an option.

The idea that each project we tackle is unique and a special snowflake is an illusion.

In reality, most of the factors that influence the outcomes are exactly the same. Same team, same knowledge, same approaches, same psychological biases, same methods, same politics, and so much more.

Those are the things that influence timelines most; not the project itself.

It's not that it's unique it's that the problems that arise in them are unique.

Time estimation will always be needed, just to decide whether something is worth doing.

Not always true. My project has to be done since it will involve a whole host of hardware installations, marketing and other large expenditures. Canceling or changing my project cannot be done. Thus estimating to decide if it's worth doing is unnecessary. In this case it has to be finished regardless of estimates.

> Canceling or changing my project cannot be done.

Oh, yes it can. It's called your parent company going out of business.

My parent company is one that will never go out of business unless the US tanks completely.

Sure, but as long as there are projects where estimation is needed, estimation won't die.

Simple: time is a resource, it is a cost. When one bids for a project, time is also taken into account when selecting a vendor. So, just as one tries to offer the best price and experience, the same goes for time. One knows that the project will most likely take longer but if that realistic timeframe is put into the proposal, there is risk in losing the contract. It is a safer bet to later ask for reasonable time extensions than not getting the project at all.

Yes, I call it the 'conspiracy of optimism'. The bidder wants to win the business and implement the project, the customer-user wants the capability provided by the project, so they conspire against the customer-payer.

Realistic proposals fail. An under-resourced under-priced and short-scheduled proposal is functionally approved by the customer-user, then signed-off as the cheapest/shortest option by the customer-payer.

The real negotiation is in the T&Cs for change management, scope creep, responsibilities for delays and the structure of payment according to milestones.

The fact that planning estimates represent a probability distribution is well known and there is an established process (PERT)[1] for estimating the expected time of a set of estimates - essentially instead of asking for a single 'most likely' estimate you should as well explicitly walk through some of the risks around this and the ask for an 'optimistic' estimate (which is frequently very similar to the initial estimate) and a 'pessimistic' estimate (which is frequently much larger) - given:

  m = 'most likely' time
  o = 'optimistic' time
  p = 'pessimistic' time 
You can then estimate the expected time 'e' by modelling a triangular distribution based on these and sum the estimates on the critical path.

  e = ∑(oi + 4mi + pi)/6
This is actually a very useful technique but sadly is done very infrequently (probably because it usually comes up with an number that people don't want to hear - but is usually much more realistic).

[1] https://en.wikipedia.org/wiki/Program_evaluation_and_review_...

The program (or project) evaluation and review technique, commonly abbreviated PERT, is a statistical tool, used in project management, which was designed to analyze and represent the tasks involved in completing a given project. First developed by the United States Navy in the 1950s, it is commonly used in conjunction with the critical path method (CPM).

I'm in an organization that uses this method, and I don't think it helps. I believe our "optimistic" metric is supposed to be something like "We have a 10% chance of hitting this goal," our "realistic" metric is supposed to be something like "We have a 50% chance of hitting this goal" and our pessimistic is "We have a 90% chance of hitting it". (Or something like that. I don't remember the exact values off the top of my head.) But what happens is the engineers estimate what they think it will take, claim that's the "realistic" option, then add or subtract some percentage from that for the pessimistic and optimistic values. Nobody really understands how (or if) it's supposed to work, so the estimates are still bad.

As a side project, a friend and I are developing a probabilistic scheduling tool "Probaplan". It not only uses PERT but also incorporates risks for more accurate scheduling. Using the planning, risks and PERT distributions we simulate the project in a monte carlo simulation. The result is a distribution of simulated end dates, these dates can be used to say: The probability that the project is finished before June 2016 is 80%.

If anyone is interested in trying out the beta version, drop me an email at andre /at/ thebroadbaycompany.com

The typical PERT three point distribution is better than assuming the mean, but does have a few drawbacks when modeling joint probable outcomes. I honestly can't remember if it over our under represents the optimistic or pessimistic result.

The other quirk is that it breaks down when you map from effort or work to duration, the actual time something takes when you factor in availability, interruptions, communication costs, etc.

I think that the real value is in forcing people to think about optimistic vs worst case estimates - in my experience the initial estimate is almost always actually the optimistic number and when they start to think about risks people tend to become much more conservative.

Is there any reason to use a triangular distribution these days? I can see it made sense in the 1950s, but now it should be trivial to use a more accurate distribution.

Probably not, but I would guess that the difference is well within the error margin of the original estimates.

There are unknown unknown in projects, people won't pay you for a full analysis and there is always the option that one library you rely on has a nasty bug you cannot foresee.

One doesn't have to go very far and look for psychological issues when a major source of indeterminism comes from directly the imperfect tooling itself we use.

Sometimes software is compared to construction, except you don't get to start with a perfectly detailed environment to build your software on. Until we have this major source of indeterminism in the tools themselves, pretending to fix issues on the estimation process alone is a fool errand.

I believe a as-400 project can be estimated fairly well now based on previous experiences. But how would you go estimating a project relying on, say, local browser storage? The only way is to go and build it and identify all the pitfalls yourself, and then the browser landscape changes and you get set back all over again.

Just following Safari rules on IOS to obtain a full screen mode sets us back a week on almost every releases. We started before they started messing with stuff, now we include a week of fixes in the schedule for every major IOS update, but we had no chance to predict this when the project started.

Building construction is actually a good analogy.

For stuff that's been built a thousand times, you can state with a fair amount of confidence how long a project should take and how much it should cost. You can obtain financing and insurance because the dataset is large enough.

For stuff that's unusual or bespoke -- the kind of thing that will appear in an architecture magazine or in a newspaper investigatory report -- then estimates are very likely to be wildly optimistic.

So it's the same, insofar as the further you stray into research, the less certainty there is. The bigger the bet, the fewer such things have been built, the bigger the risk will be that things go awry.

I have a book in my collection -- Industrial Megaprojects -- which makes fascinating reading for enumerating all the ways that chemical process plants, giant mines, gas pipelines, gigantic factories etc can blast through the budgets and schedules.

My personal favourite: a chemical plant built relying on an adjacent river for cooling. To save costs, only one water temperature sample was taken during planning ... in winter. Three billion dollars later, the owners found that the plant was inoperable for about half the year because the river water was too warm.

I worked building a bank once where a detail cost us a bunch of heartburn estimating and building. Having the architect stop by, look at it, and say "I had no idea if you guys could build that" was infuriating.

This is such.an elegant way of explaining the problem:

For every step in a project, there’s about a 50% chance of completion under or on the median step completion time. And there’s about a 50% chance of not. If a project is composed of 2 steps, the probability that both steps are at or under their median times is 50% * 50%, or 25%. For a 3 step project, it’s 50% * 50% * 50%, or 12.5% and so on. If a project has 6 steps, the chances of some of those steps going over its median is greater than 98%.

But some of those steps will go under the median as well. The problem is the distribution is not normal -- if the range of possible outcomes below the median n is [0,n], the range of possible outcomes above the median is far wider than [n,2n]. Going by the chart the article offered, I suspect a sort of mixture distribution: the better understood tasks have normal distributions, and the poorly understood ones have bullshit estimate distributions.

When I was doing more project work I outright refused to give estimates, explaining that I could come up with a random figure that's so low they won't believe it or so high they won't like it, and in any case the figure would have nothing to do with when the project would be complete.

Instead I offered broad time ranges that would narrow down to more accurate ones as work progressed. The managers who had to report those upwards didn't always like that first but they grew to appreciate how the dynamic worked across the project timeline. In the beginning, they didn't know how long because nobody knew how long. Well, maybe definitely more than a month and definitely no longer than a year, and "it all depends". But every week we knew more and they knew more, and the time margins could be shrunk incrementally as soon as difficult tasks turned out to be not so difficult. So the more progress the better everyone knew how we were doing.

That is one gratifying slide to completion but I do admit it probably doesn't work for every team or company.

"instead I offered broad time ranges that would narrow down to more accurate ones as work progressed"

Well, that is an estimate according to managers who actually understand what they're doing ;-)

This falls into my line of thinking, I tend to be liberal on my project estimates as well.

Unfortunately some project managers / product GMs, and sometimes worse, technical salespeople, fail to understand what all is involved in building the software, web functionality they've come to expect to "just work".

That being said, I've been surprised at how understanding people can be when you explain to them in thought out manner.

>>When I was doing more project work I outright refused to give estimates... Instead I offered broad time ranges that would narrow down to more accurate ones as work progressed.

Broad time ranges can also be risky, because people will hear what they want to hear, which is usually the lower end of the range.

Which is why I only discuss effort (in hours) rather than timeframes. All of our projects are fixed bid, so the conversation usually goes, "this project will take 150 hours of effort. We don't know how long those 150 hours will take to expend." And in rare cases where we absolutely have to mention a timeframe, we make sure it is not in writing anywhere.

There's also feature creep, which is a pretty big problem in a lot of less organised companies and organisations. The original design may well be a six page website with mostly static content, but the final version could just as easily be a web app with a built in help/support/ticket/chat system, member accounts and live updating from a third party API. One two week estimate easily becomes a six months one.

Or it's delayed because of more and more edge cases being found and someone not wanting to be consistent about them. If the first, third, ninth, sixty second and hundredth items in a loop have to all act differently for seemingly random reasons, that adds a lot of time to the development.

It unfortunately doesn't matter how good your estimate is if the company/client tosses out the orignal spec at the first possible opportunity.

But for well managed projects, yeah, it's a good writeup.

Here's the other piece: Estimations happen prior to negotiations. This is why you see so much "buffering." A much better approach is to figure out your budget / schedule first and then work on what scope can fit within that pre-alloted budget / time frame.

Hey, thanks for the twitter:description that showed up when I shared this in slack. Not nice, especially given that nothing in the piece is about the intelligence of bosses. Someone needs to grow up.

<meta name="twitter:description" content="Possibly because your boss is an idiot.">

Another common source is delay and procrastination up front.

First, a dev estimate is given of six months.

Plan is made to ship in six months.

Then, stakeholders bicker for four months on whether, how, and when to actually do it.

Then, dev gets started, being told that they already spent four months, so they should be done in two, according to the "initial estimate."

Was it "debugging the development process" that called this out? Or "code complete" perhaps?

I shared this with my bosses on slack and the subtitle for the link on slack is 'Possibly because your boss is an idiot'. FML!

Well, at least now they know where they stand with you

The article is really cool even though it is a little bit simplistic in my view (and experience). I've also made this comment on the article by the way. The first problem is that rarely steps are independent from each other. I haven't seen hard data on this (and if they have hard data, they should look at it) but my guess is that if one step fell behind schedule, there's an higher probability for the next step to fall behind schedule. That's because the reason a step is behind schedule is not purely random but depends on the internal element of the project (it is harder than usual, using new tech, a person is overloaded and always late...) and all this problem will not magically disappear in the next steps. The second issue is that project are not so simple as a sequence of task. They usually have tasks in parallel,with different dependencies and sync point. Accounting for time variations on each step quickly become a total mess.

Where I work we don't estimate in time, we estimate in points. Points are intended to represent engineering's view of the relatively complexity and uncertainty of a given story.

Pivotal Tracker then looks at story delivery over the past 3 weeks and gives a simple average: velocity. You can then look forward to see approximately when future stories will be completed. You can also see a volatility measurement, which characterises how much velocity is fluctuating.

Our horizon is deliberately short, because we move very quickly.

Nevertheless, it has a simple advantage: it is based on the true and most recent data of the exact project you are estimating.

Other estimation techniques are useful in other situations, but simply being able to say "those are the actual numbers for this project this month" is enormously powerful. There's no fudging. The numbers are right there in black and white.

It usually takes a little while for people new to this approach to accept that velocity is not a target; it's a measurement only. It's a unitless measurement that is only meaningful within a single project, operating at a floating exchange rate with calendar days.

One last thing that helps, as others have pointed out. Break down your estimation tasks into smaller units. Never accept the small headline that hides a big feature. Continuously look for seams to break big stories into small stories. When the pointing begins, summarise aloud with fellow engineers a rough idea of what will need to be done.

Psychologists call this the "unpacking effect", and I suspect that it's responsible for most of the estimation-increasing power discovered in more fully-dressed estimation techniques like PERT, parametric estimation tools or even good old fashioned checklists.

(I was working on an estimation tool for a while, so this subject is dear to my heart).

My son keeps telling me about some study that found that optimists fall further behind than pessimists, but it wasn't what they expected. They expected (as a made up example) that if it required 30 days, pessimists would estimate 40 and perhaps come in on time and optimists would estimate 20 and not make it. Reality was more like it would require 40 days and pessimists would estimate 30 and optimists would estimate 20. They both got it wrong, optimists just got it more wrong.

Another issue: It is really common for people to "take some well deserved time off!" when they finish some piece of the project earlier than anticipated, thus flushing away time that could have helped out on parts that will take longer than expected.

Cuz: Humans.

Evolution. In the distant past, some humans were able to accurately estimate schedules. Since they knew the true difficulty of any task, they didn't bother.

So only the cockeyed optimists reproduced. That's why we can't estimate schedules today.

The time it will take to complete a project is fixed but unknowable. The only thing you can control is the scope and the cost for each unit of work. Therefore the only strategy is to do the least amount of work you can possibly do for the least possible cost. If someone requires you to fix your costs you have to charge them a lot more money. Best to tell them that whatever happens, nothing could have been done to improve it and make sure they're willing to compromise heavily to get it live.

Hofstadter's law + Parkinson's Law + Murphy's Law = Delay

Hofstadter's Law is my favorite law of all time:

"It always takes longer than you expect, even when you take into account Hofstadter's Law."

— Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid

This always bugged me cause it means every project estimate should be infinite in length.

Zeno's Paradox. The solution is that a series can converge.

But then you'd expect it to converge, and that would break the law.

Cost estimation is an NP problem. Perhaps it will become practical when we can use a quantum supercomputer to estimate the time needed to build a single page web app.

No, because you'll just ask it to build it in constant time. Software costs more to measure than to do.

That's why PERT has problems. Estimate times for "best case", "likely case" and "worst case"

Expected time = (best + 4 * likely + worst) / 6

For those reading at home, the meaning is that the "most likely" is multiplied by 4, giving a triangular distribution that (very very) roughly approximates a normal distribution.

I went hunting for the origins of that formula a few years ago. I couldn't find the original source, and certainly none of the sources I found had a justification for it.

Someone who works at RAND could probably pull the original internal work and tell us. But I suspect it was chosen for ease of calculation.

While it's been criticised for being normal-esque instead of pessimistically skewed, it still outperforms single-point estimations. I think it's because of the "unpacking effect" that a full PERT estimate causes you to undergo.

There's a literature of people tinkering with the formula, but I don't think it's anywhere near as important as the unpacking effect is.

No, there is no justification, they just picked that because it has finite limits. They say it's a Beta distribution.

PERT was invented by the US Navy but these days the DoD recommends not using it but just use normal Critical Path Analysis instead where you just estimate a single time - hard enough on it's on without having to do it three times.

Right, 3-points are tedious to do by hand. That's why I began working on a tool. I figured if it's easy enough, people would actually do it.

And it also assumes that the optimal is just as likely as the pessimistic. As anyone who has ever planned anything knows, more things can conspire to introduce delay than work out just right enough to make it take less than you excepted.

Right, and there's lots of literature jumping on that and trying to repair it. Yet the original formula outperforms single point estimates.

But as I said elsewhere in this thread, my suspicion is that the unpacking effect dominates the "improvement" that's observed and that the particular formula is largely secondary.

Oh yes, you did say that, sorry. I only responded to your first point and then remembered the second myself.

PERT is taught as a time based planning tool but it also has a earned value element. The tools for that have moved on too.


I suppose the unpacking effect does make more sense though, over CPA.

"How long will it take?"

"6-8 weeks"

So what do you put in CPA - 6, 7, 8 ? PERT says 7.

It reminds me of Simpson's Rule:


haha that's beautifully recursive

One reason is perhaps to do with what really happens when someone is trying to win a project. Given that winning involves beating others in the fray, all including the winner would end up making commitments which are higher than the capabilities and competencies available with them at that point in time. This initial gap creates risk right at the beginning and if it is not mitigated in time, would amplify to bigger proportions. This mismatch eventually results in projects running behind schedule which in turn becomes a self-feeding vicious cycle with short-cuts being taken to cover up. That then leads to a compounding effect on schedule slippage.

Because prototyping requires you to build and then get feedback, because almost no company knows how to actually plan a project in isolation rather than have to go through multiple iterations. Then rework (where you're redoing things based on user feedback and changing requirements) causes forward progress to not be the sustained motion that management always expects. That's why.

Projects end late whenever they are started late.

If a project is a month past its deadline, that means it should have begun a month sooner (in order to meet the deadline).

It's surprising how little attention is paid to this fact in the professional world. So many teams and managers focus on the end date, and ignore the start date. In truth, the start and end dates are precisely equal in importance.

If you want a tool that goes beyond assuming the mean, take a look at LiquidPlanner. It can model complex plans, and is based on ranged estimates that let you capture uncertainty in ways that most other told can't.

Full disclaimer, I wrote a large chunk of the scheduling engine, so if you're curious about it, just shout.

Heads up though, my son was just born, so it might take me a while to respond ;)

The main problem with the typical estimate is that it is a scalar number. 2 hours or five days or three months. That's fine if you are looking for a target to aim for. But if you want to actually know when something will be done and how much it will cost, a probability distribution with a confidence interval is the only reasonable way to model that.

Because nothing is perfect, not the project, not part of the project. Each time you make any tiny piece a little bit better, you spend a little bit more time, you slide behind.

When you accept this fact, sometimes projects get done on schedule

First time I see a credible take on this subject ! Great writeup.

Cheap, fast, good. You can only have two.

In practice, you can't even have two. That's because the cheap-fast-good rule assumes that you're already close to an optimum, whereas in practice, things are expensive, slow, and bad, for reasons that are unrelated to one another. As a result, a team will lower the requirements, or agree to spend more money, and discover that the project still takes just as long. I've seen this happen over and over.

In the words of DevOps_Borat: Software project 1) On time 2) On budget 3) With quality. You can not able pick any.

- https://twitter.com/devops_borat/status/289782091250532352?l...

What begets all three?

Improve the system, improve the world [cheap, fast, good] lives in. Still have to balance, but you're now balancing cheaper, faster, and gooder.

Requirements != Expectations

Asymmetry. No project can finish in negative time.

Hofstadter's Law rears its head once again.

Because humans suck in predicting the future.

This is actually a good answer. If we knew how long it would take to finish this project, we would also know how long it would take a given horse to finish a race, or when a stock would hit a certain price point on the market. We wouldn't need to be working on the project because we'd be inordinately wealthy.

All this talk of medians and means assumes that you've done the same thing many times before (otherwise you wouldn't have those statistics). In software, if you've done it before, then it's already done so the estimate is zero.

Projects always involve doing new things, new requirements, using new technology, techniques, or tools; targeting a new system, and/or using new processes and people. New requirements, constraints, market situations, priorities, and discoveries rise up during the project. Sometimes all of the above. Past performance is not a predictor of future results.

It's more realistic to flip it around. Prioritize the most-important requirements and set an initial deadline. Then you'll get something (the most important stuff, or at least some progress towards it) by the deadline and can decide whether to add the less important stuff afterward.

When doing that, quality and other intangibles need no longer be overlooked but become part of the prioritization. Do you want high quality, or more stuff by the deadline? Do you want the team functioning well for the long haul or is it worth burning them out to rush this, then having downtime afterward?

To get more control, shorten the deadlines to the minimal point where you're still getting deployable chunks of acceptable quality. That acknowledges the inherent uncertainty and gives better control and visibility than just making up a random number (or 3).

Because people estimate based on ideal time.

Because when you are in a business that follows a process you are in a business that can hide behind that process and no one has to accept responsibility.


Are you saying businesses shouldn't have processes?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact