- “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.”
- “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and Google Translate isn’t good enough. I’d need to insist on professional translation in order to even estimate the second part.”
- “The problem is tracking down a bug rather than building something, so I don’t have a good way of estimating this. However, I can timebox my investigation and if I’ve not found the cause at the end of the timebox, can work on a plan to work around the bug.”
You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. They probably don’t want you to lie, but there is a small chance that they pruessure you to. If you don’t resist this pruessure, you can end up continually giving estimates that are 10x off-target, blowing past them as you lose credibility, and your running your brain ragged with sleep-deprivation against a problem you haven’t given it the time to break down and understand.
But when you advocate clearly for your needs as a professional, people are generally reasonable.
In my experience, most of the time there was pressure. I remember a funny conversation between my project manager and a colleague:
PM: "We need to have X as soon as possible, how long will it take you?"
Dev: "Oh that's, easy, I can finish it tomorrow."
PM: "Tomorrow??? That's impossible, because it needs a and b, no?"
Dev: "Yes you are right... I can probably finish it by next week."
PM: "Next week??? That is a long time! It's only a, b, c
and d that needs to be done. Does it all take that long?"
Dev: "You are right, I can probably finish it sooner. I can get it ready by this Friday."
PM: "That is perfect! The customer wanted to put in production on Monday, so we can now confirm. Thanks!"
I overheard that story (open office), and it was a real facepalm moment.
In your example conversation, the developer starts off by demonstrating that they are not good at thinking through how long things take them, by giving an impossibly optimistic estimate. Even the PM, whose job is not to figure out how to build stuff or how long building stuff takes, knows the estimate is wrong, and pushes back on it, pointing out dependencies the Dev appears to not have considered.
The dev comes back with another estimate, also provided on the spot, without actually thinking through all the tasks, problems, and potential ways to streamline / accelerate. The PM, having just had the dev's lack of thorough thinking with regards to complexity estimation amply demonstrated, asks the Dev to think more carefully, a third time.
Are we facepalming at the dev who can't be bothered to actually consider what it will take to get the thing done, or the PM who for some reason accepts the 3rd estimate, even though the dev appears to just be randomly generating numbers? Both?
1. The PM just wanted it done by Friday, because the customer wanted it by Monday. This was clear at the end of the conversation, not at the beginning. So it just could have been a "Can you get this done by Friday? Because the customer wants to use it on Monday"
2. The developer clearly had no backbone, and was just throwing numbers to see what the project manager would like. 1 day? Too short! 5 days? Too long! 3? Perfect! Estimates might be a guessing game, but not a guessing game on how long the PM thinks it should take.
So yeah, facepalm to both of them, and the whole situation. But I must say that they were definitely not incompetent people. The PM was one of the best I worked with, and the dev was a smart passionate coder. It was just that situation that was a bit absurd.
Proper way on how to do this:
PM: Can you give me a deadline (not estimate!) on when you can finish X? The customer needs this ASAP, so it has priority over your other tasks.
DEV: Let me look into it and I'll let you know in half an hour. (Works out estimate, possibly asking input, adds a buffer because deadline != estimate, looks into his own schedule of meetings etc).
DEV: I can get it ready by Tuesday.
PM: I'll notify the customer.
> PM: "That is perfect! The customer wanted to put in production on Monday, so we can now confirm. Thanks!"
> pointing out dependencies the Dev appears to not have considered.
I'm not sure if they're dependencies or other priorities in this case. I've had managers before ask for an estimate and and I've given them one (2 days), then they've assumed that I'm dropping everything I'm in the middle of and told clients it would be done in 2 days.
The problem with estimates is that the people asking for them really want promises.
or who doesn't know that "I'll need some time to get back to you on that" is an acceptable part of their vocabulary.
That's just a question. Asking questions is OK. If you have such a high need to please other people and such a low self-confidence that you will change your answer simply if someone asks you twice, that's the problem with you, not them. I say it as someone who have been like that and have changed it through therapy.
I was a nervous mess and always thought that people around me don't like me, pressure me and in general was very nervous around any interaction where I and other person have some disagreement. But turns out, firmly saying "No" is quite OK, people react well to it, respect you for it and nobody is going to fire you for giving good estimates.
But are they ever going to admit that they have no idea and accept your estimate on face without playing BS mind games? Probably not. But for some reason that doesn't stop them from trying.
I've never seen someone try and do this with a home contractor.
It might also be that they’re surprised at the complexity and want to know more. For this reason, the important part of an estimate isn’t the number. Its the task-breakdown and the identification of risk.
In most cases, people who ask "Does it really take that long?" are giving you their first emotional reaction and haven't actually thought it through deep enough to decide to manipulate you. They're upset and they want to make sure that they understood you right - which is a thing people often do when they get bad news, and I cannot blame them: if you get upset about something, at least make sure you really should be upset.
So, you should react either in a spock-like "stupid rational", giving no thought to what they might be implying whatsoever with a simple "yes", or, even better, in an emotionally-aware rational way: acknowledge their emotions and offer options. There are always options and trade-offs, they just need to be clearly communicated. Not "OK, I'll do it in three days instead of five", but "I'll can do it in three days with prototype quality so we can test it out, but it will be buggy as hell, not fit for production, and you should document it". I've never had a manager who had a problem with that, and I despite how popular horror stories are, I really doubt that there's a lot of them out there.
Your manager cares about the product, he's emotionally invested in it and wants everything to go as fast as possible - it's not a bad thing! It's only bad if you view his disappointment as your fault - but it's not. It's just how things are. As a developer, you're often a messenger bringing bad news. There's no way around it, and you have to learn to deliver bad news for people. And separate these bad news from you and your professional self-confidence.
If he was emotionally invested in the product, wanting everything to go as fast as possible is not necessarily a good thing. I can make things go as fast as possible, but then the questions become no unit tests? no integration tests? spaghetti code?
And this isn't a response from a few go around either. This is often the first thing that comes out of their mouth, right at the beginning of the project.
What happens is as follows:
1) Your manager has software development background and uses it as a superiority contest to show how much better they are than you. His lack of awareness regarding variable skill level or lack of knowledge of an existing system seems to factor little into the question. So one could argue the manager could be more empathetic in this case.
2) Your manager knows nothing about software development and really has no business asking such questions in a condescending manner. So they either pretend to know or they promised someone higher up the chain even though they had no idea, but were scarred to say so, thus the cycle has completed.
3) Your manager has a calm, rational head and accepts your estimate, your explanations, and has a good understanding of the product to negotiate with you in terms of requirements without getting emotional. The manager trusts understands that your estimate is, more than likely, not going to be accurate but accepts the changing winds and doesn't hold you to the fire.
I totally agree its better to be explicit. Which is why it matters to avoid overly emotional reactions and accept what people tell you so hand waving can be avoided.
But this is more of a prevailing problem in software requirements in general. It's rare that developers get projects with clear ideas on what needs to be done because of the ignorance or unknowns in questions that need to be asked in order to deliver a product. Often the developer has to decide, and when they do and its wrong, then they have to fix whatever they decided because no one explained, or bothered to explain, how something was supposed to work in the first place.
I'm surprised that you haven't. I've certainly seen the same kind of thing happen, though admittedly it is usually based on cost rather than time. Ultimately, though, the discussion was the client disbelieving a high effort estimate.
Questions have built in assumptions and implications. Most questions don't have enough to be of concern, but some questions definitely imply things beyond the question itself. When asked 'Does it all take that long?', many hear an implied 'Make it not take so long.' command. It is not implied all the time, but it is implied often enough. This does not mean someone has low self-confidence or needs therapy.
If you go from "my boss gave me a command" to "I must execute the command and not question it" without exercising independent judgement, then I would assert that you do have low self-confidence. Whether someone needs therapy is a deeper question that we should be hesitant to answer for another person. But therapy was tremendously helpful for me in realising how my personal and professional life were held back by my reluctance to trust my own judgement or endure the discomfort that can come with honest, respectful, collaborative confrontation.
I should make explicit: Part of trusting ones own judgement is also being willing to say, “I don’t have enough knowledge/experience to make a trustworthy judgement here.”
Here you can see the exact moment when an estimate became a commitment.
At the time I made my comment the Parent comment had a negative score and I was trying to be helpful to commandlinefan .
> under the (mistaken) belief that more hours = more software.
I'd delete it if I could.
This has not been my experience. People want ‘estimates’ at all costs, tell you to not worry about any accuracy, and then a week later tell your manager you committed to x date.
Spoken as someone who has never taken the time to fully master a programming language and, from the sound of it, has never worked with someone who has either. The difference between someone who has spent 6-12 months with a language compared someone who has spent 6-12 years is night and day. From the general tone of the article, he obviously focuses more on the business value than on the technical side and that is a pretty good approach for making money. But I'll take Peter Norvig's advice (http://norvig.com/21-days.html) over this guys when it comes to mastering a language.
To be fair most of the content of those articles is pretty decent though and it is just a pet peeve of mine when people claim that mastery of your medium doesn't matter.
If you are working with an apprentice, whatever you do, it will seem like you are really experienced.
The senior members of his team had a culture of eighty hour work weeks. Draw what conclusions you may.
The biggest thing that has helped me here is to keep a running list of my tasks, an informal Gantt chart, and show it to the manager whenever they ask me about an estimate. This helps me communicate in a way they understand.
I like to handle this problem by providing an estimate RANGE.
In the end, his company went bust because he was selling stuff to customers that could never be delivered on time, or even never at all.
Note well: Automated testing is almost certainly essential. If someone says you are “building an MVP”, then make sure it is an actual MVP. Make sure you have a clear market hypothesis to test and check that it couldn’t be built out of duct tape, Google Forms, Zapier, ITTT, manual processes, and excel-wrangling.
I suppose it's both slower and more expensive than "just ship bad code at full price and wait", but I'd still expect more companies to choose it as the intermediate option between that and comprehensive in-house testing.
I used to work for an online travel site - before every project kicked off, the business analysts would produce requirements documents that always started with a "flexibility matrix". The flexibility matrix had three rows labeled "schedule", "scope", and "resources" and three columns: "flexible", "moderately flexible" and "inflexible" and they were supposed to check one option for each row. So every single requirements document listed "resources" and "flexible", "scope" as "moderately flexible" and schedule, OF COURSE, "inflexible".
I take it these requirements documents were not honored by your non-developers? A document labeling scope as "moderately inflexible" does little if it's treated as written in stone.
The first question is of course whether that's true, since resources don't tend to flex up unless someone outside the project dictates it. But even if it's accurate, it tends to produce the situation The Mythical Man-Month was written to condemn. If the ship date can't move and the scope won't bend on core features, there's a very real risk of finding that extra resources can't fix the timeline problems.
You're right that most organizations have to operate this way, and it's not inherently bad; I've just found that "flexible resources" is often used as a way to avoid admitting that the real third category is "risk of failure".
However, Giving a developer more time will often improve the quality of the output.
Some deals are inherently negative sum - if you bought a lemon.
I worked with a group that designed telcom IC's. One of our customers developed a new product. Which then failed in the field. The parent company had to issue a recall. Destroyed all remaining inventory and canceled the product. All because someone jacked the microphone supplier down about 5 cents on a part.
That highlights a beef I have with economics. Failure to understand that business relationships are exactly that. You can't model them using the sorts of time invariant non-network models they use.
First, Not everything is a commodity with perfect substitutes, arguably most things aren’t. Substitutability goes beyond price.
Secondly, benefits are not necessarily equal or a linear function of cost.
Third, whether a game is zero, positive, or negative sum requires a look at the costs/benefits of the whole transaction. Saying that a buyer gains $1 more if the price goes up is obvious, it that doesn’t necessarily make it zero sum when you look at the overall transaction of a $1 price raise. .
Zero sum implies that price, costs and benefits are equal. I spend $20 on a product that cost you $20 to make, and I get $20 of benefits out of it.
Whereas I buy something for $20 , it cost you $5 to make, and I got benefits of $100 out of it, that’s positive sum, as both the buyer and seller got positive gain. If you raise the price, the deal isn’t as great, but it’s not zero sum.
The book contains a quiz that we used as part of a training exercise with management and the results were hilarious. Here is an online copy: https://scrumandkanban.co.uk/how-accurate-are-your-estimates... please don't look at the answers, I guarantee you will have fun completing it.
For clients, my preferred approach is to show "The Cone of Uncertainty" and ask where they think they are. Since most people have no idea what they want to built I ask if its ok if my estimates are 4x off. That usually gets me a few weeks of peace while a team comes up with a product definition and we start all over again :)
> Total length of the coastline of the Pacific Ocean
Coastlines are fractal. If your measuring stick is 1 km long, you'll get one number. If it's 1 m long, you'll get a much larger result. And if it's 1 cm long, you'll get an even larger result.
The actual length of a coastline is infinite when measured with an infinitesimally small measuring stick.
Software development effort also increases, unbounded, as discovery proceeds and the precision of requirements increases.
>In actuality, the concept of an infinite fractal is not applicable to a coastline; as progressively more accurate measurement devices are used
The notion of the length of the coastline, and why I think the question is apt as well, is that regardless of the fractal paradox, the practical answer is that there is an answer that is accurate enough to satisfy the business's or the customer's needs.
The analog for software projects, from my point of view, is scope, and it's generally one of the biggest problems with projects that take too long. Product and various stakeholders take outline broad, high-level scope requirements that may or may not actually be informed by the realities of building or delivering that piece of software. They generally kick the can down the road and take a "I'll know it when I see it" type approach that winds up drastically changing things at some arbitrary point later on on the project.
A rough measuring stick for scope always means that the answer will be longer, however, small measuring sticks for scope are absolutely impractical. What's needed is the understanding from the business that refining scope increases development time, and the high-level broad scope that is sufficient at the beginning of the project is not sufficient at the end. Many businesses want to run with the assumption that their broad scope and the estimates that go with them are accurate enough to bring a product to market.
I’ve gained some experience and confidence at this in the years since, but at the expense of some psychologically painful experiences and a firing. Our industry really calls out for a more basic book aimed at University CS grads (or bootcamp grads or MOOC)-grads on how to estimate tasks.
I think out of 15 people nobody got more than 1 or 2 right, except for my manager, who somehow got about 8 right. I'm curious how the HN crowd does on this - care to post your scores?
Going to read it. thanks.
This. I have PMs who will ask me over and over until I give a number and I've learned to stand my ground. Because if I don't, I end up being responsible for the estimate I've given (as I should).
Now I make it clear that I will not give a number until I know more. Just a few weeks ago I was asked to give an estimate of how long it would take to do an integration with something I've never used before. I said, "I need 1-3 days to learn the product enough to give an estimate". What I get back is "can't you just give WAG". But this time around I said "You have two choices, 1. you give me time to learn what I need and then I'll give an estimate or, 2. Find someone who already knows the product to give you an estimate"
You're not responsible for an estimate you give, the person who's asking for it is.
This is the key difference between an estimate and a commitment: If you give someone an estimate, they are responsible for steering the project using that information. If you give someone a commitment, you are responsible for meeting that commitment.
This is why very few people actually want estimates. They want commitments, but they will ask for estimates to get an answer and then interpret them as commitments. Most people asked for estimates know this intuitively, thus "doubling and adding 10" and other communication strategies to turn estimates into safe commitments before they are communicated.
One big insight for me, which you obviously put to use in your story, was learning to give a different number.
"I don't know" is understandably frustrating to people who are trying to plan work and answer requests. They often respond to that by pushing for a delivery date even when it won't be accurate, but a lot of the time what they fundamentally want is a numerical answer. "I'll have it scoped out by Tuesday", "I need at most 3 days to learn enough to estimate", and so on all aid that planning without committing to a project timeline. That rarely ends the conversation, just as it didn't in your example, but it does a lot to show good faith and justify not answering the initial question.
Where does the "Scientific" come in? Because I really thought it implied having a good idea, which made SWAG seem like an oxymoron.
You use things like Average/95/99, t shirt sizing, how much work can be done in parallel, 3rd party/client dependencies, ramp up time for a new library ... it's still a wild assed guess but at least there's some basis in reality and you can defend your estimate - show me what I'm wrong about or missing and I'll revise my formula.
Negotiating over the assumptions and inputs is 90% of the battle anyway. Without a model you're standing on sand.
Another nice trick if you're under time pressure is to arrange the project so that you can deliver it in stages that partially meet the requirements, or even deliver something non-functional that people can play with while they wait.
This can keep the sales team rolling or customers happy for a while, buying you some time to write the program properly.
As a bonus, you might get earlier feedback if you're going in the wrong direction, in which case less time wasted.
(Obviously this approach isn't always possible. Many problems won't decompose nicely.)
Then you can cut scope by postponing (or not doing) some of the phases. Ideally.
I had a friend who had an unusually high frequency of conflicts with various management, project managers (I expect there are always some, but this guy had a lot of bad experiences).
He showed me an instant message conversation:
PM: "Is X done?"
Dude: "Yes, but a, b, c is not done"
A, b, and c were parts of X's requirements.
Now the project manager or such should have been able to read between the lines, but man don't lead with "Yes" and actually describe a situation that is "No".
Recently I've flat out said no, as gently as I could, to doing some work - porting a software stack to run on Windows.
A customer is asking for an estimate to do it, because a project manager who wouldn't be developing or supporting (or even using) the system likes Windows. The sales manager isn't thrilled with me over it.
So he's asked the team as a whole to estimate it because I wouldn't. The rest of the guys are frontend devs. I don't have a clue, they're in a worse position but for some reason they're all up for putting a number to it so I look like a bit of a !
I'm a contractor, so I can and would just turn down the work, but it's difficult politically.
If somebody else comes out with a much lower estimate, ask them what they think about risk A, B, C... if you've got a point, this will usually work.
I have literally no idea how long it would take though, I couldn't in good faith put a number to it.
I also don't want the work - Windows isn't an environment I have experience in because it's an environment I don't want experience in. They wouldn't allow a rate change to do the work.
Of course, once you're holding certain things as non-negotiable... you're saying no.
Dev: "It's a really bad idea, will take an indeterminate amount of time, and be a maintenance nightmare"
PM: "Can I just get an estimate?"
The thing to understand about politics is you can do everything right and still lose.
Here, the damage was already done by their unrealistic expectations. You were either going to accept the job, fail and make them angry, or reject it and make them angry.
Sometimes your least worst option is to cut your losses.
The only thing that probably is going to happen is "Huh, why isn't that delivered?" and then they will look everywhere but on the fact that it was, at least partially, just a stupid idea in the first place.
Of course I don't know the exact situation so it's more of a thought on what happens in general in the industry.
Nobody will revisit the GP's reputation, and after the novice devs fail to do the work people will just scatter into different projects. They may or may not take a reputation hit, while the PM that insists so much on doing the not viable work will take none of it.
It isn't just software that goes double, look at most custom building projects or government contracts. Most everything takes twice as long and costs twice as much.
I read something once that has stuck with me. "We can never give more than a vague guess because we are literally building something that doesn't exist and has never been built before."
* I do find it annoying that the sales teams generally drive this. I wonder how they would feel about a response of "We'll get you an estimate of when it will be available as soon as you get us an estimate of when this feature will generate the $1M of revenue to cover the costs of building it. Also, please give us a list of customers that are likely to buy it and your expected contract date so we can track it along with development."
There's definitely a correspondence here, but it's a bit indirect.
Government and even private contracts are often knowingly lowballed to secure the deal. Rules like "must choose lowest bid" reward entering an unrealistic bid and then inflating costs after work is going. Or, if overruns come with sufficient penalties, doing work at a loss and then charging a fortune to support a fragile and proprietary result.
In the metaphor to non-contracted software, developers generally don't correspond to the contractors but to the government. Sometimes blown estimates are simply errors, but other times somebody wanted to secure customers, so they pitched an unrealistic scope and arranged to make the inaccuracy land on the programmers. Where the government ends up with late delivery or runaway costs, programmers end up working crunch time or sacrificing quality. And where the government ends up locked into paying for support, programmers end up locked into maintaining rushed, tech-debt-riddled work.
(As for pushing sales for revenue and contract estimates, do it! Some of the most productive teams I've been part of have been ones where sales applied this pressure, but also responded well to the reverse version and kept everyone informed.)
That was an entertaining job.
Another interesting characteristic he had was to get upset if he thought projects were running too smoothly...
All involved engineers actually had the backbone to say it was impossible, but no chance to say it. Obviously, projects did not go well at that place.
The longer estimations are very much due to changing priorities and a developing environment and understanding of what is being built though, hence the inaccuracy.
If you validate the invalid assertions of these abusers (e.g. that some unclear task can be estimated with certainty) then you're enabling the abuse.
Do you have a good definition? One whereby a scrupulous person can confidently judge “am I acting abusively?” correctly?
- the scope of the task
- the resource/people you have
- the confidence in the resource/people
- and the dependency on both the resource and the task
Estimation goes wrong in many ways. In my experience, I categorize as follow:
- estimated by non-technical PM, unable to gauge technical/business complexity
- estimated too early in development life cycle. Often time someone come up with idea X then ask right away "shouldn't be difficult, how long will it take"
- no view on team member's productivity due to lack of measurement or measuring the wrong thing
- not considering dependency in the development cycle. e.g. The question "whether our only backend engineering available for the task" is often omitted
- not considering testing, documentation, 3rd party integration/procurement, maintenance and deployment cost
The common practise of coming up a number, then double, or maybe triple it up to make the estimation does not address why estimation is off. Mis-estimation is a symptom. The cause and the cure lies in the people
The problem occurs when the client doesn't want to hear it.
"Sure I can do this in 3 weeks."
Then shows up once every two weeks and finishes around the four month mark.
"Yes we can do that in three weeks. We need the API documentation for _those_ two systems, network access to _those_ resources, and credentials to log in to _those_ services first."
2 months later we're still waiting on critical pieces of those prerequisites, while someone's screaming that the deadline they'd committed to with the customer was 5 weeks ago.
The most important part of this process is getting it in writing so that you can slap them with it later.
the problem comes when you didn't know you need Y, Z or W until the work actually starts.
1) You improve your foresight so you can get your manager rolling on getting resources early.
2) You have a concise record of what you think you need. If you notice something else mid-project (which does happen even in otherwise-straightforward implementation projects), you have a concrete point at which to raise a flag to stakeholders.
If you have zero confidence in an estimate from the beginning, then there is no point where its natural to realize “oh hey, this is going to overshoot and I need to communicate about it.”
But was not estimating the actual total time until completion which must include all the lead time, waits and dependencies.
You should do all this but there will still be times when people are unreasonable, it's just human nature.
If that doesn’t work...I’m actually not sure. But I just started reading the book Crucial Conversations and it claims to have a relevant answer.
People talk about project management, gantt diagrams and all that but you have no idea how to estimate even grossly the complexity of a thing.
Lots of people say “just do it and multiply by 3” without appreciating that The important part of estimating tasks is coming up with the task breakdown and identifying the risks.
For one, it skips the step where you break down the tasks. For another, it seems likely to ignore a bunch of variables in the search for something similar enough.
EDIT: The part about estimates is here: https://www.youtube.com/watch?v=p0O1VVqRSK0#t=36m56s
Unfortunately, the problem then becomes defending whatever timeline you have in mind for non-linear work. also doing too much of this can give bully managers an opportunity to nitpick and micromanage the whole thing.
This is my default. Even when I am fairly confident I know what the code does, I want to double check. And it isn't just so I don't short change myself on time. If I tell them X days and deliver on 0.X days several times, they'll start thinking I overestimate everything and cut me off at the knees. It pays to get as accurate as possible.
Everything gets a T-shirt size. Roughly, "small" is no more than a couple person-days, "medium" is no more than a couple person-weeks, "large" is no more than a couple person-months.
Anything beyond that, assume the schedule could be unbounded. Figure out how to carve those into a series of no-larger-than-large projects that have independent value. If they form a series of iterations, don't make any assumptions about whether you'll ever even get around to anything but the first one or two. That just compromises your ability to treat them as independent projects, and that creates risk that you find yourself having to worry about sunk costs and writing down effort already expended when it eventually (and inevitably) turns out that you need to be shifting your attention in order to address some unforeseen business development.
At the start of every quarter, the team would commit to what it would get done during that quarter. There were some guidelines on how many small, medium or large projects they can take on, but the overriding principle was that you should under-promise and over-deliver. Lots of slack (1/3 - 1/2) was left in everyone's schedule, in order to ensure ample time for all the small urgent things that inevitably pop up.
There was also a log of technical debt items. If the team finished all their commitments before the end of the quarter, their reward was time to knock things off that list. Best reward ever, IMO.
That's pretty much exactly what I've ended up using on past projects - a little more fine grained (start at a half day, went up to months), but that was my approach as well, and if I didn't know everything it went up a size.
If you're doing quarterly planning, the difference between half a day and a couple a days is meaningless, and there's not really any point in distinguishing among them.
If you're doing 1-week sprints, the difference between half a day and a couple days is enormous, and the product planner might get some value out of distinguishing among them.
If you're following a more kanban-y approach, the difference is perhaps meaningful, but not particularly actionable, so I think I (personally) still wouldn't bother to capture the distinction for planning purposes.
I don't know how Spolsky is (was) as a boss, but it's is way easier to just adjust your time reports according to the estimates and make your boss happy than to actually try for no benefits to either you or the project. As long as the estimation errors are white noise-ish and in linear relation to actual task length, which is the assumption for his model to work, manipulated time reporting and not manipulated make no difference.
I've noticed that when programmers are introduced to Agile they initially are bad at time estimates until they learn that you actually can report time according to the estimates and no-one will ever notice. Extra points for looking on the burndown chart before reporting time to make it smooth.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better. Other developers (like Jane) have very precise ship dates that are just too late: they need to have some of their work taken off their plate."
Milton is probably doing fair estimates. When planning four months into the future you probably can say where you are within 2 months with 50% probability (or what ever boxplot he uses) of where you think you will be at that time. Janes reporting is obviously manipulated to match the estimates.
Here is the link to the time estimation described above with Beta distribution.
I used it in the past and I think it is a good framework to discover uncertainty and make it more visible, because it makes you talk about optimistic and pessimistic cases.
The estimates are also good enough to come up with a draft schedule.
The three things that still bother me about that idea are:
1. I haven't tried fitting it to the dataset in OP;
2. It's bounded to the right, which seems unrealistic;
3. I haven't come up with intuitive interpretations for the alpha and beta parameters in this context. If the beta distribution means something, then its parameters must have natural interpretations as well.
> A clue to the underlying problem with the planning algorithm was uncovered by Newby-Clark et al., who found that
Asking subjects for their predictions based on realistic “best guess” scenarios; and
Asking subjects for their hoped-for “best case” scenarios . . .
. . . produced indistinguishable results.
> So there is a fairly reliable way to fix the planning fallacy, if you’re doing something broadly similar to a reference class of previous projects. Just ask how long similar projects have taken in the past, without considering any of the special properties of this project. Better yet, ask an experienced outsider how long similar projects have taken.
The relevant excerpt (mostly an anecdote that serves as an introduction to a whole chapter about it) can be found here: https://www.mckinsey.com/business-functions/strategy-and-cor...
Also, there is a law named after author called the Brooks's law : "adding human resources to a late software project makes it later"
In most industries, if you are running behind schedule, you throw more workers at the problem to catch up. For example, laying railroad tracks, digging ditches, deliverying packages, harvesting crops, etc. By adding more workers, you shorten the time it takes to complete the project. But which software engineering, the reverse tends to happen. If you are falling behind, just throwing more developers at the problem worsens the problem. Most likely because you need the new developers to get "caught up" with the code/project/tools but if you rush that process, then they won't have a full understanding of the project/code/tools and introduce bugs/problems themselves which exacerbates the problem.
It's a fun read if you have the time.
Sadly, managers want to believe that you put more effort into estimating deadlines so I'll just whip up a gannt chart retroactively based on the 2/20 rule and they're happy.
Isn't this Parkinson's law in action, i.e. work expands to fill the time available for its completion? That would mean you overestimate up to 2.4x each time.
Oh, so if I estimate every task as 1 minute I'll finish really quickly?
The 2.4x is to overcome one's innate overoptimism (even if you see yourself as a pessimist), and to add contingency for unplanned complexity, bugs and general overhead.
It's a rule of thumb, not an immutable law, so if one has completed a similar task many times then less padding is needed.
The doubling-it is for when you build the whole component on top of a library, then discover that the library has a fatal bug you can't work around, and you have to rebuild the whole component on top of a different library, then discover that other library has another fatal bug, so now you need to include both libraries with logic around when to use which one, and then that seems to work but you add plenty of tests and documentation to make sure it works that way in prod too and not just on your dev machine.
I don't see what that has to do with learning from previous mistakes. Pretty much all but the most basic programming turns out to be like that -- dealing with unforeseeable and undocumented problems.
(The 20% is because you weren't planning for sick days, an unforeseen emergency bugfix on another project, nobody remembered about the afternoon retreat next week, etc.)
Problem is the fact that you can only justify with a straight-face 45% (100%/220%) of the estimate you give to the client... Then you need to get clever at "bluffing" your way at "inventing" another extra +35% of work that you can't imagine/explain, but you know for sure that will come out of the blue - you know there's this extra 35% work needed for sure, you just have no idea what this work will be. Then finally you can say with a straight face that 20% of estimated time is for unforseen circumstances. There's two ways to see the same math:
(1) 45% + 35% + 20% = 100%
(2) 100% + 100% + 20% = 220%
If we could just be honest with people and show them formula (2) instead of a make-up-ladden version of (1). But people don't, so we get to play the damn elegant-bullshitting game over and over again...
This is how I always have done it across multiple jobs and I always called that extra contingency. Depending on the project I sometimes had to ask for over 50% of the total as contingency. If anyone queried it I would explain that it is very likely we'll need it based on my experience of many other projects.
Then the client would either accept it as is, or they would want to split the proposal into "what we will definitely pay for" and "what we will pay for only when it is actually needed" which is also fine for both sides if proper progress reporting has been agreed in advance.
Someone could think that no client would agree to such a contract as it would open them to be taken advantage of by an unscrupulous supplier, but at the end of the day what matters in business is trust. If a client doesn't trust the supplier they shouldn't hire them. And in all my previous projects I never met an engineer or developer that would deliberately use up the contingency time when it is not needed as everyone knows the availability of this extra time might come very important towards the end of the project.
e is also just an arbitrary number, but it looks like someone calculated it using sophisticated math.
So if you estimate to Level 1, and then break down to Level 2 sub-tasks, then Level 3, your estimate will be out by the same amount each time. and your total error will be a multiplicative series.
If you have an infinite number of levels - which is reasonable for most real-world projects, although this may not be obvious in the planning stages - e will get you nicely in the ballpark.
Humans are optimistic by nature. Even somebody as pessimistic as me.
I notice some devs like to underestimate to make them look more productive. Most of those tend to spend a lot of time fixing bugs that QA pushes back to them. They sometimes spend ever more time arguing that it's not their bug.
I used to do 2.5, I read somewhere 2.5 was fairly common
However, as the tables in this article show so clearly, one task can dominate others. This serves as a great illustration of a perception problem the above strategy can create. Outside observers watch your progress and will probably evaluate it in terms of number of tasks completed.
You have (say) 10 tasks to complete, and from their point of view, all they know is a really long time has passed and you haven't even completed 1 of them! Since they are further removed from the situation than you are, they are almost guaranteed to not appreciate why the variable task is that way. They're likely to just think your team is performing badly.
So, maybe it's better to order tasks so that risky stuff is spread more evenly across the timeline of the project. It could create less confusion. Putting risk at the beginning is a strategy that requires a whole lot of trust and buy-in.
Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally distributed. You can measure the variance to get error bars.
Of course 900 stories to get good estimates ;-) However, imagine that your stories averaged 2 days each. Imagine as well that you have a team of 10 people. That means that you will finish a "sprint" of 30 stories in 6 days (on average). 30 sprints is 180 days -- the better part of a year, but you probably don't need a 95% confidence interval.
You will find that after a few sprints, you'll be able to predict the sprint length pretty well (or if you set your sprints to be a certain size, then you will predict the number of stories that will fit in it, with error bars).
The other cool thing is that by doing this, you will be able to see when stories are outliers. This is a highly undervalued ability IMHO. Once a story passes the mean plus the variance, you know you've got a problem. Probably time to replan. If you have a group of stories that are exceeding that time, then you may have a systemic estimation problem (often occurs when personnel change or some kind of pressure is being applied to the team). This kind of early warning system allows you to start trying to find potential problems.
This is really the secret behind "velocity" or "load factor" in XP. Now, does it work on a normal team? In my experience, it doesn't because groups of people are crap at calmly using statistics to help them. I've had teams where they were awesome at doing it, but that was the minority, unfortunately.
You are saying: take enough of them and it is normal.
I think those really long tails are more of a problem when you are working with "features" that are much longer. If you have 1 day stories and you've been working on the story for a whole week, you know you have a massive problem. It's time to back up and see if there is a way to break it up, or to do it differently.
If you have a feature that is a month, by the time you get to 5 months, you have so much capital invested in the original plan that it's very hard (politically) to say, "Nope... this isn't working out. Let's try something else". Of course, it is very hard to get your organisation to plan to a 1 day level of granularity.
But even if they did listen closely to what you said, "estimate" is not even a great word. If your car is in a wreck, a body shop gives you an estimate to fix it, and they may treat that estimate as a commitment. It's pretty common practice that people are held to an estimate, or to not going over by a certain small percentage. Maybe we need a word like "forecast" or "prediction".
Scrum (The org behind that flavor of agile) actually changed the language used in their guide back in 2011 from developers making a "spring commitment" to making a "spring forecast" for exactly these reasons.
Many places treat budgets the same way.
From the technical side of things, uncertainty can mean a few things here:
(A) I've never done this kind of task (or I don't remember or didn't write down how long this task took in the past)
(B) I don't know how to leverage my historic experience (e.g., implementing an XYZWidget in React and implementing the same widget in Vue or Elm for some reason take different amounts of time)
Considering (A)... Rarely does a seasoned developer in the typical business situation encounter technical tasks that are fundamentally different than what has been encountered before. Even your bleeding-edge business idea using modern JS' + GraphQL' is still going to be built from the same fundamental pieces as your 1999 CRUD app using SOAP and the estimates are going to be the same.
If you disagree with this you are in the (B) camp or you haven't done the work to track your estimates over time and see how ridiculously accurate estimates can be for an experienced practitioner. Even "soft tasks" like "design the widget" are estimable/repeatable.
This whole you-can't-estimate-software accuracy position is entirely a position of inexperience. And of course all bets are off there. You are talking about estimating learning in this case, not doing. And the bets are especially off if you aren't modeling that these are two different activities: learning and doing.
There are other sources of uncertainty. NASA is probably the only organization that fully specifies product requirements before building stuff - often, even the customer doesn't know. Some problem domains preclude you from knowing in advance! When building the thing, complete understanding of the existing code, frameworks, libraries, and infrastructure is not possible. All of these abstractions eventually leak and break down at scale. Every one of these parts is also continually changing! Even if you could solve all of those problems, you'd still have to deal with people problems - incentives never align, budgets change, and alliances shift.
I don't have a formal proof, but I'm strongly suspicious that the only actual way to know how long something takes to build is to actually build it, which of course, may be impossible.
There are a lot of inexperienced software engineers and very little good guidance written for them. What is a new CS grad to do when asked for an estimate? How can a new grad learn to produce accurate estimates within 3 months?
A new grad entering industry is going to be doing a lot more learning than doing -- or rather learning while doing.
I know from experience that explicit doing is highly predictable/estimate-able for the experienced software practitioner.
I have a suspicion that the learning side would be predictable, too, if our industry could do a better job of articulating what the practitioner skill set actually is -> then a pedagogy could develop and it could be said that learning X-Y-Z takes on average this long for a set of students. Etc.
But we as an industry do not seem to be near this level of clarity -- in large part because we don't even have the vocabulary to frame it this way... in terms of learning vs doing...
Now what this means for the new CS grad is not the best story. You'll rather have to play the chaotic game a little bit, which includes a mess of things like doubling estimates and working weekends or what-have-you depending on the work culture you find yourself within.
That ^^ in the short term.
In the long term what you should do is practice on your own:
1) ALWAYS privately plan and estimate your tasks to the best of your ability--on your own, you may not get benefit by exposing your "practice" to the higher-ups
1a) hint: your tasks should be as scoped/small as you can make them and they will look pretty simple, like this: "design widget X", "design API interface", "implement API interface", "test API interface", "implement widget Y", "learn how framework X lifecycle works" (yes! even learning tasks can be estimated!), and so on. The key is that you try to keep them under a day and ideally even smaller on the order of hours.
2) RECORD the time it takes and compare to your estimates. --> LEARN from this.
If you do this conscientiously you will find that your estimates improve dramatically over time and you'll be able to out-estimate your peers to their own wild confusion.
This skill set will pay off in your ability to deliver AND have weekends and hours with your family. Because you will be able to "see time" and protect yourself. You'll have protection and ammo even in the worst pressure-cooker environments because you will be able to say "No" with confidence. Or rather you will learn how to say "Yes" and what to say "Yes" to. (Ie. you will get really good at priority negotiation etc.) And you'll cultivate delivery trust with superiors and everyone will be happy.
The main reason you get these claims that "business just doesn't understand software" and "they put too much pressure on us" is because "business" side doesn't trust us. Once you get good at estimates and delivering on them, that trust is restored and everyone can be made happy.
BUT -- and here's the rub -- it takes time and conscientious effort and variety of experience to reach this level of confidence. My advice: Ignore all the philistines who say it can't be done....because they'll just try to talk you out of the effort that they weren't or aren't willing to do themself.
The steps to do this task are enumerable and would be similar across similar tasks.
I believe that the conclusion "software estimation can't be reliably done" comes from a standpoint that claims tasks like these ("reverse engineer X-Y-Z...") are somehow completely novel each time we encounter them. In my experience, they are not... even "soft tasks" like "design a Thing" and tasks with unknown details like "reverse engineer X-Y-Z..." -- when you do them more than once -- will be seen for the predictable beasts that they are.
But only if you approach them with the attitude that they aren't these novel beasts. That almost all of the work of the working practitioner is NOT like that of the researcher on the bleeding edge of some novel discovery.
This takes advantage punters' failure to accurately multiply probability - 10 events with 80% probability have joint probability <11%.
Something similar happens with planning, where people fail to compound many possible delay sources accurately.
Dani Khaneman covered this in Thinking Fast and Slow, also showing that people overestimate their ability, thinking they will outperform a reference class of similar projects because they are more competent.
It's profitable for bookies because they have a house edge, and that edge is increased the more subsequent bets you make. The house has more edge with an accumulator than a single bet.
People like to do accumulators because it's more meaningful to win large amounts occassionally than more regularly win less meaningful amounts.
So it's a "trick" to simply increase gambling.
If you had to pick out 8 evens shots in sequence and intended to roll over your bet each time it would have the same effect / outcome, but starting with a pound by the last bet you're effectively placing a 128 quid bet on an evens money.
It's not that the player thinks that they have a better chance of winning than 1/256 it's that it effectively forces them to gamble a lot larger amount in the situations where only 7 out of 8 of their picks come in.
And that's before considering the edge. If we consider that these are probably events that happen more like only 45% of the time (at best) then instead of a 255/1 shot we're looking at 600/1 shot.
Where that gets painful is, say you estimate you can get something done in two days. In reality, you're twice as fast (there's that speed metaphor), so you actually get it done in one day. Yay, you saved a day! Now assume you're twice as slow as your two day estimate. Boo, you spent TWO days longer. So in terms of duration, getting it incorrect in the painful direction seems like a bigger mistake.
I don't think this is the same phenomenon as the author's mean vs. median dilemma. I'll bet both the mean vs. median and the productivity vs. duration dilemmas are real factors though.
If and when you're using known features of a known framework, and that's all you're doing, the estimates are accurate and the work is completed quickly.
There's always that one edge case you haven't considered, that one algorithm that doesn't work as well as you expected, that small change to the requirements the requires a completely different approach.
But I'd argue that that's not where the bulk of the uncertainty comes from. I think that in the software industry as a whole, most of the problems are solved by people with at least a decent understanding of their tools, and estimates still suck.
The problems is that the tasks we deal with are always new, unsolved problems. That's the nature of software. Unsolved problems come with uncertainty. That's the nature of unsolved problems.
Carpenters on the other hand deal mostly with solved problems. They just need to execute.
I have two habits when I estimate: First, I like to give a range or even better a triple estimate of "optimistic", "expected", "worst case". (And btw, you should expect things to weight toward the pessimistic side, because you always find new requirements/problems, but you rarely discover a supposed requirement is not really one.)
Second: I like to break down a project into tasks of just a few hours and then add everything up. I usually don't share that spreadsheet with the customer, but it helps me a lot. Pretty much always it gives me a number higher than I'd like, but which is almost always very accurate. A couple times I've ignored the result and given a lower estimate because I really wanted some project, and it has always turned out the spreadsheet was right.
Well, one time I combined these two approaches, so that my very finely-chopped estimates all had best/expected/worst values, and I shared that with the customer. Of course they took one look at the worst-case total and said, "How can a little thing like this possibly take 2 years??" I didn't get the work. :-)
EDIT: Btw it feels like there is a "coastline paradox" here, where the more finely you estimate, the higher the max possible, so that you can make your estimate grow without bound as long as you keep splitting items into greater detail. It'd be interesting to see the math for that.
1) The person(s) doing the estimate aren't qualified to do so.
2) There is a disconnect between wishful thinking and reality.
3) There is some arbitrary future milestone (i.e., "We need to ship by ____ because ____ is happening the following week.") that is independent of software development.
4) Most importantly, when the deadline is missed the estimate / estimators are not questioned, the software team is.
I've been at this a long time - too long? - and the narrative that IT is __always__ at fault is a myth that needs to be buried.
devs estimate known risks. Ideal path + predictable delays. The further reaches of the long tail are the unknown risks.
known risks are estimated based on knowledge (hence, a question for a dev), unknown risks are just an adjustment parameter on top of that estimate, possibly based on historical evidence (there is no reason a dev could estimate any better).
It should be managements job to adjust a dev estimate. Let's be real here - I've never heard a real life example of management using stats for this kind of thing, or being enthusiastic about devs doing the same.
Perhaps if management is taken seriously as a science, things will change, but I doubt it.
Bizness is all about koolaid-methodology-guru management right now, very much the bad old ways - a cutting example of workable analytical management would be needed for things to change, but this is unlikely as all the stats people are getting high pay cool ML jobs, and aren't likely to want to rub shoulders with kool-aider middle managers for middle-management pay..
I've also been at one shop where we used a more sophisticated modeling approach, but built in house. Reception was warm in at least most of management.
So my experience isn't that these tools don't exist, or that management is unwilling to use them - but rather that management has trouble accepting the implications of using them properly. Specifically, if any of the inputs of the model change, you should update those and re-run the model. With linear (e.g bullshit) GANTT, you just say "oh, we missed this week of work, move the endpoint forward a week". With more careful modelling adding an unexpected dependency or a couple days work in the wrong place can suddenly add 3 months to your worst-case model. Exec's really don't like that happening without a whole song and dance about why, so there is a tendency to "freeze" the models in ways that they become progressively less useful. Worst case they are asymptotic on meaningless.
This is an extreme example, but one I now live in daily.
My current full-time-ish gig is working on a pretty enterprisy system for law enforcement. To this date there hasn't been a single feature request, or bug fix that took more than 16 hours of development time. And so I know that I can typically finish something within a few hours to a day of receiving the task. UNLESS my manager wants to discuss ad nauseam what he means when he says "intersect an array". Or get stuck in 2 day long code reviews where my manager makes me sit behind him while he goes over every single line of code that changed, then gets side tracked and starts checking emails, chat messages, text messages, calling other developers in to check on their statuses, and even watching youtube... while I'm stuck in his office waiting on my code review to be done so I can go back to my 5th day of trying to complete a task that would have taken only a couple of uninterrupted hours. /rant
And this is why I pay $120 a week for therapy.