Hacker News new | past | comments | ask | show | jobs | submit login
Why the last 20% of work takes the same amount of time as the first 80% (matvoz.com)
192 points by matvoz on Oct 22, 2013 | hide | past | web | favorite | 52 comments



Love the analogy!

The worst is when you finished the last 20% and get out of traffic only to discover you drove to the wrong city ;)


Love this analogy because you can extrapolate it even further.

I'm the kind of guy that travels the first 80% on the highway, reaches the traffic-bloated city, and actually makes a U-turn to head towards a totally new city, thinking "I'll visit that city later".

Or, quite often, halfway towards my destination (let's say 40%), I'll see a board saying "Next exit: even better city", and take that exit. After taking a few of these exits, I will have spent hours or even days on the highway, without having reached any city, or even coming close to one.


I love walking around in the city.

But by the time I get there I find all my money has been spent (fee) on transport, and I have to choose between window shopping (unpaid work) or jumping on another bus (client) and hope it goes past my original destination..


I can't if you're being sarcastic, or just exhibit some really strange driving behaviours


The analogy is nice, but I'd enhance it a little bit to emphasize the responsibility of, ahem, driver and make all of this more useful. Unless it isn't his first trip to the city, the driver needs to reduce his optimism about the arrival time. Especially at the beginning when he has a lot of time, he needs to avoid all those tempting things like: playing with his new GPS to design the best route; exploring the new and/or intriguing parts of suburbia, changing the cars for fun and so long ...


This a million times. To continue the analogy: Only with experience will the driver know approximately how long the drive should take. Read about the traffic all you want, but until you drive there multiple times, try not to give a set arrival time (or prepare to arrive late).


I would say the same. You need to be a local to know your way around. Until you become one, you are going to miscalculate your arrival time. This is why good developers are exponentially better than bad developers.


your comments show the power&depth of this analogy. the challenge for an effective local driver is to avoid routine and remain cautious and flexible to react to the changes in his leaned-by-heart territory: new rules, roadsigns, changes in traffic organization, self-driving cars etc.


True. I may be the best driver in Slovenia, but when I want to drive to Manhattan, I will have a bad time (pun intended). I will learn to drive there faster than somebody who is also bad in Slovenia, but I will still need my time.


That's the most annoying comma I've seen in a while. :) Great article though.


The author is probably not a native speaker of English. For example: Their name doesn't sound German, but that comma makes perfect sense when you translate the sentence to German. Whatever their native language may be, it might have similar rules of comma usage.


This is what happens if you think in one language and write in another. :) Yes, I am from a bit south of Germany, from Slovenia.


Great Analogy, the basic idea is similar to this quora post (linked below) both in the travel metaphor and the fact that both deal with software development effort estimation:

http://www.quora.com/Engineering-Management/Why-are-software...


Simple, when you said you where 80% done...you actually weren't (presuming your metric is time).


In the analogy, the metric is distance, not time. So I think the metric in the software proverb is feature-completeness.


How to measure a project's progress? If time is your metric, then the last 20% should be done in 20% of the total time. (And that's right, when the OP says she was 80% done, she actually was 50% done.)

If your metric is something else, what could it be? And perhaps it is not the best metric after all: a loading progress bar that goes quickly to 80% and then slows down (by a factor of 4) until the end may not be the best progress bar after all.

I guess that the difficulty is to get a precise idea of the time you need to finish something.


The metric given to the client is that of time, but the metric that ends up being measured is (for lack of a better term) "visible product completion", i.e. percentage of product that 'looks' completed. If I read OP right, the last 20% that is being talked about in the article is mostly a nip here and a tuck. The overall product, from this point onwards, almost always remains largely constant.

To bring in yet another analogy, think of the project as a car being refurbished. The last 20% of work may not change much in terms of external chassis, but it is all about fine-tuning the engine, and the drivetrain. The client still sees the same car that you showed him at "80% done," except now the engine runs much better and the car purrs like a sexy, sexy kitten.

The problem, IMHO, is you can't give an accurate metric no matter how much you try.

If you give a liberal estimate and end up doing 80% of the product in 50% of the time, the client will expect you to finish the 20% in 12.5% of the original time-frame - the very 20 % that almost always takes about 80% of time. On the other hand, if you take 80% of the estimated time-frame to deliver 80% of the product, the remaining 20% will always trip you up because it almost always needs another 80% of time.

I've found that the best way to keep everyone happy is to finish 80% of the product in 50% of the time but show the client 50% of the product and spend the remaining 50% of time, finishing the 20%. It's a win-win, IMHO.


>I've found that the best way to keep everyone happy is to finish 80% of the product in 50% of the time but show the client 50% of the product and spend the remaining 50% of time, finishing the 20%. It's a win-win, IMHO.

That is a very clever approach. I don't think I've ever heard of that before. I'll have to give it a try.


You have a very good point; it can be very difficult to judge how far you are through a project. In retrospect, time is often the best metric.


Time is a great metric, but the total is only known when you're done. Before that, you need other metrics that you can compare to your estimates, if you want to be able to correct.


In 2002 Microsoft noted that by fixing the top 20% most reported bugs, 80% of the errors and crashes would be eliminated.


I think it is because people are terrible at considering hidden complexity and the odds of something unexpected happening.

My biz partner has a philosophy of "there is no such thing as a simple app." Hidden complexity is everywhere.

Consider user login. A form with username and password right?

Do you show the password as they type it or obfuscate it? What if the user forgets username or password? How do you handle password reset? Did you make sure you are sending over https? What about captcha? Do you use one at all? Do you have password strength requirements? How do you want to store the password data? What about security questions? What about error handling? Do you support oauth like facebook login? Will this work for login via api? How are you going to test login? Will you write feature specs, unit tests, etc?

Now, how often do people think through features in that kind of detail when estimating? Look at the above questions. All of a sudden the last 20% starts to look like 80%.


In case the site isn't loading for you (wasn't for me):

http://webcache.googleusercontent.com/search?q=cache:www.mat...


Sorry about that. I don't get so much traffic usually.


The saying is something like: once you're 90% done, you're halfway done.

For code, it's because you've solved all the easy stuff and can finally see a product. It's the unexpected UX problems along with the dumb-but-hard-to-solve bugs you're now handling.


Totally. I find it's especially true for UX or animation stuff. Like you get the whole application working and then animations need to be put in, but those end up causing issues with states/transitions and smoothness, which take a long time to iron out into a nice place.


While doing some research for an article[1] I read a paper[2] in which the main causes of underestimating are given as:

1. "Frequent requests for changes".

2. "Overlooked tasks".

Agile methodologies are good at placing #1 front and centre. But importing that fusty old concept, the checklist, might help a lot with #2.

[1] http://confidest.com/articles/how-accurate-was-that-estimate...

[2] Albert L. Lederer and Jayesh Prasad. "Nine Management Guidelines for Better Cost Estimating". Transactions of the ACM. February 1992.


My Math teacher in College used to tell me it takes 5 times the effort to go from 70 to 100 marks, than it took to go from 0 to 70 marks. Finishing well is hard and often takes more than just raw application of hard work.

To get it really ~100% percent right, you need to not only work hard, or work more but you need to develop a strange obsession for quality. And that only comes from continual reading, and upgrading your skills. To constantly learn from experience and other people's wisdom and apply it to your daily craft.

Getting to the 80% done part, if you note you will be often working on stuff which is largely solved and rewritten many times. Things like working with DB or a file, or parsing an XML- Basically stuff like that is what constitutes first 80% of the project. The remaining 20% is what your actual project is, its functional requirements, its quality and other stuff like that.

This is not just with software, building a home is very similar. It takes very little time to build the frame structure(walls, foundation, roof etc)- A lot more time to actually finish the home which is livable.

For that you can only do it right if you go slow but steady.

Remember you either go far or fast, never both. You will have the stamina or strength to only do one of them.


Home building also follows the same analogy. Our house was one of the first completed in our neighborhood, and we moved in before most of the others began construction. The foundation, framing, roofing, and siding all go up pretty quickly. As the house nears (visual) completion, the detail work and coordinating the subcontractors and inspectors stretches out the timeline.


The last 20% takes time because you're in the realm of the last inch.

Now listen to the rule of the last inch. The realm of the last inch. The job is almost finished, the goal almost attained, everything possible seems to have been achieved, every difficulty overcome — and yet the quality is just not there. The work needs more finish, perhaps further research. In that moment of weariness and self-satisfaction, the temptation is greatest to give up, not to strive for the peak of quality. That’s the realm of the last inch — here, the work is very, very complex, but it’s also particularly valuable because it’s done with the most perfect means. The rule of the last inch is simply this — not to leave it undone. And not to put it off — because otherwise your mind loses touch with that realm. And not to mind how much time you spend on it, because the aim is not to finish the job quickly, but to reach perfection.

From Aleksandr Solzhenitsyn's book "In the First Circle".


I should add that as a pragmatist, sometimes you have to leave the last inch - reaching perfection isn't always compatible with life in the real world :/


How is driving similar enough to coding? You can take 2 coding projects and the times for a team to complete them and their estimates will differ wildly. Commuting on the 280 to Palo Alto from San Francisco is 45 minutes on average. There is nothing mysterious about this or complicated. Coding is much harder and the OP is guilty of a bad analogy fallacy.


An analogy is like a car. If you take it too far, it breaks down :)


I knew a manager whose favorite saying was that we just needed to do the 20% of the work that got us 80% of the way there.

Well, needless to say, he never got a minimal viable product out there.

It definitely makes sense to be "lean" and not do anything that doesn't need to be done, but if you're lean you still need to complete 100% of the MVP.


This kind of fractal complexity is true per batch. The problem being described here is quite simply large batch sizes, with many projects treated even as just a single batch. The smaller your batch size, the more constant your overall project velocity is.

80/20 is therefore a sign of a poorly managed project (since visibility of true velocity is hidden, planning is impossible at best or based on false data at worst, and overruns are collected and unpacked only at the end).

Unfortunately very few managers involved in planning get this, and as a result don't recognise their responsibility is for delivering the whole at a predictable rate by managing this process. (Instead we often see a sick project management culture of not giving the client what they need and whipping the delivery horses harder, under the illusion that this is just how projects get delivered - always painfully and late, if at all.)


This is true, but not intuitive or readily explainable to "people" (usually senior management, hence the quotes :).

A great read on the subject of batch sizes and limiting WIP: http://www.amazon.com/The-Principles-Product-Development-Flo...


I think this is a great analogy. The reason why it works is that we tend to relate progress in one intuitive metric (distance travelled) to a less intuitive one: time. Why shouldn't 80% of the trip be done when 80% of the distance is covered? Likewise, a merely sketched-out project can look dangerously complete.

In both cases you get caught by complexity. It's easy enough to drive in a straight line for a while, and your app's login system will likely work the same as most others'. But the devil, in both the city and in your code, is in the details.


I agree to some extent. My hypothesis is that during a project you take a lot of shortcuts, because solving that last part of a task sometimes involves a lot of work to get everything perfect and working for every user. It could be making a responsive website work in legacy browsers, or it could be a script I know works in my dev environment but I haven't yet tested in production,. Even so I close my eyes, mark these task as done and move along.

So the last 20% is actually a bunch of 0.5 percenters pushed to the end of a project. At least for me.


The obligatory quote:

"The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time." — Tom Cargill, Bell Labs

[http://en.wikipedia.org/wiki/Ninety-ninety_rule]


I believe you need to include arriving at your location and finding your directions were wrong/GPS was out of date/were given the wrong address, and need to re-do most of that last 20% drive again.


Great analogy.

I think next time if we try to guess the amount of time we need to complete a project, we will just ask how long would it take to complete up to Stage 80%. And just times it by two.


Wow I totally agree. And it's probably why I don't finish anything, because I am conditioned to expect it to be same work as earlier.

Not just for this, but almost everything.

The Paredo Time Principle?


Pareto Principle

The Pareto principle (also known as the 80–20 rule, the law of the vital few, and the principle of factor sparsity) states that, for many events, roughly 80% of the effects come from 20% of the causes.

http://en.wikipedia.org/wiki/Pareto_principle


Great analogy. This is just the thing I wanted to see. I am in that last 20% phase with my project and it really is a lot more work to get to a 100%.


It's easy to write a script that does one thing in exactly one way, but it's hard to build something that everyone can use.


I always thought it was because the last 20% is also 80% of the project because everyone underestimates the task by roughly 60%.


Lol, am I the only one who thought of Michael Douglas in "Falling Down" while reading the last paragraph?


Now the analogy is complete.


Applies only to software programming? Maybe its the same for any life task?


Because we get bored.


If you don't overcomplicate your software, this problem is much less significant. If your code is easy, fixing bugs will be easy.


Yes but then that first 80% looks much less impressive because it took a lot more time to write. When the product is finished it shows, though.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: