Hacker News new | comments | ask | show | jobs | submit login

Agile will be dead as long as it's the dominant management methodology and there are page-views to be had. It'll be replaced with essentially the same process but with different terminology. Long before the Agile Manifesto, when I was working at NASA, we talked about the reality that the state of knowledge about a project is at its lowest at the beginning, and that any realistic project planning and management process must acknowledge and accommodate that fact. Subsequent research has confirmed that human beings consistently underestimate complex tasks, and that this bias persists even with experience, so no matter what methodology you adopt, you will always be under unrealistic deadlines. This results in distortions to whatever process you use, and flaws in whatever product you produce, and these distortions and flaws will be blamed on your development process, since no one will accept that it's simply an unfixable feature of human nature.

I've always felt the problem is that we're trying to estimate something unestimatable. My thought's always been that if the management needs of the company are that developer time should be quantifiable and that projects should have a direct ROI, then the most sense would be to only chase ROI in small projects of little risk, and leave business growth to big projects with lots of ambiguity and lots of risk. This is the same for any kind of business venture; why should writing software be any different?

In other words, if your process involves predicting time needed to complete a project, you've already failed.

My father-in-law is a project manager for a gigantic construction company, so it was interesting talking about estimation for him. Estimates are much more predicatble in his field, even when working on multi-million dollar projects with timelines that span several years.

The difference was that in construction, the same subprojects have to be done over and over again, so subcontractors can easily be given and held to realistic deadlines, and your only bloat comes from the usual hiccups. On the other hand software engineers _by definition_ estimate things they've never built before. If they've already built it, they can just port it over and start working on something new.

Combine that with everyone's tendency to underestimate, inability to determine unknown unknowns, and the usual hiccups, and we're left with the usual refrain: accurate software estimation is basically impossible.

The interesting thing is that in software engineering the cost of additional sub components is zero. Copying bytes is effectively free, so we gain no useful information on how much it costs to build a widget.

2 observations from my experience:

1 - You can fix the time or the scope of the project, but not both. I can say, "I will deliver project X in 4 months" but you have to be willing to let me toss out scope, and the earlier the better. This does work. Just like I can say, "I will promise to deliver all the scope in project X, I just can't promise how long it will take." Agile helps break this into smaller chunks, and allows you to get more visibility into what's realistic more quickly. I've seen too many waterfall projects slip by 3-6 months at a time only when the final deadline approaches.

2 - I've seen more projects slip because of "Unplanned work" and "Missed Dependencies" than due to "Misestimated work". It's either scope creep, or "I didn't realize we needed to test integration" or "Person X didn't give me what I needed." The only counter-examples I've seen are when engineers aren't consulted in the estimates.

I would certainly agree on both of these assertions.

I think, Digitalization helps a lot to change the mindset of management by just removing unnecessary management levels and by bringing the remaining levels closer to product development. This happens to a point where management by deadline becomes a weakness, because the now missing management levels cannot hide weak links (from upper management, investors, share holders, customers).

>> "the state of knowledge about a project is at its lowest at the beginning"

Sounds like the Cone of Uncertainty:


Do you have any references to "subsequent research"? I am truly interested in digging deeper.

Daniel Kahneman's book "Thinking Fast and Slow" is a great overview of the field of cognitive bias. For a more academic treatment of his research, check out "Heuristics and Biases". The part of his work most relevant to project estimation can be found in the "planning fallacy" section. Aside from Kahneman, there are numerous papers on software estimation, many of which note these consistent biases (for example Jørgensen, https://pdfs.semanticscholar.org/fd87/d248dd55f59d8d93742fb3...).

I know that Code Complete 2 has quite a few discussions about the phenomenon of consistently missed estimates and many references about those. I would definitely look there for a starting point about the impossibility of successful estimation in software.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact