Hacker News new | past | comments | ask | show | jobs | submit login
Agile won the war but lost the peace (allankellyassociates.co.uk)
167 points by mpweiher on Nov 11, 2018 | hide | past | favorite | 165 comments

Sprints are where all of the issues with agile really come from. Constant arbitrary, meaningless deadlines that serve no purpose but create rushes, tech debt, missed “commitments” that serve as a meaningless metric to use against teams while made commitments give no benefit.

Everything around sprints to try to avoid the negatives results in huge wastes of time in meetings attempting to plan better rather than actually getting work done.

Sprint are the root cause. Just use Kanban and you eliminate most of your problems.

Last week, at the end of a work day, we started talking about what the schedule was going to be for Saturday (not a 6th work day -- we're not on a M-F schedule here). Someone asked "Is it a goal-based day or a time-based day?" And there was an answer ready! Fantastic.

In 20 years, I've never heard anyone ask that in the software field. In software, I've always been expected to work as fast as possible at all times, and when we get near an (arbitrary) deadline, to stay late as needed.

Software is full of young people, and people who have never managed any kind of team before. I have no problem working for someone younger than myself, but I do think that people should have experience managing some kind of team before they're thrown into the deep end. Software is much more complex than almost anything else we make, and it's no wonder that just figuring it out as we go along has yielded less than stellar results.

I've seen some young people who are very good managers, but I've also seen some absolute disasters. I've also seen some older folks who are absolute disasters. I shared a story about both kinds of manager in How To Destroy A Tech Startup In Three Easy Steps:


I've often heard this (or a similar?) distinction as "man-hours" vs "calendar time"

ken doesn't describe what a "goal-based day" means, but I don't see how it could match the distinction between man-hours and calendar time. The difference between today and tomorrow is one day of calendar time. If you have a team of 3 people working 5 hours a day on a given project, then in one work day of calendar time 15 man-hours of work will be done on the project. Man-hours just aggregate calendar time contributed by multiple people (and ignore calendar days when no work occurs). But none of that has anything to do with goals.

In defense of sprints, not everyone is the same. Deadlines for me are a strong motivation to get things done, whereas "no deadline, just do it at your own pace until it's done" will cause me to procrastinate and get it done much more slowly.

There is no silver bullet for software development process. It takes hard work and flexibility to optimize your team's work flow.

I've been reading "It Doesn't Have to be Crazy at Work" and they have a chapter about this. Deadlines are great! Estimates are terrible, and humans are terrible at estimating. Scope needs to be elastic to accommodate this fact of life.

If the project is not done in the 6 weeks you set aside for it, just ship something anyways and move on. It was only worth 6 weeks when you planned on getting it done in 6 weeks, so why spend more? Go ahead and plan another sprint if it turns out what you were able to build in the time allocated wasn't enough, and you really need more. (Put it up against the other opportunities you have, and give them all a fair chance at those finite development resources, something else might also turn out to be more important.)

Just don't spend forever and drag out the deadline, throwing off the timeline for whatever you had planned to do next, because you needed more time for the thing that underestimated how big it was, so it wasn't really possible to do that whole thing in only 6 weeks.

The release that is reduced in scope might be totally insufficient, or, it might be just what your users needed! The knife also cuts both ways – if you're willing to accept that project scope can change in either direction, you can occasionally deliver things early too.

I really like everything about this book.

If you set aside 6 weeks and it takes 12, you suck at estimating. Not only are you likely to ship a broken product, but you also have no way of making intelligent business decisions about possible projects.

Project A will bring in $X. You estimate 6 weeks. It takes 12. Project B will bring in $Y. You estimate 6 weeks. It takes 7. If X is similar to Y and you bet on the wrong horse, you've dented and/or sunk your cash flow for no reason.

Saying "Humans suck at estimating" is nonsense. It's a skill that can easily be improved - usually at the cost of some up front planning - and there are plenty of resources to help you with that.

The real problem with Agile is that it gives management an excuse to do no estimation at all. Some alpha-wannabe manager bangs their fist on a desk and says "I want this in three weeks".

Instead of saying "You're insane" teams are pressured into trying to make up for someone else's laziness and lack of contact with reality.

>Saying "Humans suck at estimating" is nonsense.

And yet it's the conclusion every major thinker (and practicer) in our field has come to (plus there's historical evidence from tons of mis-estimated projects, regardless of scope and budget).

So, whether does the idea that it's "a skill that can easily be improved" come from? Wishful thinking? I don't think the universe works that way.

Sure, you can learn to be less widely off. But you can't never learn to be regularly correct at estimating, or even close within a small margin.

At best, what happens is that quality get shitty and corners get cut, and you get the product at the correct deadline, but it's not the same that you wanted to ship before development started (even if the features are nominally there).

> So, whether does the idea that it's "a skill that can easily be improved" come from? Wishful thinking? I don't think the universe works that way.

I've come up with two names for this kind of thinking, one a bit grumpier than the other.

The less-grumpy one is the "Is From Ought Fallacy": It's well-known in philsophical circles that going from "Is" to "Ought" is wrong; that is, saying "something is this way, therefore, that's the way it ought to be" is a terrible argument. Well, when you flip it, it gets even worse: "Things ought to be this way, therefore, that's the way they are" isn't just a bad argument, it isn't quite sane. Yet look around you, and see how often it's used. (Hint: Abstinence-only sex ed. Hint: Drug prohibition.)

My grumpier version is "Arguing With Reality": You can argue with people, whether you define "arguing" to be a screaming match or a logical defense of a thesis. That works. It's sane behavior. Going out and arguing with the rain that it needs to be sunny now... isn't. Rain's gonna rain, people are gonna be bad at time estimates, and making logical arguments about why the real world shouldn't be like that doesn't change the real world one iota.

Maybe they're just variants of the "Utopia Fallacy", or the perfect being the enemy of the good: "If the world were perfect, it would be like this, therefore, doing anything else, even if it gets good results, is imperfect and compromising perfection is wrong." Yeah, pull the other one.

> And yet it's the conclusion every major thinker (and practicer) in our field has come to (plus there's historical evidence from tons of mis-estimated projects, regardless of scope and budget).

I agree fully and I think the main issue is this: People estimating, wanting estimates, "fueling" the workplace and ways in which businesses operate where estimates and milestones are the norms we work around...

The big thing I see missing is the fact that mostly everyone involved in these "mental work simulations" (as I'll call them) is calculating all of the tasks for completion against time it takes to complete tasks.

There have been some people commenting on, "I am glad to give an estimate so long as I'm allowed to add ample padding time for unexpected issues, scope creep, and then I'm allowed to double the entire estimate or at least add 75%. If I can get that estimate signed off on I'm fine with estimating".

And that's probably where I'd say I sit and feel most comfortable: "I'm fine with giving estimates so long as the person singing off on them realizes I'm really just giving a best case scenario guess with a bunch of worst case scenario 'what-if's' built in as well as some 'inevitable scope creep / customers deciding to change specs on the fly' type stuff." But something is still missing. There is a lot of lingo and business-speak and cross-department buzzword ad-libbing taking place but not a whole lot of ground is being broken in terms of moving things forward. Why is that?

The big thing that's missing from ALL OF THIS (IMHO)? Everyone is basing all of these estimates as if things were taking place in a vacuum. Even in that vacuum, over time, estimators, project managers, and developers have come to realize that including padding time for inevitable issues, scope and feature creep, and unforeseen circumstances is an absolute must. Even then, that's inside a vacuum of sorts.

I truly believe it's impossible to plan without falling prey to the "Estimate inside a vacuum" issue. Even if you are a driveway paver and you've paved 10,000+ 100' driveways in your illustrious 20 year career there is going to be something unforeseen that comes up that will prevent your 10,001th job from being completed under time and under budget. The problem is that there is no way for us to truly determine any and all possible issues that could arise. The bigger issue is that bosses and clients don't tend to care when money is on the table, jobs are at stake, and careers are on the line. So we create estimates and we miss them and promise to get it right next time. Even though we know we never will, vacuum or not.

Do you have specific resources you would recommend for learning to produce better estimates?

I'm also curious what kind of work you do that you find estimating effort to be reliable, and what kind of tracking you do to keep yourself honest (e.g. I often see people say something will take two weeks and the only reason it happens in two weeks is they end up working day and night for the last 5-7 days when they realize that's what it's going to take to make it happen).

One thing: More, smaller estimates, have the property of being wrong by a lesser amount, and they are likely to cancel each other out. Estimates in hours instead of weeks are far less likely to block your work in progress for as long. Teams also get more feedback and opportunities to practice estimating, and more granular work is rewarding more often.

That idea works out, and is the basis of agile vs waterfall, but there's a point where you shrink the timelines so much that you'll end up spending more time estimating work than you soend doing it

Quick estimates that are good magnitude approximations but often somewhat wrong have a way of getting better, whereas estimates whose main purpose is to be within some delta of the actual time are working from the wrong value metric. A reasonably accurate aggregate of estimates doesn't require them all to have been perfected first. Teams that are obsessed with this will be slower to a build.

This is a theme in parallelization that I remember from my more senior and upper level Computer Science seminars at RIT. I remember vaguely this coming up in one talk in particular about Manticore and Concurrent ML.

The work splitting algorithm needs to monitor the overall depth of the queue, as well as the number of workers that are online and their propensity for finishing jobs.

It doesn't help to divide a task into 100 chunks instead of only 20, if there are only 8 workers, especially when each split has a marginal cost that will add up again and factor in to overall cost at the point of final integration.

But with a pool of 1000 workers, dividing the work into only 20 or even 100 chunks might still be leaving significant amounts of power on the table undesirably.

You could try to optimize for both, strike a balance between achieving an earliest possible completion time, and minimizing waste. But there are going to be maxima and minima of either of those variables along a continuum, may not be practical to find the sweet spot.

One place where this analogy breaks down, is that computers do not care one iota if their work is redundant, or if their work must be thrown away. Human developers care, even if it might save the employers some bit of money or reduce a risk. Humans usually will not produce identical outputs either, when they are assigned redundant workloads. Most humans would probably prefer not to work on a project that is redundant at all. Computers don't care, ML doesn't care. But it would be soul crushing for devs to have to work in an environment like that. I'm sure it happens more than I would have imagined.

I find this (short term estimates) only work when you are self sufficient as a person or team. E.g. as soon as something like: “I need to get access to the managed AD service” or “the service desk needs to be trained to handle requests on this new feature” come in, I find planning to be very difficult. That AD access technically takes 10 minutes, but might take 2 weeks of approval meetings and emails.

Especially prevalent during the initial start of a new project

> Saying "Humans suck at estimating" is nonsense. It's a skill that can easily be improved - usually at the cost of some up front planning - and there are plenty of resources to help you with that.

Not sure if we think about different things when saying "estimate", but I rarely was able to estimate more precise than "6 to 12 weeks". Unless I am doing something absolutely straightforward, I never really can give good estimate (and since I am forced to do - I just give some random number with a lot of padding which I "feel" I should be able to fit in)

In my practice so far to give good estimate means doing 80% of actual work and leaving these straightforward bits to finish it off. These last pieces are easy to estimate usually, but not before main work is done.

Providing accurate [practical] estimates is like writing software without bugs. If you could do it, you were golden. In reality, you can't.

> Saying "Humans suck at estimating" is nonsense. It's a skill that can easily be improved - usually at the cost of some up front planning - and there are plenty of resources to help you with that.

And yet, we do. Look at the number of failed government contracts in recent history:


And while much of that is about the failures in program management for large contracts, the government aims to build large things at once (as opposed to an MVP and iterations).

When a critical dependency takes longer than expected, the cost snowballs down the chain.

The civilian government does MVP pilot programs all the time and looking at them is one of the first steps in evaluating the merits of expensive proposals. You can't MVP a bridge or a fighter jet though, because the political and economic cost of iteration is ruinous.

But you can mvp fighter jets - thats what prototypes are?

Of course, it may seem to you like the majority of the work is done in the case of a prototype. This is not the case - a vast majority of the work that goes into developing a jet fighter consists of the documentation as to how fast it can fly, its flight ceiling, etc.

Maintnence manuals, and developing a supply chain to ensure we can have a resilient delivery despite war or changes in the economy.

Training programs, to ensure that pilots are able to safely learn to fly the thing

Testing, which of course requires that a prototype be built in the first place.

the only estimates i've ever been good at are the kind where i think through all the problems and break stuff down hour by hour. I still forget things, and some parts inevitably explode. but the blast radius is pretty small.

I can't break project down to 240 units of time on the spot. even 4 hours is still crazy tight, and produces poor estimates. that's barely a minute of thought for each hour. occasionally a manager will be comfortable with me staring off into space for a couple of days. but that's not super common.

I think anybody can come up with good estimates. Almost nobody is willing to say, Take two hours and figure out how long it will take to estimate Project A and Project B.

Estimating in the small is not impossible, and if you are doing sprints you need to know what a sprint's worth of work looks like. But the whole idea of agile is that requirements will change during the life cycle of the project and you need to be able to adapt. You'll show the Product Owner your work from the sprint, and they will want to refine the specs, or see potential improvements, or add features. If you sit down with the initial specs and implement them, you will build the wrong thing - so it doesn't matter if you estimate the time correctly.

> Saying "Humans suck at estimating" is nonsense. It's a skill that can easily be improved - usually at the cost of some up front planning - and there are plenty of resources to help you with that.

This is completely dependent on what you're doing. If you're implementing yet another CRUD app, then sure, you should be able to estimate that to within 10%. If you're developing something a little less predictable, like a game or something for R&D, then estimates go right out the window.

I loved an estimation tool called Construx Estimate, by Steve McConnell. The few times I used it it was far more accurate than the other estimation techniques we were using at the time.


Another tool in this vein, is Manuscript

This is more like an issue tracker/project planner with statistical modeling capabilities built-in, I don't know if it's doing the same thing under the hood, this is part of their "special sauce"


  > The real problem with Agile is that it gives management an excuse to do no estimation at all.
  > Some alpha-wannabe manager bangs their fist on a desk and says "I want this in three weeks".

  > Instead of saying "You're insane" teams are pressured into trying to make up for someone else's
  > laziness and lack of contact with reality.
There's nothing Agile about that. Estimates should come from the people who do the work, not from above. If a stakeholder absolutely wants something in 3 weeks, they can only get what can be made in those 3 weeks. If the dev team says it can't be done, it can't be done.

Don't blame bad managers and bad companies on Agile. Agile needs to empower the people doing the actual work, or it's not Agile. Managers need to facilitate, not bang their fists.

You can also use modeling, which for a bunch of STEM people seems to almost never come up. Take projects you've done, do Monte Carlo simulations, and then start giving estimates with confidence bounds. Can also use things like Little's Law to determine lead times.

That only works if you regularly get similar projects.

Throw a new type of project or a new client with special needs, and bets are off, Monte Carlo or not.

Yea I've been down this discussion path before. I'll just refer you there, and you can feel free to chime in with any new thoughts.

My general thoughts is that typical software engineering does not have drastically more degrees of project uniqueness than other engineering fields.


>My general thoughts is that typical software engineering does not have drastically more degrees of project uniqueness than other engineering fields.

I really doubt that. In construction for example, tolerances, materials, requirements, etc are know well before even a single brick has been laid, and if they are to even consider changing them mid-construction, detailed new plans about everything that changed is in process.

That's almost never the case with software requirements.

The same tools, processes and technologies are also used for decades or in some cases centuries totally unchanged. In software a domain (e.g. web programming) can be completely different every 5-10 years.

Humans don't suck at estimating, humans lie

lie to ourselves about our true capabilities

lie to our boss / team about how good we said we were

lie to the business about how well spec'd the job is, about how the current architecture matches their known requirements, lie about how their stated requirements and the real requirements match up.

Lies to avoid punishment, lies to gain a job that otherwise would go to competition, lies that simply hide the complexity of every job and estimate

without lies estimating would be easy

I'd prefer to say for my team's benefit that we suck at estimating, rather than to say "we lied to the customers about exactly how well the current architecture matches their requirements" or "the partner lied to us about the exact depth of their business requirements."

Being charitable, I'd prefer to say that mistakes were made, without pointing the finger. We're all not perfect, and pretending that we can all be perfect all the time sets the wrong expectation, so I wouldn't try to pin it on a lie.

And I'd prefer to not have to see my boss going under the bus when I have to say "the reason we're not done on time, is because the boss told you this deadline and list of features was reasonable on that day without perfect information. If she had come to us and asked for our opinion, and given us time to talk it over, we would have worked it up and told you, it couldn't be done without making some concessions!"

If only for the reason that, perfect information is not usually available in the time allowed for making a decision. That's not just being charitable to the boss either, that's also self-preservation. The boss can take a lot more of those kind of hits than I can. Even more if we stay together on issues like this. And extending the time needed to make the decision also can have its own opportunity costs associated.

I agree with your sentiment in general. But not every bad datum that costs in accuracy, is the result of a lie. I've seen it said frequently that you should work to defer every decision until it's absolutely required, so you can make it with the best possible information and avoid those costs.

I think that's honestly a lot easier said than done.

I used to enjoy the medical TV series House with Hugh Laurie. The producers had as his dictum, "Everybody Lies" and his role as diagnostician was to work out who was lying, usually leading to a cure. A good manager is a good diagnostician for his or her organisation, able to take conflicting imperfect information and tease out the lies (or incorrect information)

A good leader is able to create a safe space where most people will release their defensive positions and many of the most intractable lies will fall apart.

But we all do lie, most often to ourselves, and not always with evil intent

But I fear the attitude that says "we expect you not to lie (withhold information, fail to speak out) because to do so will hurt the organisation, and you should have the best interests of the organisation / team at heart

Humans don't work like that.

It's not that we estimated 6 weeks and it takes 12. It's that we get a bundle of tasks up-front and are told "it's going to take 6 weeks" for reasons that are negotiated at a higher level than our pay-grade. It's my situation anyway.

The core idea I took out of Agile is that once the high-level estimates are completed, the developers are meant to break them down into bite-sized tasks, put estimates on those, and prioritize them in a backlog with the project manager, customers, and any stakeholders. (In my case we're not building products to go to market with, I work in Higher Ed. We're building solutions for campus partners. So it's not really "Project B will bring in $Y," it's "Project will further our mission and better enable some common workflow." Not to go off-topic...)

What's happening instead is, we get the high-level tasks and then break the projects down into bite-sized chunks, then estimate the parts, and are told "great, now make that fit into 6 weeks, because that's all you have."

Developers don't estimate in hours or days, they use points or t-shirt sizes, because honestly, we do suck at estimating. It's not what you pay us to do. (I mean this honestly, it's really not. This is what they pay project managers to do, and they're literally a pay-grade above us. And they don't write code, though they may be good at math and judging t-shirt sizes. The better ones also know how to keep their poker face, when meeting with the customers. But if your project is slated to last only 6 weeks, chances are you won't even have access to a PM. You wouldn't want one if you could get one, since 6 weeks is hardly enough time to plan and later change course. What would they do?)

T-shirt sizes are better than estimates at a sprint level too, because all you're really doing is horse-trading. If a feature is XXL according to the devs, maybe it moves to the back of the queue in favor of something that costs less and is worth more. That's what you do at Sprint Planning.

Back to devs, we're experts at writing the features, and sometimes we can tell you how long it's going to take and get that right, ... and sometimes we can harmonize in concert with the project leadership, but that's not our core competency. It's just moving faster, and doing it better. Sometimes you find out that the specs are wrong after the project has already started, and it's going to take longer.

We know that some projects are going to take a year, and some are going to take two. And those projects get project managers, and they are frequently delivered on time, with all of the promised features in them! Six weeks isn't an estimate, it's a budget. (And it's not a real budget, hardly even registers on the balance sheet. It's more like an experimental timeline.)

The problem is, customers prefer that we blow the budget rather than delivering part of a product on-time, even if it might have turned out to be enough. They're not willing to put things in order of priority, take an honest look at what will take 6 weeks to deliver, and settle for that. They'd rather cut corners, and then we allow them to make those concessions. Only they don't really want concessions, so we wind up having to spend extra time papering over the gaps.

"The core idea I took out of Agile is that once the high-level estimates are completed..."

This is not a core idea of Agile. The high level estimates are not part of Agile- fixed scope, time, and cost is the opposite of agile. You are just doing extra fake work around what is a fixed design process.

"Developers don't estimate in hours or days, they use points or t-shirt sizes, because honestly, we do suck at estimating." No, they don't estimate in hours or days because those are commitments that are used against them- regardless of whether they are expected to attend other company meetings or other priorities are thrust upon them. Points or sizes are intended to be complexity or quantity of work estimates, which could take different amount of time.

In addition, estimates, whether of time or complexity, have uncertainty. Many enterprises are unable to deal with uncertainty.

If you fix cost and time as part of the budget, what you are left to be flexible on is scope. This is the most natural place for flexibility, as there is are often a wide variety of implementation choices that limit cost and deliver a low budget version of a feature versus one that would require more time.

> This is not a core idea of Agile. The high level estimates are not part of Agile- fixed scope, time, and cost is the opposite of agile. You are just doing extra fake work around what is a fixed design process.

In the first edit of this comment, before I submitted it, I said "we don't really do Agile, we do Agile-Lite" – I cut this line because I've heard this repeated by leadership and I've asked what does it mean, and never got an answer. I almost wanted to include it, so maybe I understand now!

Developer teams are Agile. Dev leadership is Agile-lite.

> The high level estimates are not part of Agile- fixed scope, time, and cost is the opposite of agile. You are just doing extra fake work around what is a fixed design process.

Leadership tries to fix scope, time, and cost on the long-term calendar because we want to deliver predictably, to not appear unpredictable or unreliable. As much as I tell upper management that you can't have fixed scope, time, and cost, they persist in trying to fix all of those.

And their budgets and goals seem to win over my objections, damn near every time!

I'm not doing extra fake work, I'm taking direction from senior leadership. Their work may be fake. Your opinion on their fake work, is duly noted, and I'm going to decline to state whether I agree for political reasons (in so many words, I like my boss and I rarely get time of my own to interact with the bosses boss.) I try to do my work and not rock the boat. Which is why I might get a demand to do something in 6 weeks that is clearly going to take 12, and just do the work. "We'll be given the extra time when we need it." That's healthy, even if it's somewhat dishonest.

> No, they don't estimate in hours or days because those are commitments that are used against them- regardless of whether they are expected to attend other company meetings or other priorities are thrust upon them.

That's a fact. But at some level, if you have this toxicity in your culture, someone is going to have to translate from t-shirt sizes into hours and weeks at some point, and when these estimates turn out to be wrong, someone's head will have to hit the block. (Hopefully theirs and not ours.)

I'd like to think that doesn't actually happen here, and also that's not why we use t-shirt sizes. Maybe just why your team uses them. Or maybe I'm just naive about it.

I somewhat agree. When I had a content team, sprints made sense, since we adapted it to go in line with Advertising's publishing calendar. Development had to be done at a certain point and once we shipped the pieces, we published.

On my old CMS team and with my current development team, we use kanban because it aligns with how we function. We get ad hoc tickets in our queue, we pass them through a triage lane, then they're picked up by data, dev and qa as they're completed and within the items allowed with our WIP.

Waving a broad brush to say x is what everyone uses, will fuck everyone up. Neither agile, waterfall nor kanban are the ideal solutions for everyone.

My rule for deadlines is that it must have a “Why?” associated with it. Deadlines with a Why are real. Usually tied to an event outside of the dev teams control.

Anything that doesn’t have a Why next to it is essentially carrot dangling. You can have targets/forecasts when you expect to have something ready while making it clear that these are guesses, but if it’s not real then it creates real problems if you pretend that it is.

Usually the "Why" is always going to be some variant of, "The client wants this ASAP!", or "The client called and they are complaining about this bug".

How would you force your manager into converting that "why" into one you would accept?

When dealing with clients, ASAP = Rush Rate. If it’s really needed that fast, they will pay extra for it. If it’s not, they will wait for the timeline that you can hit.

All that said, client work is the one place where sprints serve a real purpose. The scope doesn’t expand because you finish early so there is a built in reward, agreed upon price for the scoped work, and a built in penalty.

My issue with sprints is not exactly the rhythm itself. It's the sabotage of continuous delivery and operation (DevOps in the original idea).

Yes, this also. In my organization, we have a problem with "just ship something" because the cost of making a release is so high. We know this, so rather than ever "just shipping this part that's ready" we often drag projects out longer so we can "be really done with them."

Only sometimes, we could really have been done with them, without necessarily finishing all of that pet feature which has thrown off the whole timeline, by being more complicated to implement than anyone was able to imagine upfront.

I’m not sure I understand how they interact. My teams have always continued to practice continuous delivery totally independently of sprint boundaries - that is, released were totally independent of sprints.

Yes, that's how it should be and basically it sounds like you are then already doing a Kanban style working mode (and what purpose does the sprint rhythm than serve)?

Scrum by the books usually implies that releases happen after the po or the customer has accepted the increment at the sprint review.

> (and what purpose does the sprint rhythm than serve)

I find a good sprint rhythm stops the constant task shuffle. When business knows the dev team is mid-sprint it forces them to think about priority.

We do CI/CD, and only big items (in reality UI/UX changes) wait until a sprint review to go out.

Don’t run sprints if you don’t need them - that’s the answer.

But honestly, I’m not sure what the problem is. I’ve found they they’re perfectly suitable at providing:

- an opportunity to check in with a wider team and demonstrate features.

- the chance for the development team to periodically review progress and identify any recurring issues.

- the chance to plan upcoming work and identify any external tasks that need to be taken care of.

Is the problem maybe that some organisations are over-using the “commitment” aspect as a metric? To my teams, “commitment” has generally meant “what we have agreed among ourselves we will be able to deliver in a unit period”. The metrics are purely internal, and useful for allowing us to see if we frequently over- or under-estimate how much work we can do; the sprint is really just the period of time across which we measure and refine.

> Don’t run sprints if you don’t need them - that’s the answer.

Except the people being forced into the sprint aren't the ones with power to opt out. It has always been my experience that there is a "product owner" or agile coach or some other title who makes the decision, and isn't one that has to live inside the system they create.

So I agree with you in theory. Some people thrive in the sprint, others suffer and struggle. If everyone could opt in or out, it would be great. But in practice I've never seen the ability to opt out.

You are experiencing the fake 'agile'. Where in the agile manifesto do you see anything about mandatory sprints? See, you are doing a forced process but some sleazy management is calling it agile because the word once got popular among software devs.

The universal answer to problems with agile is that the complainer is not doing agile correctly. This may even be true. It may still represent a problem.

The usual response to that is to read the agile manifesto, read a bit of history of it and watch some videos on youtube from the people involved in its beginnings. You should then discern the original agile movement values from the salesmansy bullcrap that the word agile turned into.

The problem is “sprints” need to alternate with cooldowns. When are the cooldowns?

Nobody sprints non-stop, that’s a marathon.

If they were called something like “timeboxes” you get all your bullets without the demoralizaing notion of inability to maintain an unsustainable pace.

Right - the plan in agile is to spend your entire work life madly 'sprinting' at some eternally moving finish line. You can't sprint non-stop. It's madness.

I don't think anybody is arguing against you. "Sprint" is just a terminology, maybe not the best one but it's not like you're supposed to "maintain an unsustainable pace". Working after hours would be undermining yourself in Scrum for exemple (your daily speed would now be supposing you do overtime)

"The metrics are purely internal"

bwhahaha, good one!

Well, I'm describing how my teams have worked, and it's true!

I'd be even stronger on that point – not only should metrics probably not be available outside of the team, but they are even completely without meaning if they're used outside the context of the team. Maybe one of the biggest problems people experience with Scrum or Agile generally is the inappropriate use of metrics.

I really disagree with this.

Sprint objectives are by no means arbitrary or meaningless - or any more so than other deadlines - they have value to the extent they are intelligently formulated.

Sprints have a very important operational dynamic, in that they set a fairly clear, achievable, short term goal.

Software is always a pandora's box, and left to their own devices, a lot of devs could work on the smallest thing ... forever. We are all guilty of this.

Long term milestones (i.e major product iteration) are hard to internalize, they are vague to each individual even if they have meaning at scale.

Sprints provide the means for large things to be broken down into achievable pieces, which much more strongly enable small teams and individuals to focus on that specific outcomes.

Maybe they shouldn't be called 'sprints' maybe just 'mini milestones' or something ... but they are useful. Or 'waypoints'.

Getting lost in a sea of software is very easy ... so setting 'waypoints' I feel is very, very useful.

> Sprints have a very important operational dynamic, in that they set a fairly clear, achievable, short term goal.

Lately, the trend with those using Jira in large organizations is to just "carry-over" what's not done in the current sprint into the next. And so instead of talking about why something didn't get completed, and how to resolve it quickly (move it to someone else, change the way it's done, postpone it until something else makes it easier), the work snowballs, with even more work getting piled onto the sprint. That's because while the dev team is agile, management deadlines often won't budge.

The other problem is dependency upon another process/service internal to the organization that is not on your sprint cycle. Typically they have other, different priorities, which don't match yours. So they may get around to it eventually, which might mean a week to a month typically.

I think Kanban is in someways vastly superior in these cases. You can still do "waypoints" at say every two weeks to see how things are moving along, but at some point if your company isn't "agile" then it really makes no difference if your team is.

Checking in is useful. The problem from sprints comes when each one is treated as a deadline instead of just a point to check in, evaluate new information, potentially pivot.

When they are treated as deadlines, the problems come. And there is no reason you can simply forgo sprints entirely...and still check in, see how things are going, pivot, evaluate new information.

To this point in my career, I’ve never seen sprints present when they didn’t go down the destructive path for teams.

"And there is no reason you can’t simply forgo sprints entirely...and still check in, see how things are going, pivot, evaluate new information."

Because 'checking in' on something doesn't create incentive to completion, or actual material outcomes.

You can have 'the software developers doing stuff and meeting objectives on their terms' - or have deadline driven wherein business/software objectives are set and people move towards them.

The problem with the objection to 'deadlines' is the fact that inherently software is not created for it's own aesthetic sake, or for it's cleanliness, or architectural integrity, or some arbitrary measure of quality.

'Time' is the key ingredient often missing from the architectural/development equation, because 'time is money' both in cost, and opportunity cost.

'Time' (which implies cost, ROI etc.) will likely be driven as a business impetus and has to be put into the equation in very serious terms.

Many devs might complain about 'technical debt' or 'messy software' or 'some things that don't work' - but in reality, those situations may actually represent optimal business scenarios.

It may not make sense at all to make certain systems perfect, or at least, in the short run or on first iteration.

In other terms, the 'QoS' expected is I think always way higher on the dev side than real business terms may mandate. What devs view as 'a mess' may be a perfectly good condition.

'Building the wrong thing' is the #1 reason that companies fail, not 'technical debt'.

If customer outcomes are generally good, revenues are increasing, then technical debt is normal, and it's a function of pragmatism at that point more than anything.

Taking too long to build something, or taking a long time to build something that is 'the wrong thing', is almost guaranteed death.

Finally ... I think milestones with deadlines are important also for the psychological and organizational factors.

In NATO, for examples, all 'orders formats' are the same. Any NATO General, from Polish to American, gives a set of orders that are crudely similar to those given by section commanders to their privates. That's to enable operational efficiency. The 'most important' part of the orders are the mission objective with a timing. Literally it has to be said twice. I'm not kidding - while giving orders, the mission statement is verbally repeated. Always. This is because in the fog of war 'everything goes wrong', nothing goes according to plan, things break, people die, obstacles abound, opportunities arise (just like at a startup) - but if everyone involved knows what they need to do, then they can adapt and make it happen by hook or by crook. So 'the gap at grid reference 0000 1111 needs to be bridged by 0600h tomorrow so 6th calvary can cross with heavy tanks' is a mission. Everything else about the plan is secondary, everything could go wrong, but when individual sappers/soldiers, NCO's, junior officers all have that 'this spot has to be gapped by 0600 for heavy vehicles' ... then they can rally around it whatever happens.

Obviously there are tradeoffs and obviously devs have to participate in the establishment of their own deadlines and objectives ... but in my view it has to be done, and it's healthy to break it down.

There are extremes that sway a conversation.

If you have a dev team with no incentive for completion, you have a dev team problem.

If you have a hold up to deliver perfection instead of what the business needs, you have a scope creep problem.

Neither of those are solved by sprints but they are often used to justify them.

"If you have a dev team with no incentive for completion, you have a dev team problem."

Devs naturally lack incentive - it's an existential nature of software, i.e. it can go on forever. It's not like building a house.

"If you have a hold up to deliver perfection instead of what the business needs, you have a scope creep problem"

A 'Quality' issue problem, not so much scope, but again - there's a natural dissonance between Engineering instinct, and outcomes. Engineers naturally want to build something robust, clean, and performant. It always makes sense to them, pushing against that is always hard.

Ergo - the need for clear milestones - with timings.

>Devs naturally lack incentive - it's an existential nature of software, i.e. it can go on forever. It's not like building a house.

But we're talking about adult humans who have agreed to a contract to fulfill business needs in exchange for money, I tend to presume that its on the developer to guarantee their end of the contract, otherwise business won't fulfull there's and I further presume business gets to decide what the criteria are. If things are otherwise, which this seems to be suggesting, then that would be an egregious business error, keeping on developers that aren't delivering anything, and if your tech lead isn't giving you honest assessments you let them go too. This looks the business problem is being coupled into the warrant behind sprints being key cause devs "naturally lack incentive".

I also don't know how to agree with developers "naturally" lacking incentive, given they could lose their job, but I've definitely met developers with little motivation. That level of motivation tends to be tied to wider organizational issues as well...

"who have agreed to a contract to fulfill business needs in exchange for money, "

Technically, that's true, but in reality not.

Devs go to work, build stuff, and get paid every few weeks.

Their direction really is oriented only towards the goals they are given.

Business needs are often counterintuitive to the natural position of a developer.

The reason I believe this, is because I'm a developer who moved into the business world. I see this utterly and fundamentally different now. Even how I approach writing code is different.

Business needs are very difficult to understand for anyone who has not sat in a true business role: customer facing, selling i.e. direct sales etc.. A few people have a 'natural born' business instinct, but it's rare.

Even people in regular marketing jobs (i.e. not marketing ops, like branding, creative) I find don't have an instinct for it.

I like specific goals, that are attached to timings. They're not always going to be made, but both of those things are important.

Developers moving in the wrong direction is more common in software than not.

It’s still a scope creep problem. The deliverable is the milestone.

There is a big difference between infinite knob-tuning and solving the problem in a manner that meets needs.

It’s very much a scope creep problem.

Breaking large tasks down is incredibly difficult, and often dismissed as though it were easy. In fact, that's really the main challenge in engineering.

It's easy to think something is a step forward, work on it, then realize it's a bad idea later.

If you know what the individual, bite-sized tasks really are, you are nearly done with the project.

" really the main challenge in engineering"

It's a mostly a business problem. Not mostly an engineering problem.

Cost, product issues, customer issues, support issues, IP issues, margins, positioning, partnerships, finance, M&A, staffing - these are all part of the equation. Eng is only one piece.

I hate the term "sprint". The word gives an impression of desperation and short term thinking. You don't sprint when you're running a marathon; it's not sustainable. It's a superficial complaint, but names do matter.

Let's rename it to 'checkpoint' then. Even in a marathon a runner is checking their time to where they are every X miles. The runner then adjusts their pace if they have a goal time. In software a checkpoint should result in the same behavior.

Just change the name to 'iteration'. Removes the speed/intensity analogy and reinforces continuous improvement.

I came to a similar conclusion and prefer Kanban to Scrum.

If there are not enough tasks you pull from backlog, if it's too much then they get finished in next sprint. Work never ends, so why the need for extra overhead to group it into 2 weeks?

I always see it as a "marathon", constant sprinting is no good long term.

However, Kanban works as long the team knows priorities and can choose between good and over kill solution, which is easier said than done. Working in a good pace without a deadline is challenging itself, since you think you have all the time in the world.

Sprints are often misused and misinterpreted. They are not hard deadlines. A deadline is when there's going to be a real problem when you don't make it. When a new campaign, shop, whatever opens, the corresponding website needs to be there. That's a deadline.

A sprint goal is meant to provide focus. You've got something to work towards, but if you don't make it, you don't make it. Presumably there was a good reason you didn't make it. It's good to examine that reason to see if that reason can be taken away, or otherwise it can be taken into account in planning. But it's not the end of the world if you don't complete a sprint on time, and you should certainly not work late to complete the sprint.

I've worked in very successful teams that never finished a sprint on time. I think that's totally fine. It's not about the deadline, it's about the focus.

Fair, but sprints are effectively what have forced a lot of software dev organizations into actually thinking about making tasks bite-sized....which was one of the problems (both conceptually and logistically) about waterfall.

The lack of feedback breaks classic waterfall. You get a plan, and the plan is assumed to be correct.

Of course it never is. As mistakes are revealed, unexpected challenges appear, and requirements are updated or clarified, the plan unravels instead of adapting.

The only real benefit of Agile is that it should - in theory - allow the plan to be modified, so projects can be steered towards a target instead of being treated as fire-and-forget. (And miss.)

Task granularity should not be a specific benefit - it should fall naturally out of competent goal-based planning in whatever paradigm is being used.

I've never see a waterfall process that did not have feed back.

But not all tasks can be made bite-sized. Or more specifically, sometimes the bite-sized pieces aren't "complete enough" to be a valid unit of work to be nicely wrapped up and finished during a sprint.

What happens as a result, is that more major tasks never get sufficient priority to actually happen. This can easily lead to an incremental "refuctoring" of the code base, where you have a lot of individually-nice bite-sized tasks that, in aggregate, turn the source tree into a total mess.

"Planning better" is a hugely valuable thing, if you can pull it off. You'll end up with less pressure, happier stakeholders, and less frustration with your own output. Even in a purely selfish better-for-the-team way: an accurate plan and capacity projection (relatively - if you have bosses who expect perfection here, that'll bit you) makes a great input into decisions about headcount, hiring, and project allocation. If you can't plan well enough to know you need to ask for help, you're screwed.

Agile as I've seen it in the wild is easy to twist into "plan more verbosely" instead of "plan better," though.

The most recent way I've seen this manifest is coming into a team that planned out each ticket in great detail when it came to execution - gotta maximize that committed/completed ratio - but invested very little in making sure they were planning the right thing. Someone would ask for something, it would become a ticket. But whether or not that was the right thing short-/medium-/long-term? That's a conversation that didn't have anywhere to happen.

Another is confusing "producing more estimation artifacts" (subtasks, hour allocation, etc) with "investing the right amount of time in planning." Going from a 30 second story point estimate to a 2 minute subtask breakdown in sprint planning isn't what it takes to make an accurate estimate for a project.

Sounds like management is using sprints to enforce deadlines and releases. Our team uses them to improve our ability to estimate, which in turn requires us to think each set of technical requirements through and anticipate issues before we start pounding keyboards.

We use them to structure our work, but decoupled them from releases. The Dev team owns the sprints, but the business owns releases. DevOps uses Kanban however, because it works better for them. That makes us more flexible. Agile.

It's been a years-long process but I'm really happy with our "Agility" these days. Our sponsor is the VP of Tech and he gets it (used to be our QA lead). If we have rollover we talk briefly about estimating better but it's not the end of the world, because we actually have gotten better at estimating. Some sprints we nail it. Other times we miss, but management isn't using those metrics as a bludgeon so it actually helps us to work smarter.

The problem is not agile, it is an all-alige approach. It is at least in our case. Sprints are feature-focused, which leaves little time for software development craftsmanship: reducing technical debt, updating outdated dependencies etc.

To give some context: We have multiple pieces of software we build and maintain for multiple clients. When a client requests some features, they buy some sprints and we do the work and then it's done. During the life cycle of the software there is a maintenance contract for service and support and resolving bugs. This does not have much room for software craftsmanship either.

This is just where I work,but I can't imagine we are the only one working like this. This is an implementation problem, not an methodical problem.

One of the ultimate causes is that our org is project based, so no project, no work for that client. And the software quality suffers IMHO.

_This_. I've been advocating Kanban for years. Why try to adhere to rules which you don't need at all instead of having a non-restrictive framework which is adaptable?

> Sprint are the root cause.

I disagree. there's nothing inherently wrong with short iterations or a predictable release window.

trying to do agile with fixed budget, fixed requirements and a fixed timeline is the core issue.

most of the problems with agile, including "devs having to do this much tasks or die" within each iteration, comes from trying agile on an inflexible plan.

What's wrong is to have a team wide iteration xycle instead of iterating within features / use cases

Why? Fixed team global sync points sound great to me. Fixed interval measurement windows also help with correct burndown estimates.

Also 'move fast, and break things'. Without understanding what you are supposed to do.

You move a lot without going no where, with a lot of broken things.

This always happens with promising technologies or methodologies once big companies get their hands on them. OOP was a great idea until it turned into a strict religion, Agile was good until it turned into a micromanagement system, the internet liberated people until it turned into a surveillance machine, collaboration is good so we got a workplace with open offices that's so distracting that it's hard to get work done. Watch the same happen with FP or things like diversity.

Companies are basically dictatorships that want to control their employees ("resources") and everything else so it's pretty natural that they will distort every new movement into a tool to extend that control.

I think it's easy to look at the short-term and miss the larger picture.

The printing press being the only other well documented time of this explosion of knowledge/technology allows us to see that no matter how power expands and then coalesces, it takes the good ideas, fragments away, and then repeats the process as a sect of the greater power.

I think the problem with Agile as implemented by larger corporations is not anything described in the article, but that it’s just fundamentally antithetical for corporations to work agile.

There is just no way that some salesperson is going to be able to sell a product that’s going to be developed in collaboration with a customer in 2 week increments (unless the customer is already specifically dealing with a software consultancy).

They sell a package of ‘this is everything the product is going to do’ and then somehow try to squash that waterfall list of features into ‘stories’, and call it agile.

In agile, it’s the salesperson that’s the customer, not the actual “customer”. Unless you’re doing professional services or something agile is an internal matter. As with many things the less your customer knows about your inner machinations the better!

That's just factually false: to give a counter-example my company (~200 people) does exactly that (1 week sprints, though).

Except that we don't sell "a product", we sell time. We'll do everything we can to help you getting a product out of this time, but if you can't, you were doomed from the beginning.

It's just a different thing to sell for a salesperson, they have to adapt. But it works.

E.g. the one exception I made in my message, a software consultancy.

It's also the problem that software is not strictly a product, but a mix of service and product. There's the core product that you are selling (say e-commerce) and there's the customization service that's been sold to increment the product per-customer basis.

Traditional sales knows how to sell either a product or a service, hardly the mix of them, which makes software special (but not unique).

Isn't this a problem with the sales team not being able to sale the current product?

If there is a current product. I think they’re generally trying to sell a future ‘better’ product.

I never saw agile as anything other than some useful observations about project management. Get feedback fast, sync with team members often, don't try to plan the whole thing in one go.

Any particular incarnation of it, say Scrum, is going to have issues when applied to real world problems. Guidelines like "get quick feedback" are useful, but decrees like "one week sprints" can get tyrannical.

Luckily a lot of people seem to understand that and I've never met anyone who insists on doing exactly according to some specific scripture.

To me, I think everyone took the processes WAY too literally which caused the whole big "A" Agile versus little "a" agile. I have seen many orgs get super wrapped up in all the typical nonsense of stories "falling over", religion about who owns the backlog which takes away autonomy of the team, religion about the process for breaking a sprint as new discoveries are made, arguing over product work versus addressing tech debt or working on automation, etc. You end up focusing more on "being agile" than delivering new features.

I 100% agree with the other comment around "checkin" points so that work doesn't linger forever, but the forced short delivery boundaries very likely slow teams down overall especially when all the process overhead of points, backlog grooming, standups, sprint planning, demo session, etc are all factored in and mandatory every 2 weeks.

Commit to some scope, on some cadence/date, loosely check in on progress, and adjust as needed as a team. Deliver as frequently as possible and demo your work when you can/should.

This sounds pretty accurate to what I’ve witnessed.

I’ve seen Agile done well, which I really liked. Mostly what I see is water scrum.

It’s important to pack every single thing you can into a sprint based on your estimate of points, otherwise you’re wasting precious resources. Of course that means you never get everything done so huge chunk of the sprint always rules over.

Everything is dictated from the top, there is no autonomy on the part of the team (or almost none).

As bad as all this is, a quickly falls apart whenever something doesn’t go exactly the plan. In other words, constantly. So you just get further behind and more things roll over into the next sprint.

They’re “agile”. They hold demos of the tiny bit that got done and congratulate themselves. Because they don’t do waterfall anymore. They GET IT.

But none of the benefits. Because that would require actual change.

The problem, obviously, is not Agile.

This will, naturally, cause hairs on necks to rise waiting to scream back "What are you talking about? Of course Agile is the problem!" But really, the problem is that any business trend/fad is inevitably doomed to become a hollow shell of what it was intended to be.

Or more pointedly: Businesses don't really want to overhaul their development process, and any new process integrated will inevitably become just an extension of whatever they were doing before. This is basically what the article's saying without directly saying it.

I mean, the very first phrase in the Agile manifesto is:

> Individuals and Interactions over processes and tools

And yet see how many anecdotes in this thread are about how the most annoying part of Agile is the rigid adherence to certain processes. If 1 week sprints are too short, change them to 4 week sprints. Figure out the cadence that makes sense for your team. That's what Agile is supposed to be about.

Regardless of whatever Large Co wants to call their development process, let's stop calling it Agile and start calling it what it really is: Waterfall with a Kanban board and shorter deadlines.

> The problem, obviously, is not Agile.

Sure, any true Scotsman can see that.

What reason do we have to believe that Agile is so great, if (as you point out) most business aren't actually doing Agile?

>Sure, any true Scotsman can see that.

I call BS. Yes, the "any true Scotsman" fallacy can happen in a discussion.

But it's also true that some things are presented as X but are not X, and it's not a fallacy to point this out.

Here the parent specifically called attributes of Agile that are there in the original manifesto, but are not adhered.

That's not shifting the definition (which is what a no true Scotsman is about), it's about pointing that a thing doesn't adhere to the original definition it was supposed to hold.

>What reason do we have to believe that Agile is so great, if (as you point out) most business aren't actually doing Agile?

That's an orthogonal question. Agile might be great or bad.

But we can't deduce from businesses doing something non-agile and calling it Agile whether agile is good or bad.

At best we can deduce that Agile has a tendency to be wrongly applied.

> What reason do we have to believe that Agile is so great, if (as you point out) most business aren't actually doing Agile?

Read the anecdotes on the other end. People who have reached the shining city the article describes. It's a nice place to be. There are a few in this thread alone. I've personally experienced it as well. It's great.

> Sure, any true Scotsman can see that.

To continue the analogy, this is roughly the equivalent of declaring myself Scottish because I'm wearing a kilt and playing bagpipes. I have no connection to the country of Scotland presently or historically, but I'm doing the things that Scottish people are stereotypically associated with, so I must be Scottish, right?

At that point, I think it's fair to call my bluff.

> Individuals and Interactions over processes and tools

Indeed, it's my experience this is often ignored, along with "ditch what doesn't work for you". I've seen countless articles (and job descriptions!) claiming, for example, that "pair programming" is the sine qua non of Agile; if you don't do it, you're doing something wrong. Or "sure, use what works for you, except if you don't follow these guidelines you are on your own".

I, for example, consider pair programming extremely exhausting. It works for me in very short spans and very limited applications; pushed too far it makes me consider switching jobs.

Isn't agile based on the idea that engineers are fundamentally interchangeable, and so doesn't everyone involved need to establish some fairly significant cognitive dissonance in order to implement agile "correctly"?

It's fairly obvious that engineers are not fungible, and that an entire process centered around that idea, is not only degrading and disrespectful, but an outright lie that needs to be propped up by illogical thinking, and faith-based arguments.

I'm not sure how you determine what is true, but using empirical evidence and sound reasoning is how I prefer to be presented with justifications for a process change. So far, I haven't seen any research showing that agile is an effective management system.

> Isn't agile based on the idea that engineers are fundamentally interchangeable,

No, “people and interactions over processes and tools” is based on exactly the opposite.

OTOH, most concrete methodologies sold as “Agile” do seem grounded in the interchangeability of workers (not just engineers, though those are the primary workers involved), but the idea of concrete development methodology (instead of metamethodology that specified how a group adapts dev. methodology to group needs) usable “as is” by multiple organizations is directly contrary to the Agile Manifesto.

Doesn't “people and interactions over processes and tools” seem a little bit like fox news trying to sell itself as "fair and balanced" or those multi-level marketing campaigns like "it works!"

The core premise seems like it's pushing a human-centric idea really hard only because it actually removes the humanity from it.

Getting hounded every day to try to justify your existence in a company does not sound human-centric to me.

> Doesn't “people and interactions over processes and tools” seem a little bit like fox news trying to sell itself as "fair and balanced"

No, because the Agile Manifesto isn't a slogan for a concrete methodology, it is the explanation of a set of values.

The concrete methodologies come from different sources than (and post-date) the Manifesto; the Manifesto wasn't written to sell them.

It's true that there is a mismatch between the values articulated in the Agile Manifesto and the concrete methodologies that bill themselves as “Agile”, but that's not due to the Manifesto being a dishonest sales pitch for the methodologies, it's because the methodologies are dishonestly appropriating the label associated with the Manifesto.

I agree there's a mismatch between the Agile Manifesto and most of the concrete methods described as "Agile", especially when it comes to how much individuality is valued. I am not at all convinced it's fair to blame this entirely on the methodology side. The eXtreme Programming book was published in 1999 (and had been talked about before then). Scrum is a little harder to pin down, but a book was published in 2001 (around the same time as the Agile Manifesto), but the ideas were definitely circulating before that. There were other concrete methodologies which fed in, _e.g._ Alistair Cockburn's "Crystal" methods, which seem to have faded away.

In any case, the people pushing the methodologies overlap very substantially with the people who originally signed the manifesto.

But it's not really the methodologies themselves as the approach to methodology that is problematic, and of the major “Agile” methodologies, only Scrum really bakes the inflexibility in to the first party approach to the methodology (if you aren't following every jot and tittle of The Scrum Guide™ under the guidance of a certified Professional Scrum Master™, you aren't really doing Scrum.)

Most “Agile” methods are fine starting points for adapting a methodology to your team needs, which is the only thing a published methodology should be if one takes the values of the Agile Manifesto seriously. But the even where they aren't actively pushed as rigid, one-size fits all, certification and standardization driven monoliths like Scrum, they tend to fail to actually address Agile application...

Scrum isn't the only offender. By-the-book XP is pretty rigid about some things, too (collective ownership and pairing in particular).

Agree that the consulting/certification industry makes things worse. Again, that overlaps with the signatories for the original manifesto.

Scrum and DSDM significantly predate the manifesto with Ken/Jeff and Arie all signatories.

So true. But then, Agile solved problems developers had, not project managers, sales people, or accountants. In fact, it caused those people more problems.

It was only ever going to be actually implemented in developer-run organisations, because only then are developer problems more important than everyone else's problems.

Maybe that's the answer - more developer-run organisations.

> Maybe that's the answer - more developer-run organisations.

I don't agree. Once you put a developer in a project management role, he starts to experience problems associated with managing projects and using solutions to those problems.

When that happens, the developer ceases to be a developer and becomes quite plainly a project manager.

The whole idea of a sprint commitment, of pointing, of measuring velocity, of a sprint demo, is in deference to the business need to plan and adjust priorities. It does them based on realities, whereas older more waterfall style methods tend to due them on hypotheticals, decisions made with the least amount of information, and then the business wonders why the heck everything is late.

I think agile is usually presented as an approach that tries to reduce a split between the business and the developers.

If you take the business out of the equation you usually get development models that just-for-fun open source projects have.

Waterfall is cool with customers and sales, at least it was if it wasn't risky. So agile is a risk minimation strategy.

I was once on a project where the managers loved it, because it gave them the ability to micro-manage a project like never before. They got daily status meetings, and bi-weekly cycles to adjust things.

And finding developer-run customers. I wonder how many such customers exist.

Some Buddhist wisdom:

In life, we will always find “shining cities” to chase. The perfect house, the perfect job, the perfect romantic partner, the perfect software team.

But, like a mirage, when we reach these goals it seems that the thing we thought was perfect is actually riddled with flaws.

“The shining city” is purely a mental construct, one that gives us structure and purpose, staving off “existential nausea” - the anxiety that comes from living a directionless existence.

The way to everlasting peace is to be able to acknowledge: “we live in the shining city right now. Everything is operating according to the laws of physics, humans follow human nature and their genetics. Everything is exactly how it should be.”

But this isn’t “giving up”. It’s ok for us to have “shining city goals”. It’s human nature. It’s fun and allows us to experience a journey.

But at the end of the day when you see the mirage fade, smile about the journey and the experiences you had along the way.

The way to everlasting peace is to be able to acknowledge: “we live in the shining city right now. Everything is operating according to the laws of physics, humans follow human nature and their genetics. Everything is exactly how it should be.”

This is not wisdom, it’s nonsense.

If we followed this advice 2000 years ago we would still be practicing open defecation and dying at the age of 30.

I bet if you measured contentment levels then and now you would not find much of a difference! In fact you might find people are less content now than shitting in a ditch 2000 years ago.

But I did not mean this as an excuse for “non-progress”. Humans will “progress”, it’s their natural instinct and drive. It makes more sense to make our lives easier than to make it harder.

But the shining city doesn’t exist, and there’s nothing that will significantly increase our internal contentment but winning the internal battle of acceptance.

It is much more nuanced. Peace is about: accepting and appreciating what you have, and don't have (i.e appreciate the actual reality). Progress is making it better.

You will need both, change and the ability to see what is good and works in the current reality.

Yes 2000 years ago, when people were defecating in the open and dying at 30, things were just as they should be, since it brought us to our current state where we have sanitation and live to 100.

Maybe that's where it should have stayed...?

On top of this, not everyone's "shining city" is the same. At a minimum, the "shining city" mentioned by the author was apparently incompatible with management's "shining city".

I've yet to see anyone actually do Agile where the tasks were broken up into reasonable increments. All of my Agile experiences involve a project manager throwing a sentence up on the board for what they want done and saying "yeah I think it'll fit in this sprint" for something which no one has any idea how long it'll take because the scope is unknown and no preliminary investigation has been done (a process which would take maybe a week).

"Task estimation" for sprints never seems to involve any actual data on which to base the estimates.

I’ve experienced scrum done “right”. Company I was with made it a strategic thing and set aside time and money for the org to make the transition. It actually worked quite well in the end but it was a lot of work. It’s an outlier though - I’ve never come across a company before or since that would go to this much effort.

Usually the conversation with management about implementing agile ends at the point where they realise the upfront costs.

There is a concept of "spike" work. Basically, when you do not know what it takes to implement something, you work on a "spike", the result of which will be new stories better describing what needs to be done.

What happens when the spike is done early in the sprint. Do you just hold off a week or so until you can report back? And who works on those follow up tickets? The person who did the research? They've had the most experience, so that makes sense to me, but I've been told everyone should be able to pick and choose, and if I don't grab the tickets fast enough, I'm sol.

Part of the spike should be documenting the results so that anyone else can use them if there are the person who picks up the next ticket.

If the person who does it just remembers it themselves and doesn’t do anything to help the other developers then they haven’t completed the spike.

There’s nothing worse than having to execute someone else’s minute breakdown of work. People should be responsible for executing their own ideas. Then you get feedback on what was and wasn’t correct, and have only yourself to blame.

Having that ownership stimulates a much greater level of craftsmanship and skill growth. Documentation is great, but the idea that a spike author and implementor should be different people is awful.

The spike doesn’t define HOW the later work is done, only the information it was supposed to research.

How to do something would be sorted out during grooming or perhaps when adding the story to the sprint.

> Part of the spike should be documenting the results so that anyone else can use them

Do you have other people on the team review this? Or do you just trust that it's complete?

Sorry for all the questions, I just feel like these are things that are never answered adequately.

I would think it would be presented/discussed with the team.

I’m not sure what I’m used to at my job is correct, so I’m trying to stick to my understanding of how I think it’s supposed to work.

Agile always fails, because bookkeeping and financial processing in companies are not agile. Contracts with customers are also not written by sales with agility in mind. So in case time/money runs out agile processes and rules (like "never break a sprint" etc.) will always be ignored, because finance needs the numbers and the customer was promised a fixed feature set at a specific date for a specific price.

mandatory Dilbert Comic: https://www.flickr.com/photos/lucaminudel/6059269914/

I don't think that's a real comic. Looks like the Dilbert site used to have a feature to modify / make your own comics, but I can't find it anymore.

Do devs really hate agile? I’ve really enjoyed it in workplaces despite it not always being perfect. I’ve never felt micro managed either, just regularly managed. I understand that maybe some want more free will to pursue problems the way they feel is right, but it’s a job at the end of the day and you are a human resource being paid to deliver results. Make sure you’re being compensated enough, at least to not feel like you’re being exploited.

Absolutely. We adopted 'Agile' a little over a year ago and the perverse incentives carried with it brought a markable decline to the quality of the software being produced. We've made strides in righting that ship by returning to proper software development planning methodologies to build good software again (the results we're being paid for), but at the expense of angry PMs who don't see it as fitting their Agile system, which makes for an unhappy work environment.

Shame too, as this used to be a really great workplace.

The part I really hate about Agile is how it almost encourages "sloppiness" in requirements (execution too sometimes, but requirements is the part that really drives me crazy).

You need to always be "sprinting". There is no time to take a step back and think deeply about the product you are trying to build. No time to think through what a new feature means for your customer and for your existing system.

How often do you work on tickets that are completely ironed out? All the actors of the feature are described, all the interactions with their outcomes, all the things to test for, all the new metrics to add to the system?

And don't hear me wrong. It's probably the right MO for small companies/products. But once you've reached a certain size for the company, the system and the user base, it's not gonna work. People often laugh at how long it takes for Twitter to introduce a minuscule change in their product, but seriously, I wish a lot of "agile" companies would apply this extreme instead of the completely opposite side of the spectrum: "just start coding this shit, and we'll figure it out later".

I hate it. Constantly being pressured to move some stupid card or add some stupid tag so people’s graphs look nice. I just want to get work done, Scrum just adds overhead.

How much overhead is it really? 15 minutes a day at most?

Not if you have the misfortune to be a single developer project or having a lead role on a team.

Someone needs to convert those Jira/Trello/Wall/... items into nice Excel sheets because upper management won't look at anything else.

While you are at it including some nice Powerpoints for the weekly/monthly updates.

And on the remaining time do those development tasks, while coaching junior devs at the same time.

If you have a lead role on a team the large majority of your time should be leading the team by doing all those non-development tasks. If you're trying to also cram in mission critical development you're trying to do too much or your company's leadership sucks. If it's a single developer project, 1) Agile might not be the best approach, and 2) updates to management should take very little time since you just need to sum up what you personally have accomplished and are planning to accomplish vs. whatever deadlines exist.

That's the nice theory and then there are the actual real life projects, hence why we reached the point discussed on the linked article.

Yes, that is the nice theory and also the nice real world. And yes, I've been on and managed dozens of projects at 10+ companies over the last decade so I feel confident saying that. Sounds to me like your company's leadership just sucks. That's not a problem with the methodology.

yup ! just let me solve interesting problems... stupid tickets ! lol and then you get plugins like "everyhour" so you know.. they can track... EVERYHOUR !!

The advantage I do see for Agile is in shops where management is justified in their belief that developers are being over-cautious as well as too relaxed - this is exactly the atmosphere in many low-throughout software organizations I’ve seen in lots of old, non-software companies. If management is fine with lowering quality in favor of more shinies, Agile is pretty alright - it is Worse is Better codified and institutionalized. If the codebase is so bad it is impeding your ability to deliver business results, you do have justification, but time boxing efforts is important to avoid going down rabbit holes of endless refactoring.

I think most of the developers that are hating Agile are probably more disgruntled with their organizations and their culture (usually hostile to high quality engineering in most places fundamentally) than Agile itself. In a way, I think Agile (similar to a lot of automation efforts in infrastructure / IT processes I’ve worked on) simply brings out the true nature and culture of who is driving, who pretends to drive, and where things are falling apart in team dynamics. The software is usually a casualty of bad dynamics is the implication, not bad team members, but we all know this is not true.

I think the #1 thing that is missed is that the implementing team should have more control over what they work on. Someone writing a bunch of tickets and pointing them out alone reduces team ownership / responsibility and is pretty much how traditional Taylorist work models are built. Agile is fundamentally much more in line with Deming’s ideas giving much more control to the implementors and that is antithetical to at least 80%+ of businesses in the West and probably Asia too.


That’s a gross oversimplification.

I’m a competent, experienced developer. I’be worked in agile, waterfall and “faux-agile” teams. The agile ones have been the best - they adapt quickly to change, have good estimates of progress, and keep all of the team members and wider stakeholders involved in the project.

If there are useless meetings in your development team, remove them. If you can’t remove them, then your team is being mismanaged and no form of development methodology will fix that.

And also, bear in mind that even among developers on a team there are different sets of skills and preferred working patterns. You may not need some interactions, while other team members benefit from them; remember that the useful measure is the amount of work the team as a whole can achieve!

My experience is that it is more likely that "hero devs" who love to do what they think is important rather than what the team needs will hate any attempt to reign them in.

A big insight for me about where Agile seems to work and where it doesn't was learning about the Cynefin framework [1], which tries to characterize different states to guide the style of organizational decision making.

When viewed through the lens of Cynefin, Agile seems to make sense when you've narrowed down a problem space you want to tackle, but you still require rapid iteration and experimentation to finish defining all the finer points of the goal (the liminal state when moving from Complex to Complicated).

My impression and personal experience of Agile being a bad fit seems to be associated with a misunderstanding of the decision domain, and the general Agile "religion" that has arisen where it's applied uniformly to every situation.

[1] http://cognitive-edge.com/videos/cynefin-framework-introduct...

Usually the claim is that if Agile is not working for you, you're 'doing it wrong'. This article is no different.

People who work in sick environments will not be saved by Agile. These techniques are meant for teams that already have members who trust each other, in a company that trusts the team, to work with customers they know, who would like a usable product that they can develop and extend the requirements for as they learn more about how it will work.

It helps prevent the sicknesses that come from attempting to do all this up front, where an eager team and an innovative customer try to specify everything up front and nail down a price tag right away.

It should be sold to management as a form of risk management, not as a way to extract more value from a fixed amount of dollars. If you have $4 million to spend, and you spend that up front, you may waste $4 million. If you spend it bit by bit, and have a functional system after every few weeks, that you can migrate to if you like, then you have not wasted any money, even if you end up throwing it away partway through.

I agree with anyone who says the problem with Agile is that it is watered down. It was a least-common-denominator of all the various similar techniques, and I think a lot was lost by encouraging people to adopt Scrum instead of eXtreme Programming.

Nevertheless, I've consulted on a lot of projects and there are environments in which I would not want to attempt to implement any new methodology. The people already hate each other and the system is totally sick. They need something more like Deming's management techniques where they get the top to commit to empowering the bottom, retrain and move out people who are in the wrong job, and get a remaining nucleus of players who trust each other to implement a new method (or their own).

What ends up happening is programmers come into sick environments, run by "Agile" managers, and say that Agile is the problem and if they could just go back to their office and ignore everyone they could get it done. They're probably half right, in that they also need the full support of the customer. Since programmers are a noisy bunch, they complain far and wide about Agile, saying that it's the cause of all the diseases in their world. But sick systems often have names that are misleading. Deming once saw a poster on a client's break-room wall (or something) touting "Total Quality Management" and he said something to the effect of, "I hope that you aren't predisposed to think I'm wrong because of this", meaning that someone had already set up a sick system and claimed it was inspired by Deming.

Do you know any substantial software in wide use that was created using "agile" methodologies?

Meta-point: Nothing wins the peace, peace is just war that's managed to make itself sustainable.

Disagree. Peace means not swinging the axe. If you can't tell whether the axe is swinging, you shouldn't be applying a war/peace metaphor.

Also, war can be sustainable, especially if it isn't total war. Look at Europe a few centuries back.

What a sad perspective.

I'm my experience, the irony is that Agile is agile about everything except itself. Companies adopt scrum, after reading a book or worse hiring consultants, and try to follow it by the letter without applying any intelligence or taking into account that one size rarely fits all.

Personal anecdote from working at the largest ISV, which went gung-ho on scrum such that careers called scrum evangelists were created, engineers and engineering managers became scrum masters and the program managers, jobs that should have become obsolete to an extent as scrum mastersv take in lots of those responsibilities, started doing who knows what. But of course anyone saying anything against the scrum was a dinosaur.

Just history repeating itself.

They are rediscovering the PMI ad-hoc.

Not because the Agile ideas weren't better but because human and hence organisational dynamics still are and will be the same.

No surprise then, that as Agilists are gaining traction and power they are making their Agility more rigid and dreaming of Chief Agile Officers and roles like that.

Organisations have power structures and position there determines progress and survival.

They have to de-agilize agility to survive. If not they will have nowhere to go, nothing to latch onto when things go wrong

The title is a nice turn of phrase, but there's no real content in the article, just a series of unsupported assertions painted over with a sense of wistfulness. It reads like a nice proposal for an article that has yet to be written.

Was hoping the author would explain what people get wrong about agile.

Good point, bad article. He never gets to the "why".

"Agile" hasn't even looked like a word to me in years. It's become nearly meaningless.

Yeah it doesn’t mean “agile deadlines”

From my understanding, Agile was developed by developers sick of the rigid and static waterfall model and the problems caused by such a model. In discussions on Agile I often see a lot of complaints about managers not doing it right, wrongfully expecting developers to commit to certain things and then actually deliver on that. As a manager, I must say I feel a little bit offended. Not only did the developers specifically ask for Agile in order to fix the issues with late projects that doesn't fulfill their purposes, they are then telling us in threads like these it's our fault. What I read from these comments is a lack of understanding and ownership of the problem, and capability to solve it.

To sum it up: Problem: Missed deadlines/cost overruns and lack of success in fulfilling the actual problem. Solution in theory: The theoretical, engineering/developer, solution to the problem: Less commitment and more flexibility. Basically set the right expectations that we'll drive in the wrong direction and then need to correct it rather than continue because that's what's originally agreed. The "solution" in reality: Less commitment and less responsibility and ownership of the business situation (which hasn't gone away and never will). What happens: Less focus on the original problem. Developers thinking that deadlines and not fulfilling the purpose is even more something that is the problem of managers, and not developers. When shit still goes south because developers are still often not any good at tackling the original problem (although the promise was made by them it'll be better if they run the show), they (as obvious from the comments here) still put the blame on management.

In other words. Developers thought they were better at solving a problem than a person in a role dedicated to solving those types of problems (compared to developers where it's not in focus at all), still blames management when their proposed solution failed. Ironically (but very logically) the least capable are the ones complaining the most and owning the problem the least.

What I believe has caused the situation: An immature industry (due to young age), a complex problem and a lot of demand for developers making companies/managers not having enough leverage to be able to pressure devs into focusing on the right thing/development (the rockstar mentality). Plus of course inexperienced managers that doesn't dare to take control over the situation.

The worst part of it all is that it takes away focus from the actual problem, and hides a lot of incompetence (both on the manager and the developer side) which I find is often a root cause (and my understanding is that science agree with me - which and how correctly you use a specific methodology/process is not strongly correlated with overall success.

PS. I still believe Agile is good in many cases and there a lot of smart developers doing Agile with great results, but there are equally a lot of people who believe they at better at solving problems they are perhaps the least capable of. What's really sad is that many of those people are actually quite good developers.

«Хотели как лучше, а получилось как всегда» is better translated as "We tried our best — you know the rest".

Is anyone else tired of this story by this point? Agile was wonderful and beautiful, the problem is all the people who didn’t do it right. We get it.

There used to be a joke, “this job would be great if it weren’t for all the users”. It was supposed to be a joke.

Agile is like communism. Looks good on paper but you end up in Soviet Russia.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact