We recently brought on a couple of PM consultants who are deep into Agile as a methodology, and decided to give it a go -- it's been about six months now and productivity has basically ground to a halt as we spend more and more meeting time endlessly sorting tasks into different little arbitrary piles instead of actually making forward progress. It's made communication with the non-software part of the company much more difficult, because they keep getting hung up on the jargon; we have to slice up tasks arbitrarily to get them to fit into "sprints," which go out of date minutes after the morning standup when the first non-planned bug report lands. Sizing is frequently an exercise in post-facto book-cooking: 'How long will that bug take to fix?' is like asking 'How long is it going to take to answer 23 across in next Sunday's crossword puzzle?' It might be fifteen seconds, it might take all day; until I have a chance to look at the puzzle I'm just pulling numbers out of my ass. The short timeframes encourage everyone to twiddle around the edges of things instead of doing real feature development; the arbitrary milestones serve mostly to make everyone feel like they're constantly falling behind.
I've discussed this with a bunch of people. Everyone seems to agree that we're doing Agile wrong, but even the strongest proponents of the methodology seem to be in complete disagreement on what the "right" way to do it is, or how that would be an improvement over the way we were doing things before.
>Everyone seems to agree that we're doing Agile wrong
The fact that we've all heard this line (yet never heard a solution) should make it pretty clear that it's more of a management religion than an engineering practice. "You're doing it wrong" is the perfect built-in, pre-packaged defense to the Agile system that agile managers/consultants/etc can rattle off with zero effort and an air of superiority. Had a bad experience? Well, you were doing it wrong. It's not an issue with the system.
I recently worked at a place that was the text book example of agile/scrum development methodologies -- Which I don't mean figuratively, I mean in the literal sense, their big claim to fame was that they were the subject of a case study on the success of Agile methodologies. So, having worked in a place that was studied for "getting it right," it's extremely annoying to see all critiques of agile had waved away under the flag of "you did it wrong"
Just about every complaint in the article existed at this place as well. For me, at the tip top of the list was meetings. Planning would take hours. Sometimes it would be split up over two days. That is two days out of the 10 you have available for development eaten up by talking about what you're going to develop. Then there's a meeting to show what we did in those 10 days. Then another meeting to talk about what we think we did in those 10 days and how it could be better (which was of course done through stupid time wasting games ("I wish I could have ___" "I was happy when ___", "I should start doing ___"))
Then layer on top of all that a whole hierarchy of people whose whole job description is to be "agile" -- which not a single developer was able to explain to me the meaning since the day I started.
I'm ranting too much, but the whole agile thing just gets under my skin. It's such a massive waste of time. Leaving that company was a happy happy day. I now work at a place where 'planning' takes 15 minutes in front of a white board. Engineering discussions take as long as they need with the relevant parties whenever needed. Other than that, you're left alone to fucking do your job.
Back when I was a wee lad, with nary a keyboard callus upon my digits, I learned about a little thing called Murphy's Law. In brief, it says that if anything can be done incorrectly, someone will eventually do it that way. It was a warning to those designing things, to make doing the wrong thing impossible, or at least much more difficult to do accidentally.
As a result, among those taught about Murphy's Law, "doing it wrong" was a design flaw. If, for instance, you could destroy radio reception in a handheld device by holding it in a particular way--a way that did not cause pain or discomfort in an ordinary human hand--it was the fault of the device manufacturer, not the user.
If you can do Agile "wrong", then it is a problem with Agile.
As for myself, I just think to myself that someone, somewhere, has weaponized an idiot, and has launched it against me. It is equal parts Inspector Clouseau, Mister Gumby, Ali G, Drunken Master, and a particularly evil djinn. He will do exactly what you ask, in the most malicious manner possible. He will follow a process with absolute rigor, and produce the worst possible outcome.
I firmly believe that such people actually exist--individuals so brutally incompetent that they are indistinguishable from malicious trolls. So you might as well plan around a person intentionally trying to break your design while still maintaining the plausible deniability of not breaking any of your rules.
Agile cannot hold up under the assault of a weaponized idiot. There are simply too many possible attack vectors.
I agree with what you're saying, but I also wonder, if not agile, then what do you see as a better alternative?
It sounds like you're suggesting that Agile methodologies have flaws that allow projects to be easily sabotaged. While I agree with this, I'm not sure I see an alternative? Is any development methodology going to be immune to this problem?
Human communication is messy, subjective, and open to all the imperfections that come along with the human condition. Unless you're working alone, any development methodology is going to be susceptible to "people problems". I don't see this as something that's unique to agile. I'm sure I could come up with hundreds of ways that idiocy, incompetence, and maliciousness could sabotage waterfall projects as well.
People within the software industry have a great advantage, in that they use their skills to both solve business problems and automate the solutions. Increasing the efficiency and effectiveness of the software team is a business problem.
There are some common elements to that problem that encourage the re-use of certain solutions--source control, for instance. The only mandate that needs to descend upon the team from above is to examine and improve your own processes.
There is only one other profession of which I am aware that can, given the opportunity, make every tool that is required for the job. That is the blacksmith. If a blacksmith discovers that a certain specialty tool would be useful for a routine task, he could either modify an existing tool or make the new one from scratch. No sane customer would barge in and demand that he use the "wrong" tool for the job. You might just find that red-hot tongs are the wrong tool for proctology.
But that happens to software professionals all the time. And that's where the weaponized idiot comes in. Allowing the customer to tell the skilled worker which tools to use sets up the situation where you could be performing surgery with a grapefruit spoon, while wearing oven mitts, or knocking down a brick wall with a rubber mallet and a hamburger spatula from an outdoor grill set. The unbelievable outcome is that sometimes that task still gets done.
A development process is a tool that software professionals use to do their jobs. If management needs another tool to help them do their jobs, those software professionals would be all too happy to make one. But let's not try to weld those two different tools together and call it a supertool. Agile is one of those welded-together things now, and the different parts might not work well together in your shop.
If your team cannot work without such a process, it may be because the business has neglected to hire a workable mix of masters, journeymen, and apprentices. Too many masters, and they may come into conflict. Too many apprentices, and they may struggle for direction. Too many journeymen, and they will leave to be masters elsewhere.
The only development methodology you really need is to make sure that every team has at least one master on it, and to let them pick their own tools and apprentices. If they can't get the work done with that level of autonomy, fire them and hire someone else. That's not a problem you can fix by going Agile.
Taking on a new development process in the middle of a project is much harder than starting on a new one. In particular, as you mentioned there's an issue of legacy bugs. There's mounds of information available suggesting strategies for taking care of this, but ultimately you'll be in an unfortunate transition while you're still handling legacy bugs that makes it hard to move forward.
Most of my experience with agile is in the XP flavors, where the mantra is no bugs. I know it's hard to believe, but I promise it's an achievable goal if the whole team single-mindedly works towards that goal.
You're right, estimating is hard. The thing is, people are actually really good at relative estimation, it's absolute estimation that we completely suck at. That's the idea behind estimating in points: you free yourself from trying to think "how long will this take me?", which you'll inevitably get wrong, to "is this harder or easier than this other thing" which almost always is pretty easy to do.
> The thing is, people are actually really good at relative estimation, it's absolute estimation that we completely suck at. That's the idea behind estimating in points: you free yourself from trying to think "how long will this take me?", which you'll inevitably get wrong, to "is this harder or easier than this other thing" which almost always is pretty easy to do.
That sounds great on paper. However, every implementation of Agile that I've ever seen or heard of eventually translates points to time intervals ("How many points should this be?" "Well, a one day task is three points, so...").
Well, then, to give you a piece of anecdotal evidence; my experience where I currently am is that we never talk about how long it'll take, unless it's so small and trivial we can say "Give me an hour and I should have a first pass in place". Even then we're not saying it's ~done~, just that we're familiar enough with the code that we can get some prototypal code in place in a time frame. When actually estimating tasks, it's purely point based, and then those can and do vary with how much time they actually eat up, but tend to average out just fine.
- Middle management mover-and-shaker. Most recent technical contribution to a product: Some time in the last ice-age. Super power: secret meetings that lasted for hours, discussing how much all the devs were late, and who to blame.
- Underflunky. Ran daily stand-ups to gather status.
Last technical contribution to a product: Never. Perpetually terrified. Super-power: "Since we need this done by the end of today, let's have a bunch of meetings about it. Are you available for an hour at 10, 2 and 5?"
- Fantastically technical PM. Last technical contribution to a product: Helped debug a gnarly problem over in the other building about ten minutes ago. Super-power: Writing intelligent specs that devs understood and that actually described shit that would work. Achilles heel: Incredibly over-worked and a burnout in five months.
We had one or two of the latter PMs on the last project I was on. They were great; every other PM did negative work.
I did get our underflunky PM to start calling our scrum meetings "Status".
'PM' - well, there's a problem. We don't have project managers. Product -owners-, yes.
But you have a point; a lot of places try to 'be agile', as something the devs do, without also including management. As a number of others in this thread even indicate. Management has to also change, or else there's a mismatch in expectations.
As a dev doing SCRUM, we report point totals, and allow the business unit to prioritize stories based on that, and based on the need for the feature. Point totals above 8 or so we try and break down into more stories. The only guarantee we give is that if we include it in a sprint, we will get it done in that sprint. We can estimate it at ~8 hours (~two days of a single dev's time; we build in slack, for meetings, context changes, dev tasks, and the unexpected), but we aren't guaranteeing that you'll have it in two days; it'll be the end of the sprint. Any more exact estimate is only for other devs, in case they're reliant on it being in place for something they're working on.
We have Product Owners. And PMs. And Dev managers, and QA managers. It's a giant ball of red tape and regret.
At one point the official 'scrum coach' actually convinced everyone that he had a magic formula for converting points to hours and that he could take the pointed backlog and produce a project plan out of it to produce familiar reports for management.
Everyone says we are doing Scrum, even the agile coaching team. But its really just waterfall done using Rally.
Fine, "product owner" or "scrum master" or "person who schedules meetings and fiddles with the JIRA board". Whatever. I don't care what piece of jargon you want to substitute in.
Let me ask you this: why should any customer or business unit give a flying fuck about your "points"?
Customers and business units should have a strong voice in prioritization of tasks, yes. However, in the process of prioritization, an estimate of time for the task--be it your estimation or theirs--will be a factor. If all you tell them is a number of points, then guess what? They'll back into their own time estimations from your points.
So what? Let them. They get estimates that are relative to each other (~this~ story will take longer than ~that~ story), and can turn them into time measurements them however they like if they want to; we're not committing to their estimate.
All we're committing to is to deliver what -we- choose to include in a sprint, will be delivered by the end of the sprint. We will choose what is in the sprint based on the prioritized backlog. How they prioritize those things doesn't actually matter to us. They think that 8 point story we pulled in is going to take us two days? Doesn't matter. They think that 8 point story we pulled in is going to take us two weeks? Doesn't matter. They're not the ones deciding how much we can take on in the sprint, nor are they the ones committing to get it done. All we are telling them by our pulling it in is that it will be done by the end of the sprint. And that's what I meant by management has to change; that reality has to be acceptable to them, that getting relative points, to tell them the approximate amount of work (or t-shirt sizes, or whatever) relative to other stories is all we can really tell them, and that we will pull in the prioritized stories based on what we think we can get done. If they don't trust us to do that, and want to micromanage, then -no- development methodology is going to help you, not agile, not waterfall, not -anything-, because you have systemic dysfunctionality.
> So what? Let them. They get estimates that are relative to each other (~this~ story will take longer than ~that~ story)
First, they'll ask how much longer ~that~ task will take to complete than ~this~ task. Then, they'll ask how long ~this~ task will take to complete. Then, they'll use this little trick called "math"[1].
> and can estimate them however they like if they want to; we're not committing to their estimate
Well, guess what? If you refuse to give deadlines, then they'll be established for you. I don't know about you, but I'd rather have a say in my deadlines (where possible).
You also seem to ignore what I'm saying; we give deadlines. The end of the sprint. Whatever we take on will be done at the end of the sprint; they have control over what we take on by the backlog. Beyond that, we refuse to give any commitment (instead just giving relative weights to how long a task will take, to help in prioritization), and we are sufficiently empowered to not care if they try to commit us to something despite that. But they usually don't (the only time it's ever more immediate is "a bug was seen in prod and I want to know what caused it!", in which case that becomes our first priority task; again, because we consciously choose to undercommit ourselves, there is slack for that sort of thing to occur without delaying the schdule), because we work with them, listen to them, advise them, and they see that we are trying to help them do what is best for them, without causing a bunch of tensions and clusterf*ery. And because we undercommit we usually deliver more than we commit to, with time used to make sure things are reliable, so they have come to trust us.
In short, I hear a bunch of presumptions from you about what management -always will do-, whereas I know this from experience to be false. Management -can- adapt to viewing things along sprint boundaries. At least, mine has. Perhaps your problem isn't that a process is bad, but that your managers insist on micromanaging you and that you're not sufficiently empowered to push back?
You're right, eventually you have to convert points to some notion of time, if only so that you know how much work you can do in a sprint. The idea is that you make that conversion based on past observation, using a velocity estimate that represents the past performance of the team and takes into account who's on vacation this week.
There also should be an open recognition that your actual velocity will vary somewhat between sprints. One goal is to reduce that variability by improving your estimation (with practice!), but you anticipate and prepare for the variability. That's why you have a backlog (in case you need more work) and why you develop units of work that are independent (in case you need to push one).
For all these reasons it's a mistake to think of an equivalence at the level of tasks such as one day is three points or something like that.
Legacy bugs? What the hell are legacy bugs?! What about, you know, normal bugs. Normal, unforeseen problems that happen all the time that you can't account for. Do you just add 25% time to any project to cover for those? I'm not sure how that solves the issue when you have to slice everything up into tiny chunks, where a few chunks could contain all of the unknown unknowns.
It's good to realize the known unknowns, but it's the other kind that will get you.
Legacy bugs are those in the codebase from before the team started working with a test first, no bugs mantra.
As I alluded to there are many approaches to this problem, but one that I like is more or less just what you suggest: set aside some amount of time every sprint to tackle the legacy codebase. The eventual goal is to bring it up to the newly accepted standard which requires tests, but realistically that will take a while. Live bugs provide a sort of direction to help facilitate updating the legacy codebase. Setting aside a set amount of time each sprint makes sure that the issues don't get swept aside.
It's important to get to the point where fixing live bugs isn't impeding the development of new features, but until then it's simply pragmatic to set aside some amount of time to hedge against new issues, while also having the discipline to set one aside if it can wait until the next sprint.
As an aside, some of the best patterns on the subject of dealing with legacy systems can be found in the later chapters of Domain Driven Design by Eric Evans, particularly chapter 14, Maintaining Model Integrity.
Well there is an interesting concept - you're talking about jargon like "sprints" yet referring to "Agile" which is a very, very broad church.
I like Scrum for the power of the retrospective, if people actually talk about issues, the the team really is empowered you can change those problems:
a) Talk through how to handle bugs. There is no prescriptive answer, if you have a problem with defects then fix it, or develop a system to deal with them. You don't have to bring a bug into sprint after all, and if you do then get your Project Owner to drop an equivalent bit of work out.
b) Refinement is a crucial part of the process, if you can't break up the task into 1/2 weeks worth of work for a team odds are you don't fully understand the task, which means you can't provide forward estimation, which means management can't trust you're numbers.
c) "Project Managers" don't exist in Scrum, so a PM consultant guiding you on agile feels very, very wrong.
If your organisation can't commit to you working full (pretty much) time on a project working on a set of things that have been committed to without pulling you in another direction then odds are Scrum isn't going to work for you.
Scrum is very clear that there are certain things you need to do in order to say you're doing Scrum. Doing "Agile" screams following no particular sub methodology but cobbling something together. There's nothing wrong with that evolving, but it helps to be able to describe why you've gone in that direction. In your case it seems like you've had something foisted on you by external people talking out of their rear.
> If your organisation can't commit to you working full (pretty much) time on a project working on a set of things that have been committed to without pulling you in another direction
See, this is the thing: on the one hand it's supposed to be all about committing to a specific direction ahead of time and not letting yourself be pulled somewhere else -- on the other hand it's supposed to be all about developers being empowered to work on what's most important at the moment (I mean it's right there built into the name!) Those seem in direct conflict.
We don't have organizational pressure pulling us in different directions -- we have reality pulling us in different directions. I'm not going to tell a customer "no, you can't have that bug fix or that new feature yet, we have a Methodology to follow."
> "Project Managers" don't exist in Scrum.
Right. Just project owners, product owners, scrum masters....
> In your case it seems like you've had something foisted on you by external people talking out of their rear.
I think you're onto something there. (They keep changing their minds, too: one week it's fibonacci sizing, next week it's T-shirt sizes. One week it's scrums, next week it's kanban. The actual working process seems to stay the same, only the jargon shifts around.)
>developers being empowered to work on what's most important at the moment
There's the problem, the "moment" is the sprint. If you're need for a flexible time frame is less than a week then Scrum isn't going to work for you. There are other methodologies that might work (Kanban say), but I'd also question whether you're doing pure development or mixing in support as well.
> "Project Managers" don't exist in Scrum.
> Right. Just project owners. Uh huh.
"Product Owners" should never, ever, ever be Project Managers or anything like them! They're supposed to be from the business end of things. The worst projects I've ever seen in Scrum land have been developing for project manager's needs (the only time PMs get to be POs) and it went horribly because they act like project managers not business people focusing on what they actually need the product to do.
If your consultants are confusing Kanban and Scrum then I'd run screaming for the hills. I've literally just come out of a retrospective where we had a discussion about switching from Scrum to Kanban for a load of impact analysis heavy work, they are different beasts both good at different things.
My experience feels the same way. I am beginning to think Agile works for UI dev and small teams of inexperienced folks building relatively simple projects. If you have a large team building something complex, the constant meetings and tool updates destroy productivity. What's worse ... they destroy developer morale if you are actually a good dev IMHO.
The classic success cases for Scrum were projects that had (a) been done before, and (b) been done by the same team.
So they had lots of experience.
If you ask an assembly-line worker "How long is it going to take you to install this transmission?" they'll be able to tell you within a few seconds. But that's not engineering.
If you ask someone who's written essentially the same app five or six times before "How long will it take you to finish feature X?" they'll be able to tell you with pretty high confidence.
If you ask someone, "How long is it going to take you to finish designing and implementing that gozzlewog?" they won't have the foggiest idea. Ask them again, when they've re-implemented it a few times, and they'll have an idea then. Until then you can decompose the problem and do analysis on the pieces and make assumptions, but you still won't know how long it'll take until it's done because the industry has been doing this for like 70 years and the one thing we know is that scheduling unknowns and unknown unknowns is still very, very hard.
Scrum works if you've done it before. Apply it to green fields or systems with known wicked behavior, and you're likely to fail in quite a few ways, including pissing off your developers and boiling off the good ones for better work environments.
An actual example I've seen: you can ask a field technician "how long will it take to install this piece of equipment in the field?" and get a pretty good answer. You can't ask a data scientist "how long will it take to build a statistical model to predict how long jobs out in the field will take?" and get anything at all useful.
And you certainly can't ask "how long will it take to build a statistical model to predict how long it'll take a data scientist to build a statistical model?"...
If you ask someone, "How long is it going to take you to finish designing and implementing that gozzlewog?" they won't have the foggiest idea.
That's technically true but not useful. There are much better questions to ask, namely:
- Will gozzlewog or foofaraw take longer to implement?
- Is the difference minor or is it an order of magnitude?
- Will shipping gozzlewog or foofaraw make a bigger impact on the business?
- Is that difference small or large?
All of theses are easier to estimate and are more useful, too.
If a gozzlewog is something you haven't done before (and maybe nobody has done it) then you have much less data to go on. Coming up with an estimate for it can be really, really hard. Want to decompose it and estimate from there? Hey, you just fell into the trap.
I wish I had a dollar for every time I've heard someone say "That turned out to be harder than I thought." And this is nearly always the case when you're doing something new. So you hem and haw and pad and you're late anyway . . .
But my stand is that you're actually not late, you're actually on time and it's just that the estimate you made (or that someone made for you) sucked. You don't even suck at estimating because you can't reliably estimate something you haven't done before.
Oh sure, small things. But not big things. Probably not even medium-size things.
Imagine you're in a scrum planning session and someone says, "Hey, we tagged you for implementing Gozzlewogs" and you have absolutely no idea what a gozzlewog is, or maybe you read about them a while back so you're the person that people come to when they hear "gozzlewog." But you're on the hook for it, and they want an estimate and the answer of "I don't know" apparently isn't on the table.
So do you:
- Dig in your heels and say "I don't know" anyway?
- Lie and say "Two weeks" because half of the dev team is in your shoes and doing the same thing, and you'll all slip together? Besides, next sprint it'll be the same bullshit, except with frumwidgets in addition to the deep, complicated and tricky stuff you discovered while investigating gozzlewogs.
Now, clearly this is broken, and it's not even the fault of Agile, but rather the middle management that thinks developers are fungible and cog-shaped, and that everything can be decomposed and predicted, and that you can have a schedule with a granularity of days when your known unknowns are at the scale of multiple weeks.
Here's the thing: by the time you really know how long a feature will take to develop, you've done it. Until then you need to operate on an estimate. EDIT: I see you edited your comment to include the same sentiment while I was replying. Cheers!
People are bad at estimating, particularly if they're doing it alone, don't have much information or are trying to estimate absolutely. But we're pretty good at estimating relatively, doing it in groups (witness bean counting contests) and even with just a little information we can make improved estimates.
So the ideal task planning in my mind goes like this: the team gathers around the backlog to see what we think the next priorities are. Briefly we chat about each one, enough that everyone has a basic sense of what it is and no one is blindsided. We intentionally don't want to talk about it enough to implement it, just enough to get a sense for it.
The team then collectively estimates the size of each item with a hidden bid system: each person picks a number on an arbitrary scale (let's call it points) that only has meaning relative to the point values of the other things on the list. Once we have a handful, it becomes pretty easy to say "foo is harder than bar, but not as bad as qux".
Cheers, indeed. (Sorry about the edit . . . it's the way I think). Happy we're in agreement.
Part of my problem with Agile is that my bad experiences have made me cynical and somewhat allergic to it. I react badly to being micro-managed and gamed by managers who are fundamentally out of touch with the reality of writing software, and who are interested in making the deck hands row ever faster so they can look good.
So about 5.5 - 6.5 hours of meetings over 2 weeks. If you're really militant, you can probably take it down to 4 hours, but I don't think you can get much lower than that without dispensing with formal Scrum.
It doesn't seem unreasonable to me to have 6.5 hours of meetings over 2 weeks.
They are doing it wrong. 8 people should not take an hour to say: What did I do yesterday, what am I doing today, is there anything blocking me?
There should not be a discussion of every point. The only discussion should be "oh let's talk about that impediment after the meeting with a smaller group of people" or "let me follow up with you after lunch".
The standup is 10-15 minutes. If it is longer, it needs to be stopped. Period. Non-negotiable.
> There should not be a discussion of every point. The only discussion should be "oh let's talk about that impediment after the meeting with a smaller group of people" or "let me follow up with you after lunch".
Guess what? Those things are still meetings. They still take up actual time that could be spent on getting shit done.
They don't, however, involve everyone. The standup does. That's key. If person A needs to talk to person B about an issue, and it's done in standup, it wastes person C, D, etc's, time, even though they don't need to be involved. Doing it with just the required people is, obviously, required anyway, and the problem a lot of people doing agile have is they end up wasting everyone's time by bringing it up during standup.
A 45 minute meeting that spawns off of a 15 minute meeting is still a total of 60 minutes of meeting time. And yes, while that 45 minutes involved fewer total people, that doesn't mean that others won't have spin-off meetings of their own.
But that 45 minute meeting had to happen anyway, apparently. It wasn't scheduled or mandated, it was decided by those two people it needed to happen. Meaning regardless of process, it was necessary. The issue is a lot of people seem to see "standup requires everyone", and "communication should happen", and conflate the two to "every meeting requires every person", which just is not the case, and then as they see all their time is taken up by pointless meetings they didn't need to attend, they blame agile. I've seen the same mistake in waterfall development shops, and people blamed the process there too, even though it's really just cultural.
Let me give you a specific example.
Persons A-H have a standup. It's 15 minutes.
Afterwards, A needs to talk to C about something, which will take 45 minutes. B needs to talk to D about something, which will take 45 minutes. F needs to talk to both A and H about something, for 45 minutes.
A talks to C, while B talks to D. Then A talks with F and H.
Total time taken (excluding the 15 minute standup) -
This is exactly what should have happened -regardless- of your process; these are meetings where only the required people are involved, no one's time is wasted.
What often ends up happening when you hear about day long status meetings and other such terribleness, is after the 15 minute standup, they go straight into discussions, with the entire team. That is -
A talks with C, with everyone still present. Then B talks with D, with everyone still present. Then A talks with F and H, with everyone still present. For a total time of 135 minutes (again, excluding the 15 minute standup). Meaning that -everyone lost 135 minutes-.
Now, if you're claiming that A never actually needed to talk to C, and B never actually needed to talk to D, and A never actually needed to talk with F and H, then that's an organizational problem that is, again, unrelated to agile (or waterfall, or anything else). You simply have people who insist on wasting others time, and regardless of your methodology you're going to run into that issue.
If you want to say that 2 people resolving an issue is a meeting, sure, go ahead and call it that. But I would suggest that the meeting needed to happen anyway.
Scrum doesn't imply that unneeded meetings need to be created for no purpose.
It is perfectly okay to also say during the standup "I'll follow up with you over email later" or "Let's resolve this over Slack".
So, have you ever said "Anyone going over sixty seconds will be shot" and there's that one guy - or two, or three - who nevertheless always take five minutes? Yeah, turns out you probably can't actually shoot them.
There were frequently 10-15 people in the standups. And sure, you can say "if it's longer, it needs to be stopped" but management doesn't work like that.
The problem is your company has no idea of what it's doing. You don't estimate bug fixes or research and development. You time box them, and once you've spent all the time allocated, you report back to your team on your findings. If you can't fix the bug in a sprint, you simply work on it in a later sprint.
Second, real feature developments sounds like an epic. Those are stories that will span multiple sprints. There are ways to handle them.
Bug fixes, R&D, and real feature development are the entirety of what we do.
If none of those fit into sprints, what is the point of having them? What is the benefit of describing a feature as an epic sliced into stories spanning sprints, instead of just, y'know, getting it done?
No, they do fit into sprints, quite nicely. Time boxing gives you a deadline and accountability. Using research and development as an example, you go off, spend months researching a new search algorithm, only to come back to hear the customer tell your approach was wrong in restrospect. The sprint mechanism catches that sooner, preferably at the end of a single sprint, then months down the line. It's fail early fail often. Splitting a story into an epic forces the realization the feature is complex, and in many cases, seeing those numbers there quickly turns the feature from a must have into a "nice to have."
A key point to agile is that you are presenting the output of your work to the customers at regular defined intervals so the customers can decided if what you're doing it worthwhile. The daily sprints are a mechanism to detect when something is wrong.
Fail early, fail often is a fundamental component of agile.
> Time boxing gives you a deadline and accountability
An arbitrary deadline, often forcing you to interrupt your flow and then spend extra time next sprint reorienting yourself to pick up the same task again. And artificial accountability. (Did Alice overrun her time-box because she was lazy? Or because the bug was really tricky? Nobody knows! Is Bob a rockstar, or is he just really good at plausibly over-sizing small tasks? Perhaps!)
> you go off, spend months researching a new search algorithm, only to come back to hear the customer tell your approach was wrong in restrospect
That's kind of strawmanny. In no methodology should someone be allowed to wander around for months with zero feedback.
In "traditional" process the PM is in communication with the customer and keeps enough of a handle on where things stand to be able to pull the brakes when necessary. Agile seems to want to offload that work either into some vague diffusion-of-responsibility within the team, or to the customer herself (which seems bizarrely idealistic) or, most commonly AFAICT, to a de facto PM under another name.
> so the customers can decided if what you're doing it worthwhile
I have literally never met a customer who didn't want everything right now. A good traditional PM is able to manage those customer expectations. Few developers I've met have those skills or want to be spending their time doing that.
"An arbitrary deadline, often forcing you to interrupt your flow and then spend extra time next sprint reorienting yourself to pick up the same task again. And artificial accountability. (Did Alice overrun her time-box because she was lazy? Or because the bug was really tricky? Nobody knows! Is Bob a rockstar, or is he just really good at plausibly over-sizing small tasks? Perhaps!)"
You know at least 2 weeks in advance you have to be able to give your customers a report of your findings. If that interrupts your flow or forces you to "reorient" yourself, that sounds like you have poor planning and time management skills. If Alice is consistently missing her deadlines, or if Bob is consistently over-performing, the team needs to analyze the tasks being given to those individuals.
"That's kind of strawmanny. In no methodology should someone be allowed to wander around for months with zero feedback."
There is a difference between "go research this and report back" and "go research this for 25 hours and get back".
"I have literally never met a customer who didn't want everything right now. A good traditional PM is able to manage those customer expectations. Few developers I've met have those skills or want to be spending their time doing that."
Agile and the constant feedback cycle help to manage those expectations. It's another tool in the PM's toolbox.
Of course they all do. Another fundamental part of agile is the constant communication between customers and the team.
"'How long will that bug take to fix?' is like asking 'How long is it going to take to answer 23 across in next Sunday's crossword puzzle?' It might be fifteen seconds, it might take all day; until I have a chance to look at the puzzle I'm just pulling numbers out of my ass."
.
I find this attitude to be a little disingenuous. Obviously every bug cannot be estimated perfectly, but the majority of the time there is some initial data to make a reasonable estimate with a margin of error.
For instance to name a few:
- Past experience with the module associated.
- Whether or not it has been reproduced locally.
- Similarity to other bugs that have come up in the past.
It's my job [or used to be :-)] to give estimates of bug fix times to management.
It usually broke down to:
- trivial (about an hour)
- nearly trivial (maybe half a day or a day)
- "I have no earthly idea". The last one of these that I fixed involved about twenty people, deep dives into our hypervisor and interprocessor communication mechanisms, research into some hardware details and three 100+ hour weeks about four weeks before we shipped. And we started out not having any idea where in the system the problem was (that took two of those weeks).
Some or several of the points you raised probably fit more into the "chance to look into the puzzle" phase so that doesn't really stand to reason against his point 100%.
To be sure. And if we were doing routine maintenance on a longstanding codebase things might be a bit more predictable than they are. But we're trying to do new feature development on a new product, and my experience has been that beyond a very general sense of "this is going to be a big one" vs "this shouldn't take too long, unless I run into something unexpected" -- which is the same level of detail you get with or without Agile -- sizing isn't meaningful enough to be worth the time it takes to do.
Your analogy to estimating 23 across is pretty amusing. I like it a bit more than the claim that software estimates are like asking how long it will take to detect dark matter (another one I've heard), because honestly, I really shouldn't try to equate my work with something on that scale. That said, I actually think your analogy is, if anything, a bit too optimistic about estimates. In some ways, I'd stretch it to "you're good at puzzles. how long will it take to solve next week's puzzle". You know that puzzles are usually crosswords, sudokus, word scrambles, or chess problems, and you can usually get them, even the hard ones. There's variance, and sometimes you do get stumped, so it's a bit stressful, but at least you have some kind of prior history as a guide. But surprisingly often, it turns out to be something like encountering a rubik's cube for the first time. How long does it take to solve a rubik's cube? Well, once you know how to solve one, it takes less than 5 minutes. How long does it take to figure out how to solve one when you've never encountered it? Well, for some, never. For others, a month? For a few people, a day?
But the answer is, it will take me anywhere from 30 minutes to a month to tell you how long it will take me to solve this problem (over 1 month, I give up). Then, my answer will be: 5 minutes.
We're a very small team all of whom are skilled senior-level developers, working on a greenfield product. Basically boils down to back end guy, front end code guy, unicorn product designer/ front-end dev, and ops guy.
(Agile proponents have told me "Oh, that's exactly the wrong sort of group for Agile, it works best with larger teams of lower-level devs." Other Agile proponents have told me "oh that's the perfect sort of team for Agile, it doesn't work with big teams of juniors.")
The way it used to work was: Feature request comes from sales or upper mgt or customer or product designer. Designer does mockups or codes up a simple working prototype. Various team members look at it, we talk about how to implement it, everyone works on it as needed, we deploy it when it's done. If something else comes up in the meantime that's more important or more urgent, we set it aside in a different code branch until there's time to focus on it again.
The way it works now is basically exactly the same, except with lots more meetings and jargon and taxonomizing of tasks and arbitrary milestones.
(Agile proponents have told me "Oh, you were doing Agile already, you just didn't know it! Other Agile proponents have told me "oh, you're not even doing Agile now." Even its proponents can't seem to agree on what Agile is, which makes it really easy to tell its detractors that it's not Agile's fault, you're just doing it wrong...)
Getting stuff done, but in the eyes of management not fast enough. Just guessing but this seems to be the rule.
If you really need agile it will not help and if your team already rocks agile won't improve it, likely you'll already be doing something like it but customized to your needs.
I've seen a couple of fairly efficient teams utterly ruined by switching to 'agile' and I've yet to see an inefficient team become more efficient by deploying it.
A good policy is to not estimate bug fixes, to not associate any points at all with them. It creates a drag on velocity, but it's honest.
If you need to fix a bug, it means that you are correcting an error from a past (implicit or otherwise) user story that shouldn't have been accepted as complete anyway.
This article cements in my mind the issue: devs tend to take away from agile what they expect to find, rather than taking the time to understand the motivations and whole scope of activities.
Historically, most software development methodologies are excessively top-down, so people somehow still expect that from agile. Most devs are the sort that couldn't stand doing group projects in school, so the idea of organizing with teammates to produce something bigger and better than I could on my own is totally foreign.
If you don't explicitly name your development methodology, you still have one: you're practicing something called ad-hoc development. Unless you and your entire team are superheroes, there's an awful lot of knowledge being leaked away.
Here's the secret: software development isn't really about making computers work, it's about organizing knowledge. If your process focuses more on making the computers work than your institutional knowledge you're making a critical mistake.
Don't confuse some company's crappy engineering practices for agile just because they call them that. Stand ups are short. Period. If it feels like a "permanent PIP" your team could use some improvement. If it feels like a surveillance state it's probably not a very good implementation of agile.
Agile development is about making connections to your team members: individuals and interactions over process and tools. It sounds like the author's experience (as well as many other devs') is unfortunately in an organization that doesn't pay heed to that very first, most important agile principle.
>Agile development is about making connections to your team members: individuals and interactions over process and tools.
See, this kind of vague, fuzzy feel-good description is exactly why it seems to me that the one thing everyone seems to agree on about Agile is that everyone else is doing it incorrectly.
Agreed. I'd like to see some solid experiences that counter the author's claims being used in the wild that fit where he says it doesn't work well (anything other than small, short projects). Nearly everyone is taking the "it's not us, it's you!" approach.
A lot of things work well in theory. That doesn't really make them good ideas if they can almost-never be implemented properly.
"Here's the secret: software development isn't really about making computers work, it's about organizing knowledge. If your process focuses more on making the computers work than your institutional knowledge you're making a critical mistake."
But big-A Agile is a management school with its own dysfunctions. It diverges from small-a agile, which is basically a set of principles to help critique yourself with.
When you hire a manager (or "Scrum Master" whatever), they must convince top management to give them a job. Which is why Scrum Masters brag about their bosses "believing in" Agile, like a bishop happy that the king loves god. Then they impose some system which requires their highly visible involvement.
(That said, the Agile described in this article diverges from my personal experience.)
The free e-meter at every desk is a valuable job perk! The Kool-Aid tasted a bit funny, though.
Scrum-English Dictionary:
* "sprint": artificial crisis
* "end of sprint": abandonware
* "Scrum master": resume credit for my planned escape to a new company
* "stakeholder": someone you can't get away with externalising your costs onto
* "simplified": doesn't implement any of the actual business requirements
* "lightweight": doesn't implement any of the actual business requirements, but does so much more elegantly than the version that works
* "easy": project was born circling the drain (and doesn't implement any of the actual business requirements)
* "legacy": the version that works and implements the actual business requirements, even if no sane human wants to touch it
* "velocity": a speed with a direction, so skittering about following marketing's random hairpin turns counts
* "retrospective": blamestorm incoming!
* "stand-up": getting the blamestorming in early
* "We'll put that on the backlog": hahaha fuck you
I do enjoy some of the jargon. "empowered": do your own fucking job. "Could you just copy down these log files from these fifteen servers for me and put them on the shared drive? Thanks." "I'm sorry, I'm afraid you're empowered to do that yourself."
Seriously: every good idea is turned into a bad one by the relentless management quest to Taylorise clue.
"Inspect and Adapt": maybe bong hits will fix my makefile
"Project Manager:" a job that no longer needs to exist in the astounding new world of Scrum. (Pay no attention to the project owners, product owners, story-writing users above you in the org chart or Scrum Masters behind the curtain.)
"user story": a valiant attempt to extract coherent requirements and bug reports; ends up being precise specifications for the wing feathers of the desired magical flying unicorn pony
Every methodology fails from pretty much the same reasoning.
The saving point of Agile is breaking tasks into reasonable sizes, long enough to actually get something done and short enough to rearrange as needed sooner rather than later, and to delay interruptions until they can be evaluated and other tasks completed. I can't get to what I need to do if the relentless onslaught of self-important "do this _now_!" interruptions are handled as they occur; Agile provides a sane way to say "get in line" such that important issues are addressed in a timely manner relative to other issues and without compromising other stuff that also needs doing (those that never reach the top of the interrupt stack, but without which long-term progress won't happen).
The term has been hijacked. It is useless. There is merit in the idea that it once encapsulated, but it has long since been co-opted to mean something almost completely the opposite of its original intention.
Yeah. Scrum doesn't make management disappear, it amplifies it.
I saw meetings with middle managers (people who hadn't shipped anything significant in a decade) in daily meetings, going over burn-down rates of individual engineers and "expressing concern" through channels that someone was off their schedule by a couple of days. The daily scrums were just half-hour status-in-a-ring and some public shaming if you couldn't say that you did anything, so a lot of people made stuff up, or at least hedged by saying they'd completed 50% of the work or that tests were "nearly passing" (code for: my stuff is broked).
Oh, and in the mean time the build system we devs were using had been on fire and broken for three weeks and it was nearly impossible to get a working build. WTF?
I said, "We don't need Agile. We need an earthquake to collapse that wing of the building" (where the managers were; they liked to cluster).
We had Ken. Ken was there at the beginning for maybe a week, and for the next few weeks after that it was great. But we forgot that middle managers are still gonna manage, and that you have to fire them or they're still going to be there, you know, managing stuff, because that's all they know how to do.
And with the right tools, Scrum can be turned into the perfect surveillance state. (I stole that from another response in this thread, thanks).
Intriguingly you've not mentioned 2/3rds of the people in a Scrum team - the Product Owner and the Scrum Master - both of whom are supposed to be the ones who handle many of the problems you list. Build servers being on fire feels like a perfect example of an impediment. People who aren't actually developing anything feel like the perfect example of people who shouldn't be at standups, and certainly shouldn't be speaking at them.
Again Scrum relies on empowerment, with the Product Owner being the "single wringable neck" whose career is tied to the project and who has the buyin from management to get stuff done. The point being that it is in their interest to have a happy and productive development team doing what they say.
Finally there's also a really good point about internal and external metrics. I've always been a fan of management getting the stats they ask for, not being provided with access to the team's internal metrics which should really be short term to address perceived issues. If we count defects for a few sprints to reduce our defect count that is good, providing them for evermore to management is bad. Separating the two out lets your Scrum Master / line managers have a good conversation about what management are concerned about and how to effectively measure it.
As you said - bad managers won't do any of this, but then flagging up that you have bad managers is A Good Thing (TM) overall.
The problem is that, to them, "Agile" is Candyland: No responsibility or accountability for their own job (making stable plans/requirements), but hyper-accountability/scrutiny for the devs that have to spring to their every half-baked whim. It amounts to a DDOS attack on the dev process. It's just waterfall without the top part of the waterfall where they are responsible for requirements. I'd rather go back to waterfall.
Yeah, this nonsense isn't a problem of agile. It's a problem of people who don't know what they're doing, on both sides of the equation.
Which isn't to say that it's not a problem for agile -- it definitely is. I'd say that it's analogous to Gresham's law: when people can't tell the difference (until it's too late) bad "agile" drives out the good.
IF teams are using Agile like that there doing it wrong. It's meant to be recipe for collaborative, relaxed, friendly environment. Not a pressured hot house. If your goal is to use Agile to push developers to their limits, you've already failed the Agile Test.
You can twist Agile, based on your vision of good
team is. So you have to get the right people to implement it. You have to approach it from the idea of it being a collaborative, friendly, no blame etc for it to work.
I've only ever used estimates in story points to prioritize things(That's why its so abstract), not to monitor and push developers. If I wanted that, i'd use actual time.
The idea of everyone is junior is also bad implementation. It means even the most junior person is capable of contributing using their solution over a senior persons solution. The right idea wins, not the right person. I can understand why this pisses of senior people though.
IF teams are using Agile like that there doing it
wrong. It's meant to be recipe for collaborative,
relaxed, friendly environment.
I'm at a place where (IMHO) scrum works reasonably well. But some of the scrum language I hear people using kind of invites misunderstandings.
For example, "sprint" evokes runners running 100m at a pace they can't sustain for minutes, let alone weeks or months. And "committing to stories for this sprint" implies it's some sort of failure or emergency if the target is missed - when in truth nobody should be skimping on testing or staying late to get things done.
Yeah, I feel like the language is one of the worst things. A "scrum" is a bunch of sweaty men moving down a field in a pile. The idea of that makes me want to get a different job. I'm attracted to programming because I love sitting in chairs, not engaging in violent pile ups.
I agree with what you said about "sprint," as that is the second worst word in the canon.
And user stories is trying to change the term "feature" so that developers will be more user oriented in their thinking, but the term "story" is disturbing.
I forgot the other one: "epic." I can't put my finger on exactly what it is, but it really disturbs me. It's just too serious sounding, like it's presumptuous to declare any component of your program as "epic" before it is done. I can think of very few software systems or components that I would refer to as epic.
Are you writing the story of the flood? Is it the oddysey, the iliad, the history of your people? No, it's just some crappy new feature that's going to take your team a few months to complete. That is totally not epic.
> It's meant to be recipe for collaborative, relaxed, friendly environment. Not a pressured hot house.
The vast majority of work environments cannot be described as relaxed and friendly (hell, most of them probably aren't all that collaborative). Far more could be described as pressured hot houses.
That's the point. Managers often use management techniques that encourage that. It works if your on simple assembly tasks. I.E Worried if you gonna be fired if you don't make 200 widgets.
Doesn't work if you're on knowledge work.
A lot of people approach agile as its an extension of those approaches. When in fact it was a reaction to it, by trying the opposite approach.
One thing about Agile that I don't see mentioned much is that it provides a way for bad managers to marginalize the effects of low performers among their groups, though at the expense of high performers.
Every company has "cowboy" programmers who break shit and ruin their coworkers' lives with massive 1am check-ins, and a lot Agile seems designed specifically to put up road blocks to prevent this from happening. Managers love it because it provides an escape from dealing with problem employees directly. Most of the vehemently anti-Agile people I've worked with were grossly overconfident in their abilities and difficult to work with, which I think supports this idea.
Unfortunately, the only way for this non-management technique to be effective is to do it dogmatically with no deviations (otherwise the cowboys will always have some excuse for why it shouldn't apply to them), and this is where things start pissing off the actual talent.
The micromanagement - if management doesn't trust you to get on with work without having to justify what you've done every single day, why did they bother hiring you in the first place?
People aren't perfect. It's easy to get distracted, get off track, think that you'd be able to get back on track if only you could get your head down for a couple of days, and suddenly two weeks have passed, you're miles behind, and the people waiting for your work are all thrown off track because it's not there yet.
Spending 15 minutes per day making sure that everyone knows how the team is doing strikes me as a very easy way to avoid that.
Heck, it can be significantly less than that, if you use a tool to keep your tasks up to date and can just say "I got the frobulator finished, and moved on to the doohickey. Dave's giving me a hand with a small holdup there, and it should be finished on time." at the standup then the whole thing should take less than five minutes total, except when there _are_ issues that need to be dealt with.
I think that standups are one of the best parts of Agile, if they are time boxed and moderated well ('let's take this [larger issue] offline' should be heard fairly often). Super easy to implement, and very helpful. I've only worked with small teams (less than 8 folks), but I don't think it'd scale well.
It's not meant to be for that. It's meant to be let people know whats going on, because someone might know something that would save you time. I.e Collaboration.
Agile is like CrossFit: the techniques are sound, but the techniques really aren't the point. The point is to create a revolution by wrapping sound techniques in an us-vs.-the-world ideology that has strong emotional appeal. This makes it easier to sell trainership certifications, which are where the money is.
Agile done wrong destroys more than Agile done right can fix.
If you feel too good, or too senior for a programming methodology like Agile, then you are bound to not like it. Is that a fault with Agile, or a fault with your attitude?
A cowboy coder is not micro-managed, does not seem to need atomized tasks, does not need a direct line with the customer and decides on the complexity of the problem and the solution all on his own. They are also terrible to manage and hit-and-miss when it comes to actually shipping something with business value.
Bluntly spoken, Agile is there for the project managers, not the project developers. Agile should be implemented for as long as productivity increases, customer feedback loops create desirable features, and iteration cycles become tighter and shorter. If you know a way to increase those stats without Agile, then write your own methodology (http://programming-motherfucker.com/ is taken) and join management. If you don't care about those stats, and feel too good/senior for Agile, then start your own company and proof it to yourself. You don't start to measure a project's success by first counting the number of developer complaints. You could probably keep the complaints at zero, and never ship anything of impact.
If you get stuck in an agile environment, here's a survival guide:
1) Attack the lowest-priority and easiest stories to shoot for quantity, not for quality. This way you'll be closing stories in no time, pleasing the scrum master and product owner during the daily standup. Leave the largest and most complex problem for last - if nothing else, you can kick the can down the road by claiming it's tougher than it seemed, requires more points and should be moved into the next scrum.
2) Do not worry about the quality of your work. In fact, the lower it is, the more stories you'll have to fill in the next scrum - "widget for feature X" with some effort can become "fix bug Y with feature X", "improve widget performance when more than 1 user uses the product", "improve logging and monitoring of widget X" and tons of other, lower-priority, but higher-quantity stories (see point 1 of this). You'll likely be closing a story a day, earning points way ahead of the other suckers who choose to focus on long-term critical design/architecture issues.
Reading over his points make me think the OP is a PM who doesn't understand Agile nor his role in it. The number of misconceptions he presents reads like someone who has never bothered to research agile, but only knows what consultants have taught him.
I think the number 1 misconception is the sprints. He seems focused on 2
weeks as being the requirement, when it's on the recommendation. Some team have gone as long as 4 weeks. The whole point to a sprint is that dedicated time to a delivery with fast feedback from the customers. It's mainly so you can fail early, ail often. It's much better to spend 2-4 weeks building the wrong product than 6 months.
Second, his claims that product development, research and design, and architecture don't belong in a sprint are also misinformed. The focus on user stories to remind people that they need to focus on users, not gold plating. In any technical project, there are a plethora to improve on the design. People can (and have) sat down and refactored and "rearchitected" for days and days. At the end, they build beautiful systems that don't offer any advantage to the customers, or have implemented a number of features the customer didn't want, or have fixed a series of concerns the customer wasn't concerned with. There is give and take of course, as the engineers needs to give feedback to help the customer understand the benefits of the feature or risks of not fixing the issue, but those need to happen first before people go off and construct digital masterpieces.
Third, micromanagement. If you have endless meetings in agile, you're definitely not doing it right. The 3 big meetings, planning, retrospective, and demo, are ones that take the most time. Standups take no more than 15 minutes. The other tasks, meaning, backlog grooming and prioritization should not involved your sprint team. Those need to be done offline by the product owner. Agile may be about teams, but you still need strong leadership.
The only point of his that has any relevance is the last one, where consultant firms have made big bucks spinning out variations of agile because people just can't understand how simple it is.
Agile development is another tick in the long series of desperate attempts by management to avoid The One Thing That Actually Usually Works™: Hire great developers, pay them a ton of money and give them a lot of freedom and time.
I don't see how this makes sense. If I hired a bunch of great developers at great cost I would have even more incentive to maximize their output than I would if I hired a bunch of mediocre people at a lower cost.
> I see right away one Agile defect: It prefers near-term "user story success" over long-term investments like creating debuggable architecture.
Neither the Agile Manifesto nor even the far more specific Scrum Guide even refers to user stories.
A particular organization using Scrum may decide that architecture-related concerns are not important in the definition of "done" for items in their product backlog, and focus only on "user story success", but, to the extent that that's a problem, that's not a Scrum (and much less, Agile) problem, its an organization problem.
Agile is about what concerns you favor in figuring out how to get work done, Scrum is about how a team gets work done and defines the work it is to do within the framework provided by the wider organization, but the specific definition of quality standards for work is not directly addresses by either.
> Why spend time now making things easier for the rest of the cycle if it's not a priority in this iteration?
Why wouldn't an organization have maintaining (or improving) the overall architectural standards of the work a priority for every iteration -- in Scrum terms, part of the Sprint Goal -- unless there is a conscious decision to take on technical debt to meet some other goal? Agile, or even Scrum, doesn't tell you what goals to set, and you shouldn't blame them for you choosing to set the wrong goals.
If you're doing SCRUM, hopefully, when planning, you're building in slack for the devs already. Your sprint commitments should always be conservative. If something ends up taking more time than you planned, then, it'll just take up some of that slack.
With that slack, then, when things go right, you have time for dev priority tasks. You hit your sprint commitments two days before the sprint ends? Great! Time to work on some of those long term investments.
You can also scope stories based on dev priorities. You want to add a feature to part of the code that is in need of refactoring? Scope the story to include that refactoring.
Etc.
If using Kanban or something, it's a bit harder; you can easily take time for dev tasks (since you're not making commitments), but finding the right balance of taking them on but still getting feature work done is harder.
"Agile and Scrum, however, single out and humiliate anyone who works for 2 weeks and doesn't have something to show for it."
Holy crap, how wrong can some orgs go? The team should own the process. If there is R&D that does not match with scrum then the team should have a) a r&d framework or agreement in place b) the authority to say this is r&d.
The process does not own the team. The team owns the process.
I always try to differentiate between Agile and agile. Agile (proper noun) is the overly defined practice of scrums and points and backlog grooming etc. agile is the practice of continuous iteration at all levels. Agile is a great starting point, but if you aren't challenging the prescribed format and iterating on your process itself then you are missing the point of agile.
We have found, for instance, that the 3 questions of standup are pretty useless. They make each person feel like they are justifying their existence at the company. By dropping those questions and framing it in terms of a deliverable we found the meetings to be far more productive and better for morale.
We've decided to give ourselves 1 day into the sprint to better guage progress so we can define achievable sprint deliverables, as we don't often know enough information during sprint planning.
Each of these is set up as an experiment, designed to solve an identified problem with our workflow. And the results are great.
I don't know how the author is doing Agile, but it doesn't match my experience.
I've implemented "Agile" processes on four different teams, and while certain aspects were more successful than others in different contexts, nothing was at all like what is described in the article.
For starters, a very significant part of the article relates to performance anxiety over story points. Over 6 years of doing this, I've had two instances where people were called out for not putting points on the board, both of which came from outside managers not familiar with Agile.
Besides that, one of the core tenets of Agile is that you adjust the processes to work for the people involved. Trying to enforce some arbitrary subset of Agile processes on a team is a recipe for failure and frustration.
This reads to me like "bad managers are bad at managing Agile processes".
The most problematic part of SCRUM (and the first thing to start killing productivity) are the interminable meetings it facilitates. The SCRUM master is constantly trying to gauge scope on this or that feature, and everyone feels like they need to bring up what they've been working on during the standup in order to feel like they're not appearing to be slacking off.
I've abandoned SCRUM in favor for a Kanban approach that puts the onus on the Product Owner to manage and estimate scope and deadlines, thereby leaving us developers alone to focus on our work.
Its your prerogative of course but peppering your blog posts with stale meme gif after gif is an easy way to get me to not read your content. I don't understand this trend at all for talking about a professional concept. What you did this weekend, sure, but this?
Know your audience, I guess. I enjoy gifs, and enjoy getting a sense of the author's personality when I read an article. I don't think I've ever bounced from an interesting article because of a handful of cheerful gifs, but hey, that's just me.
The original (rather long) title is "Why do some developers at strong companies like Google consider Agile development to be nonsense?", and it seems it was shortened. There's a spurious "at" left in there now which breaks it.
The thing that I don't understand about the way Agile development is treated is that it is often treated like a silver bullet. As the article points out agile has use cases where it is effective but many treat it as a "Waterfall Killer".
Why do you need a capital-P Process? Does it need to have a name?
Best teams I've been on have been self-organizing, with a few senior engineers who did heavy-lifting and some less experienced engineers learning new stuff. Management was minimal (the more we had, the more things sucked).
- Minimize management. Minimize management. Minimize management. Management exists to set very high level directions and to remove obstacles.
- Make sure that different teams talk. A lot.
- Get real. Don't blow smoke up people's asses. Making a schedule with a granularity of a day on a six month project is fucking bullshit. Just stop it, okay? You might be able to do this a week out, but not much more than that. If that deadline is important, be clear on why. (I've been on projects that have been absolutely killed because someone felt they had to ship on magic date X, when shipping a few months later would have saved the product).
- If you are working steady 60+ hour weeks, you are doing it wrong. If you are doing regular 80+ hour weeks then you will burn out and leave.
Ha. Ever have a client? "Dear Enterprise Customer, We'll get it done when it's done. Cool? Can we have the gig?" Yeah no. That's not at all how the world works.
We have process because we have deadlines, commitments, responsibilities... and there needs to be an easy way for people to come and go inside of an organization without having to invent the wheel each time.
Dave Thomas (one of the original agile signer peoples), gave this amazing talk criticizing modern, commercialized agile instead of the developer-focused way of "developing with agility".
I've seen many criticism of agile/scrum as they (don't) apply to long-term or systems-software environments, but another disconnect I think people need to know about is with open source. Here's the "upstream first" development model typically used by Red Hat (my employer) and others.
(1) Company decides to devote resources to a feature.
(2) Developer writes code and submits patches upstream (e.g. to the Linux kernel).
(3) Upstream takes some indeterminate amount of time to get patches into a release.
(4) Company defines a downstream (i.e. commercial) release based on a particular upstream release.
(5) The possibly-pristine but also possibly-unrecognizable code from (2) is pulled from the upstream release into the downstream tree.
(6) More testing, debugging, tweaking, etc. on the downstream tree yields an actual release.
Just in (3) we see one part of the problem: upstream quite likely isn't agile/scrum themselves. Often they don't care about a particular vendors' release schedules; if they do, then someone would argue they're "dump and run" fake open source, and not a true independent upstream. They're almost certainly not going to give a crap about sprints or story points or anything else about the local agile/scrum process. These methodologies pretty much don't work unless everyone is completely committed, and with separate upstream/downstream that's just not the case.
Even with everyone on board, the disconnect between (2) and (4) is a problem. Pushing code upstream and pulling it downstream are two separate tasks, even if they're done by the same person, and there's this big gap between them. At best, this makes scheduling more complicated. More often, it makes agile/scrum style fine-grain scheduling and tight coordination impossible. Measuring "velocity" and all that isn't possible when there's so much externally imposed variability.
For something that's long term and systems-oriented and open source, these methodologies are practically useless. Actually they're worse than useless, as they misdirect effort and interrupt useful work. YMMV.
Any methodology in the wrong hands will be a terrible experience.
Let's face it. There is no methodology that will make all software development projects around the world go smoothly and fun to work on. If you work with, or for, a bunch of dicks, you won't enjoy what you will be doing all day. No methodology is going to change that.
I would like to go as far to say that "any" methodology used by great people will automatically result in great things.
Experienced engineers have seen a lot of "magic bullets" both on the process side and the product side. Generally they turn out to not be a substitute for hiring good people and getting good leaders to define a quality product and focus the team on what is important both short and long term.
If scrum metrics are used to identify low performers then that's so bad it's not even wrong. The whole point of agile estimates is to teach the team to estimate and pay attention to it. It's a learning tool for the team.
First off, Agile isn't a software development methodology, its a set of principles for an organization to apply in selecting/developing in its own software development methodology (you could call it a software development metamethodology.) Scrum is more of a methodology (and, while it may have emerged from the people involved applying Agile principles in the context in which Scrum was developed, and has features designed to support use in an organization applying Agile principles, there is nothing fundamentally Agile about Scrum -- or any other development methodology; if some decisionmaker is sold on Scrum by some consultants and adopts it as a top down practice, that organization may be doing Scrum, but they aren't doing Agile -- in fact, they are doing exactly the thing that Agile is a reaction against.) Almost any time someone uses the phrase "Agile/Scrum" they either mean Scrum or something that is neither Agile nor Scrum. In this case, the criticism seems to be of something that is neither Agile nor Scrum, though it may be related to the latter (but not at all the former.)
As to the specific points:
1. Here, the claim is: There is no place for an actual senior engineer on a Scrum team. The only way to move up is to become a "Scrum Master", a bullshit management role that involves responsibility without power.
There are several errors in this description: First, "Scrum Master" isn't management role, its a facilitation role focusing on expertise with making Scrum work in the particular organization. The only quasi-management role on (or related to) a Scrum team defined in the Scrum Guide [0] is the Product Owner (which does have authority to go with that responsibility.) Either of those is a role a team member could develop into; the idea that the SM role is the only "way up" on a Scrum team is incorrect even within the narrow bounds that are defined by Scrum.
OTOH, Scrum itself intentionally -- to support its use in organizations using Agile practices -- defines only a narrow slice of practice, particularly, a basic model for operation of individual development teams. It does not prevent having different roles outside of the defined interactions within the Scrum guide for members of different seniorities or specialties, it only requires that for the purposes of the interactions it defines, only a narrow set of roles ("Team Member", "Product Owner", "Scrum Master") matter. So, consistent with Scrum, there can be room for moving up on a team. And real organizations using Scrum on projects greater than a single 3-9 member team use complementary practices like the Scrum of Scrums [1]; being a Scrum team's representative on the Scrum of Scrums and thus playing a larger role in the coordination of the overall work of the project is one way for a technical contributor on a Scrum team to move into a role with
broader scope and exposure.
2. Here, the claim is "It's aggressively short-term, by design" and "if you carry this way of working on for months or years, you get a lot of technical debt and you get low morale."
It is certainly the case that Scrum, specifically, largely addresses tactical-level, short-term processes. This is because Scrum isn't a full-stack methodology, its a narrow set of practices designed to adopted as part of a larger set in an organization working in an Agile manner, and to be relatively neutral to the other practices. Certainly, if you apply Scrum alone with no higher-level strategic processes that set standards that constrain the definitions of "done" for work done in sprints to avoid technical debt, there is a lot of potential for an organization applying only what is in the Scrum guide to lack long-term focus and development technical debt.
3. The claim here is "It has no regard for the programmers' career needs or desires."
Agile, as a set of principles, certainly does; Scrum, as a particular set of practices, does not, for much the same reason addressed in the previous point. Largely, though, this seems to be a restatement or alleged further consequence of the first complaint about lack of a way to move up (it essentially seems to be, "because of #1, Scrum leaves me nothing interesting to put on my resume"), and so the response to the first complaint above largely addresses it.
4. There's a lot here that deserves separate responses, so I'm going to break this one up.
> "It's micromanagement."
No, Scrum is the opposite of micromanagement, since the "management" is done almost entirely by the team.
> "User stories and backlog grooming are there to control what the engineer works on."
Sure, those things exist to define work, but they are done by the team, so can hardly be viewed as micromanagement of the team.
> "The absurdly frequent meetings (and so many different kinds of status meetings!) are to intimidate him into not slacking."
Scrum defines four kinds of meetings: the Sprint Planning meeting, the Daily Scrum, the Sprint Retrospective, and the Sprint Review. Of them, the Sprint Review is the closest thing to a "status meeting", though even their the focus is demonstrating the concrete deliverables for the Sprint rather than status reporting. All of the rest are primarily action/planning meetings. "Status" of a sort plays a role in those meetings as an input to the planning aspect, but if they become status-focused meetings rather than planning-focused, you've definitely lost the defined purpose of the meetings in the Scrum guide.
> "The story points are there to track productivity (in some superficial, inaccurate way)."
Story points (in methodologies where they are used; they aren't part of Scrum) exist as a planning tool for the team, not a productivity measure.
5. The claim here is "At identifying low performers, it has an unacceptably high false-positive rate."
Scrum (and, a fortiori, Agile) is not a process for identifying low performers, or for performance evaluation at all -- its completely out of scope. Its not that it is good or bad at it, it doesn't even address the issue.
6. The claim here is "It punishes R&D, and it hurts the best engineers the most" because "Agile and Scrum [...] single out and humiliate anyone who works for 2 weeks and doesn't have something to show for it."
Research is a separate domain than product development, and there is no reason that the use of Scrum as a product development methodology should have any impact on how research is done. Of course, Scrum is a generic enough process that you could use it to manage research efforts as well, but the backlog items, associated definition of "done" for those items, sprint length, etc., would all likely be very different for a research effort than a product development effort. You could mix research items into the backlog of a team also doing product development, but then you probably have a problem defining work items that are appropriate for the product development sprint length. (This is less of a problem for non-Scrum methods that use a flow method for backlog management rather than a timeboxed Sprint cycle, because then its easier to mix items that have a longer-than-usual time involvement into the same backlog; the sprints used in Scrum place an effective upper bound on the size of work items.)
7. The complaint here is "I have actually seen it kill companies" backed by a single non-specific anecdote and the assertion that it generalizes. From the description, this appears to be a top-down all-at-once conversion to a new defined methodology. If one has processes in place, for the same reason one adopts incremental change to a software system where you've got an existing, functional system unless there is some reason you can't do incremenetal change, the same thing should be done to a software development organization. Failure of an organization (or many organizations) to manage a big-bang change rather than incremental change doesn't invalidate the methodology they sought to change to any more than failure of an organization to manage a big-bang software system replacement invalidates the technology they sought to change to.
To go through some of the points I fundamentally disagree with:
"There is no place for an actual senior engineer on a Scrum team" - There absolutely is, who is peer reviewing the code, working with the BAs to design the architecture, identifying the technical dependencies in the backlog? Scrum is not a methodology for writing code, it is for getting a series of tasks done. All the things a senior dev needs to do on top of code can be broken into tasks, Scrum adds a weekly "is this still the most important set of things to be doing?" meeting into the mix.
"It's aggressively short-term, by design" - The whole post talks about 6 week projects. Scrum is terrible for that as it imposes far too many meetings. Scrum works for the complex space where you say "12-36 months" as your gut feel estimate. It is a reaction to the waterfall world where you'd discover 2 years in you weren't building what the business wanted.
"It has no regard for the programmers' career needs or desires" - Stories make for excellent opportunities for juniors to "own" bits of work from initiation to completion. Old school projects provided very little opportunity for juniors to work on requirements gathering and thus a big chicken and egg problem for promotion. It also offers plenty of opportunity to interact with your customers, which is almost always a good way to get promoted.
"The story points are there to track productivity" - Oh noes the people paying you want to know how long the project is going to take to be done. The talk of many status meetings seems decidedly anti-Scrum and is an excellent example of the sort of thing that people should be screaming about in retrospectives.
"It punishes R&D" - Ironically it can be a really useful opportunity to conduct R&D. Scrum makes you have something measurable and a time frame in which to conduct it, which makes for an easier sell than an open ended request. I think what annoys some is when they get shut down too quickly, or are forced into taking a less technically interesting approach that delivers enough business value. Scrum isn't there for pure research into the theoretical, but then no-one is claiming it is.
Killing businesses - Much like every tabloid headline everything can kill you. Value destroying mergers exist in abundance, and poor leadership/management is a real killer. Plenty of companies do very well out of Scrum, especially when they are in a cost centre environment.
I might sound rather "no true Scotsman" when it comes to Scrum but there are only a few things you need to do to be "doing Scrum". What it does give you is a powerful question to ask, and a regularly mechanism to raise it via the retrospective - "why are we doing X when it isn't in Scrum?".
What is wrong with that sentence? TFA focuses far too much on the coding aspects of Scrum by asking "what does the senior dev do?", my point is all the things a senior dev does are fit to be tasks or stories in Scrum. It's outgrown being developer focused, and even more literally writing code is a small subset of "software development".
Points don't have to track time intervals to give you a sense of progress. Averages are a powerful thing, hopefully management is focused on long term graphs not getting het up about the number of hours difference between your tasks for two similarly pointed stories.
Oh and Scrum makes no requirement re using story points - it only requires that backlog items be estimated. We estimate tasks in hours and stories in points for instance, Scrum doesn't care. All our Product Owner shows to the stakeholders is a burnup with just numbers on the x axis.
We recently brought on a couple of PM consultants who are deep into Agile as a methodology, and decided to give it a go -- it's been about six months now and productivity has basically ground to a halt as we spend more and more meeting time endlessly sorting tasks into different little arbitrary piles instead of actually making forward progress. It's made communication with the non-software part of the company much more difficult, because they keep getting hung up on the jargon; we have to slice up tasks arbitrarily to get them to fit into "sprints," which go out of date minutes after the morning standup when the first non-planned bug report lands. Sizing is frequently an exercise in post-facto book-cooking: 'How long will that bug take to fix?' is like asking 'How long is it going to take to answer 23 across in next Sunday's crossword puzzle?' It might be fifteen seconds, it might take all day; until I have a chance to look at the puzzle I'm just pulling numbers out of my ass. The short timeframes encourage everyone to twiddle around the edges of things instead of doing real feature development; the arbitrary milestones serve mostly to make everyone feel like they're constantly falling behind.
I've discussed this with a bunch of people. Everyone seems to agree that we're doing Agile wrong, but even the strongest proponents of the methodology seem to be in complete disagreement on what the "right" way to do it is, or how that would be an improvement over the way we were doing things before.
So yeah. Not a fan.