There is a whole industry of selling "Agile" to corporations that is peddling this shit.
If you are implementing a set process from a book you are by very definition anti-agile.
When I implement agile at companies I don't even call it agile. And the only initial practices are:
* transparency -- you can't fix what you can't see
* improvement loop -- you need to learn from your mistakes and feed the results into next cycle
* process building fundamentals -- once you learned you need mechanisms to preserve the results. Checklists, templates, documentation, automation, infrastructure as code, etc.
Everything else is result of the learning. As we go, we spot problems or improvement opportunities and we decide how to fix it together. A lot of these fixes come from agile literature. Nobody said you have to reinvent everything. But now that we have implemented it in response to our actual problem we know why it is there which is way better than just implementing it without understanding.
Along the way people learn they have power to change things in a realistic way and not only they are welcome to do so but it is actually their job. Which is exactly the goal of all of this and yet completely missing when you put forward a book and say the process has to be implemented exactly as in the book.
Nothing has resonated more with the way I work than this.
Plan, act, reflect in practical terms. Build, measure, learn in lean terms. Vita activa, vita contemplativa, and sensemaking in academic or philosophical terms. Alistair Cockburn, Allen Hollub, and Bonita Roy in software terms. It’s all a cycle.
It doesn’t matter which page from which of those playbooks you take from on any given Sunday. They are remarkably consistent.
The observation I’ve made is that friends who succeed at this deftly avoid agile cliches and their teams are focused and efffective while checking all the boxes from crystal clear or spolsky’s list or accelerate.
This points to the fact that agile has become the cargo cult dogma it set out to replace twenty years ago.
What you described here is perfectly consistent with scrum, which Holub and Farley ridicule.
> feed the results into next cycle
Right, so you have a concept of cycles; in scrum these would be thought of as sprints (larger cycles) or single days of work (smaller cycles)
> transparency -- you can't fix what you can't see
Yep. In scrum, this would would be addressed by daily scrums (transparency and coordination on a daily basis), sprint reviews (transparency about the finished cycle), sprint planning (transparency about the next cycle), and sprint retrospectives (transparency about the way the team works)
> improvement loop
In scrum, that would be sprint retrospectives with a commitment to attempt to address at least some issues raised during a retrospective in the following cycle.
> once you learned you need mechanisms to preserve the results
In scrum, those would be backlogs (for the product and for the sprint). The practices specific to software engineering, such as automation, infrastructure as code, etc., fall out of the purview of scrum and will be picked up by other engineering practices (CI/CD, version control, etc.)
The problem with Scrum is people talking about how this should be more Scrum, or how this is not really how Scrum is/defines things/should happen.
Rather than focusing on the team, its own alchemy, its work and the very specific nature of the work.
The process should adapt to the people and the work at hand. Not the other way around. That's the very ethos of the Agile Manifesto. Scrum is used in corporate environments as the exact opposite.
If you replaced the word "scrum" with the word "agile" in the message above, would you still advance the same argument? Would you say that the problem with agile is that people are talking about how this or that should be more agile; or how a company isn't really agile; or how the way it does things violates the values or principles of the Agile Manifesto? If you would, then why does it seem a problem for you when the same applies to scrum?
Scrum, as defined in the Scrum Guide, is perfectly consistent with the Agile Manifesto. It is one of the ways of putting the manifesto into practice. Scrum's framework clearly promotes interactions between individuals, customer collaboration, working software, and response to change. If individuals, who, according to the manifesto, are more valuable than the processes, decide that the processes defined in scrum aren't working for them, they are always free to reject scrum and work out another set of processes. But if a team doesn't yet know what processes would work for it, scrum is as good a place to start as any, to actually feel what enacting the manifesto actually involves.
No, it's not. It's a very bad place to start. Because Scrum is a fixed process, with a fixed meaning.
If you started with Scrum and then stopped doing sprints because they weren't working for your team, you would quickly run into all sorts of opposition around "we're not doing Scrum properly; we need to be doing sprints".
Whereas if you started with nothing, no process at all, and later tried adding sprints, everyone would evaluate sprints by themselves with no preconditioned notion of how that should work within the rest of the process.
I completely agree. The reason I think it's a great place to start though is because this fixed process, with its fixed meaning, is well suited to the work involved in building new products.
> If you started with Scrum and then stopped doing sprints because they weren't working for your team...
...then you would stop doing scrum.
The objection "we're not doing Scrum properly; we need to be doing sprints" would only make sense in the context where you would be pretending to continue to be using scrum after throwing parts of it away. The moment you stopped doing sprints, you have made a decision that scrum as a process isn't working for you, and have switched to something else. You shouldn't keep calling it scrum to avoid confusion.
>> It is one of the ways of putting the manifesto into practice.
> It is not because it specifically mandates a given methodology imposed upon a team.
You are misreading what I have written.
Yes, Scrum specifically mandates a certain set of practices. However, all these prescribed practices are aligned with the agile manifesto. The manifesto never said be anarchic. The manifesto never said do not follow any processes in your work. If that were your understanding of the manifesto, you would have to object to all and any processes that have any predefined structure to them. Code reviews? Nope, that's a process. Pair programming? Nope, that's a process. TDD? Nope, too rigid. And so on.
Scrum is A way to put agile values and principles into practice, not THE way. But it is certainly not antagonistic to the manifesto.
> And Scrum might be a good place to start with in theory, while in practice, it's become in many corporations as the standard way of operations.
I am re-reading this sentence, and the previous two paragraphs, and just can't connect them together into a coherent thesis. Scrum is a good place to start in theory? — Well, yeah; that's what I've been saying here. — But in many places Scrum is practiced atrociously? — Well, yeah, don't do it atrociously then; do it properly.
I think it is worth to note that there is nothing wrong with the scrum techniques.
Just like any shoe you find in a store might be fully functional and useful to somebody it might just not be good fit for you.
Now, that might not be a problem for a more traditional development process of the past. In the past we would just choose one of the prescribed processes that was deemed the most fitting.
But the agile came and basically said that any set process is by definition wrong. The goal was to pursue constant improvement, forge the process as you go along and the idea was that the company that can improve faster will, over time, prevail over competition (you can't say "win", because it is an infinite game). And that the faster you can execute the iterations of these improvements the faster you can improve overall. And that the higher management is usually ill suited to do this because it just takes too much time to get the improvement iteration and that they usually don't know the specifics anyway.
And out of all this came the idea that the team itself should be responsible for improving their own process because this is the only way to do it fast enough.
In practice when the team notices a problem I may offer a possible solution for discussion and this solution will frequently be something already known from typical "agile"/scrum/kanban literature. Again, not saying it is all wrong but it needs to be viewed as a collection of solutions to individual problems rather than collectively a solution to fixing development process.
The point of agile is to teach your team to solve problems. To build ability to spot whenever something can be improved and then understand what is expected of them and the tools at the disposal.
An "agile from a book" is just a set of answers to some common problems. Your team will be none the wiser about how to deal with other problems that were not addressed by the book author.
The problem is not that Scrum doesn't have these things. The problem is that Scrum is a heavy and rigid set of processes that, besides a few useful things, adds a lot of unnecessary overhead.
Daily Scrum, Sprint Review, Sprint Goal, even the role of Scrum Master - depending on the team, some or all of these things can be removed to the benefit of the team, but Scrum itself doesn't allow it. It's either all or nothing, which is why I believe Scrum itself is anti-Agile.
> Daily Scrum, Sprint Review, Sprint Goal, even the role of Scrum Master - depending on the team, some or all of these things can be removed to the benefit of the team, but Scrum itself doesn't allow it.
Of course it doesn't. It wouldn't be scrum if these were removed; it would be something else. Which is fine; but if you prefer to organize your work in a way that isn't scrum, then why would you call it scrum?
> It's either all or nothing
Isn't it the same with everything else? An agile manifesto, but with only one value rather than four? TDD but with tests written after the code, and without refactoring? And so on?
When I work with a framework, let's say Ruby on Rails, and I don't use the build-in ORM, I still use Ruby on Rails.
Agile is an idea, a concept. Scrum is a tool (or even a framework, according to its website), and for a tool that's associated with a concept of being agile and flexible, it's incredibly rigid.
Everything in my experience aligns with what this post states. The best teams had processes that they developed themselves, organically.
However, there is a very real, pressing need from middle management to answer “what’s going on with X” when upper management asks. “I don’t know” is not an answer and neither is “we don’t know when it will be done.” These conversations ultimately control the flow of funding. So we make up these elaborate games to create guesses for these conversations. Sometimes the guesses are close, other times not.
I don’t know what the answer is, but there exists a gulf between what developers and managers need. Both needs are equally valid.
Simple: it's the job of a good line manager to embed and ingratiate themselves well enough with those doing the engineering work, that at any point they can give a reasonable estimate for how long a certain thing is going to take when asked middle or upper management. That is, WITHOUT adding extra process.
> I quoted the First Law of Mentat at her: ‘A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it.’
I spend the majority of my career in SCM, not software development. Your comment is absolutely true in my opinion. A good line manager knows what's going on, any KPIs or dashboards just make it easier for a manager to know.
Managers rellying on dashboards and reports only to know whats going on in their teams, have always teams oerforming sub par. And they aren't able to answer upper management questions any better.
This. The people paying for a project want to know how long it will take, how much it will cost, and what the feature set is going to be. This is of course not reliably possible, so we end up with an elaborate back and forth that tries to re-impose waterfall whatever the label on the process is.
Similarly, "Individuals and Interactions over Processes and Tools" is really difficult for an organisation to cope with. Everything else in corporate land is about de-personalising and standardising the actions of workers as much as possible, so that they're interchangeable. See manufacturing lines, fast food production, call centres. "Cattle not pets", but for humans rather than servers. Once you start caring about and adapting the work process to individuals, they're not replaceable.
On the other hand, there's comfort in having a bad process. Because then you don't have to blame and confront individuals as people, just over whether they've followed the process.
I agree with your concern. What’s the alternative? One idea…
> the people paying for a project want to know how long it will take, how much it will cost, and what the feature set is going to be.
Certainly … if they’re “doing it wrong”. :-)
Do VC know any of these things? The agile approach, YC, and the SAFE are predicated on these being unspecifiable to any signficant accuracy.
So a better approach may be to:
1. Change the money process from budgeting to investment.
2. Fund capable teams of dedicated fixed capacity, and you can nail budget to the penny.
3. Then, fund by mix of ROX/ROE/ROI* on prioritization of the backlog, and you’ll get out of the team what you chose to fund as the team capacity.
Done.
* Return on Experience (NPS etc.), Return on Equity, Return on Investment. And consider sharing something like “Implementing Beyond Budgeting” with the money people:
** To maintain trust with the money people, engineering cannot employ unproductive teams. It may be the tradeoff for engineers from a terrible budget driven process is better job security, since their individual performance or abilities to output great features matters less to the corporation than the estimation/budgeting/CYA processes do. In the budgeting world, senior executives get rotated out faster than IT teams.
What's missing here is that this doesn't come for free. Requiring to always be able to get information about "what's the progress" without delay means that someone must constantly calculate the current progress. And this is difficult and requires a human that can estimate all the things, including resources (will a dev get sick next week?) and delays from other teams, external blockers and their chance of happening (will the clould environment fail again this week?) and so on.
If this is desired (it may well be) a resource for this has to be planned and implemented _in advance_. That costs a lot of money. And even then, it can just give an estimate in the form of a probablity curve.
If the management has not planned for this, they can either not expect to get status information all the time but have to request it, then wait for it to be calculated and have to accept it will delay the project by the time it takes to calculate it. Or they have to live with an extremely vague and unprecise estimation.
And unfortunately, all methods, be it SCRUM, v-model xt or a randomly assembled method, will not automatically calculate a good estimate. Otherwise no one would complain. :)
The managers and executives need to have visibility into git commits and deployments. They should literally be able to see the front end and read the progress on the back end code, along with some simple reports as it goes.
If they can't understand that stuff they should not be making decisions.
But basically it's a list of stuff done, with proof, and stuff left to do.
Then they can see the progress bar moving and make decisions based on that. To get estimates, ask for explanations of what is involved in the remaining steps. Again, they should have technical competence to get a rough estimate from those reports.
> If they can't understand that stuff they should not be making decisions.
That doesn't seem right. As a developer, if I were the CEO of a company, I would not be able to read progress from the legal team, or finance team, or any other team. What's going on with that lawsuit? No idea. Do I know how long it'll take to wrap that up? Nope. I can't understand any of their jargon. I still gotta make decisions about it.
I don't need legal competence in order to run the company or make decisions about stuff, and I shouldn't need technical competence either.
It's on us, as developers, to communicate clearly and give answers and reasonable estimates, so management can make their plans and adjust priorities if necessary. The process we use to generate those answers (agile, git commits, deployments) is entirely up to us and could not matter less to the business, at the end of the day.
In a sufficiently large organization, managers are hired to have technical (or legal, or financial) knowhow and to use that expertise to provide information to the CEO to not take time from those doing the work. Likewise, those doing the work often aren't even permitted to communicate with the CEO.
In a small organization, the reality is that many hats must be worn. That may mean developers play manager to some degree, but also the CEO must be prepared to step into the trenches.
Yes, it definitely varies by org size. I worked in a medium size org and the “dev manager” could not even look at a git repo. In a years time he was never able to set up his dev environment, despite “wanting to get more into the code.”
I'm seeing a lot of people making Agile something it isn't/shouldn't be. There's a difference between core Agile (as in Agile Manifesto) and one of the many commercial imagination of it.
Agile wasn't meant for project management but instead as as a tool to organize work in development teams and respond to situation arising in such landscape. It was "weaponized" by business but (at least from my experience) everything that is wrong with it was brought to the table by those business practices.
Agile isn't for planning long term projects or managing projects portfolios. It isn't a communication method not a risk management one. There are dozens (if not hundreds) of PM methods implementing full scope of PMBOK's areas of concern that one should use for those. Some even use Agile as an internal process, but still plan using other tools.
Yet, I understand many of comments here. In many places I've seen Agile is misused if not bastardized to a level where I hated it.
But just to show the contrast: I've worked recently in an environment where there was no scrum, no sprints, no agile points, and environment was really allergic to any mention of Agile (Oh no, we don't use it HERE). There was a lead who talked to everyone on Monday and said "we want to have X by the end of the week, can you do it?". Every single person had their X and everyone worked toward that goal. That planning usually worked and each week ended with deploying next iterative version of the software. It was the most Agile place I ever saw.
The big mistake you're making is capitalizing the word "agile".
At $WORK we are agile as a software team, but we don't practice Agile. It's a subtle but important difference imo.
Unfortunately the word "agile" has picked up some baggage over the years that means to many folks the word translates into a rigid framework of practices, philosophies and ceremonies. This to me is the total antithesis of what being agile really means.
Maybe your software team is just efficient and you lack bullshit processes and broken organization that would prevent you from working.
At this point, we could very well stop using the word agile because if nobody knows or agrees on its meaning, it became a meaningless word.
I understand your point about agile vs Agile but we can't rely on people (not) putting the upper case A at the right place, and even on agreeing on the meaning for these words.
>Agile wasn't meant for project management but instead as as a tool to organize work in development teams and respond to situation arising in such landscape.
I wouldn't describe it as such. Such descriptions are how managers got the idea they can put development teams into their bubbles while restricting their access to assets required, demanding they go through multiple layers of management to get simple things done. Ideally, you don't just have a response, but the power to change things more efficiently.
Case in point: devops, CICD. Most teams would benefit from having their own CICD. Many companies silo a separate "devops" team and multiple development teams, while pipelining the demands of dev teams from manager to manager. "Devops" team gets swarmed by status pings every day, development teams are annoyed because trivial things don't get solved, management gets annoyed because "why is it taking so long?". And the team could easily teach someone the basics and get their test environment running.
Of course, above scenario requires management gives the power to change things to more people. Management in general thrives on lack of trust, so that's a no-no. "We love agile!", they say. Yes, when the teams get nicely silo'd and somehow become "more predictable", but are still as powerless as before.
I think the main thing about agile is a plan-do-study-act cycle than can be adapted to anything even project management. The trick is not to think I can just adopt a process and will then be agile. It always has to be adapted to context and what works for you.
As to CI/CD pipelines: I think a centralised devops team that blocks everyone is as much an anti pattern as a wild-west approach where 3 teams operate 4 different CI/CD tech stacks. There is much to be said about building a platform (when the size of the operation is big enough to warrant it) and using golden paths or paved roads.
I didn't say most, but many. And yes, this does happen in plenty of spaces, for no good reason other than management decision. Dev teams own the code, but they don't own the DB, the pipeline, the environment, etc.
It creates a lot of friction and bottlenecks, while keeping developers from improving.
Any company with a legacy ops/dba/it team? No. These people are now called devops because it’s trendy, and they have too much political capitsl to suddenly get rid off (plus such fun things as thinking devs are too irresponsible to touch prod).
Where I work bigger projects have some dedicated devops people to set up infrastructure build common images the whole team uses and sit next to people working on their pipelines to help. Smaller projects its all developer driven because there is no overhead and no ine to synchronize infrastructure with.
>I'm seeing a lot of people making Agile something it isn't/shouldn't be.
This is kind of expected. Agile was always very vague about what it was from the get go. At this point it's like "god" or "love" - there is no one shared definition and never will be.
Rather than arguing over what really is is or should be we should just be quietly deprecating and using a different name for what we think it should have been all along, making an effort to be specific this time.
XP was specific and good for the time it just... stayed at version 1.0 forever.
That last example, as you recognise at the end, was an agile flow, as per the Manifesto, they found what worked for them well. However they wanted to call it.
Any other flow in which management wants to standarized the process, fails at agility. I don't consider those Agile. Just because I call the fruit Orange Blue, doesn't make it so.
I agree this happens at places with a toxic management culture (one could argue that almost all management culture is toxic).
This can however work in places that are (truly) leadership focused where you A) fail fast and B) clearly outline your intention to accept failure as part of the process of improvement.
> if it doesn't work for a couple times, you get fired
That is .. not a reliable description of the outcomes for management consultants, who can inflict spectacular disasters sometimes and still come out ahead. Largely because they deliver the right magic ritual.
It's common building/engineering sense. It's no accident that (part of) the scientific method looks like that: it's the outcome of this precise process. It may not be optimal, but it delivers.
I think this is one of the biggest problems with the Agile manifesto. The actual line is:
> Individuals and interactions over processes and tools
This makes it seem like processes and tools are not important. But when you get beyond 5 or so people working on something, processes and tools are absolutely critical. There is no contradiction between processes and tools, and individuals and interactions. Better processes and tools will enable better interactions between individuals and will allow those individuals to influence the larger team.
What this should have been is something like:
Individuals and interactions <=> Processes and Tools
Both feed into each other and should be improving each other.
Not op but I’ve worked in Agile™ shops, places that are agile, and waterfall-y enterprise stuff. The biggest difference I’ve noticed between Agile and agile shops is that ceremonies or lack thereof make no difference. Pointing doesn’t matter, extensive grooming doesn’t matter, extensive planning doesn’t matter, sprints don’t matter, etc. Grooming your work does matter but 30 minutes is enough for a high performing team to hash out a week’s worth of work. Planning does matter but it should be kept to high level product goals and nothing more. Sprints should only be used if you have something you’re actually sprinting towards. Most of the time that’s not the case. Points straight up don’t matter.
All of this being said this puts a lot of pressure on product managers because it’s hard for them to express how long something will take. The key here is that even if you do all the ceremonies you don’t really know how long it’ll take anyway.
The only way you can work like this is if you have built up trust from your PM. If you can throw all of the shit away and give the PM what they want they’ll go to bat for you. For agile to work there has to be an incredible amount of trust that every team member is going to do what they say. And the lack of trust is when you get Agile™
> The biggest difference I’ve noticed between Agile and agile shops is that ceremonies or lack thereof make no difference.
The best project I ever worked on: one developer (me), one project manager on our side, one QA/test person at the client, and one manager at the client who was what we'd call a product owner on a scrum team.
No ceremonies, no pointing, very little backlog grooming. Shared Bitbucket repo for issue tracking. Deployments scheduled every Tuesday when their QA person signed off on the feature working as expected (this was in the days before CI/CD became common).
Worked great. Low friction, they got features and bug fixes, we didn't get hassled, and we could set up a call (or usually just a couple of emails) if we needed clarification or needed to explain why something was more complex to do than we or they originally thought. One of the best six month periods in my career, and I've been doing this stuff for almost 25 years.
I don't mind some aspects of scrum, but the things I don't mind (mostly around team autonomy and self-organization) seem to be the same things that are quickest to get thrown out when someone gets impatient.
So much easier with small teams. Scaling to say 1500 people is hard. Then you need some kind of hierarchy and explicitly architecture. My biggest problem with agile is that people think it replaces architecture with ad-hoc decisions. Threading in design work as spikes and regular stories needs about the same discipline as "processes".
That definitely simplified it. :) That said, we could (and for a few weeks, did) bring in a second developer with no issues.
My last job had me on a ten person developer team and I would not try to use that process there - although I'd consider splitting that into a couple of smaller groups focused on a specific product/system and using a very lightweight process for those groups.
Someone once said to me "the advantage of a team of one is that there's only one clique and everyone's in it". I think about that from time to time 20 years on.
The worst thing about Agile is the commercialisation that came with agile training coaches and certifications. It always boggles my mind when realising how many people think “agile” means sprints, backlog grooming, Fibonacci sizing or planning poker and not what’s in the manifesto.
When you take it back to its roots and cut out all the Scrum or SAFe nonsense and adapt it to your team it can really work.
Note, I’m not saying estimates are bad per se and can imagine teams for whom it does work, but it would have to be something that the team is happy with not an impose-from-above methodology…
The ceremonies and pointing are often a big waste of time. I worked at a place that really drank the Agile koolaid, and dreaded every "planning" and "retrospective" because it was so incredibly tedious. "Is it this 2 points or 3?" "What did you learn this sprint? What went badly?"
That's one of the reasons I work for a small company. There's just one boss, and he's also what in the US would probably be called VP, both formally and informally (there's a difference between the formal and informal hierarchy). He knows his limits, is technically adept although he doesn't know much about programming, and just cares about usability. As a consequence, there's no politics, no endless meetings (yes, I've done Agile too elsewhere, with those retrospectives, planning meetings and pointless burndown charts). There's no other team blocking me or forcing me to do something differently (because of course teams are self-organizing, but heaven forbid if you do something that touches the domain of another team). The salary is somewhat lower, but I actually enjoy my job.
Middle management is just nepotism, where incompetent/not-caring managers hire from their network to gain power.
>> When you absolutely need middle management, force them to also do some real work, if they can’t, don’t hire them.
That ia a mjor difference between Amazon and most other places I worked at. Amazon management actually did things beyond managing. There are some of the managers in places as well, they are a minority so.
The only practice that I've found to be reliably agile is simply having more one-on-one meetings and less large group meetings. Large group meetings tend to waste time for at least some of the people in the group. Often, if you get 8 people together, most of the conversation is between 3 or 4 of those people, while the other 4 or 5 are bored and disengaged. One on one meetings tend to be productive since the meeting would not happen unless one of those people needed to talk to the other person, so 100% of the people in attendance are needed in that meeting. (I'm excluding those companies that have mandatory one-on-one meetings, where the mandatory nature of the meeting sometimes makes the meeting as useless as a large group meeting.)
I've known managers who say "Group meetings are great because I get to meet with everyone at once and it saves me so much time!" But that only benefits the manager. Meanwhile some of the people in the meeting are just sitting there, killing time, bored, waiting for the manager to have 10 minutes to talk to them. By contrast, more one-on-one meetings, even if only for 10 minutes, allows everyone to be involved during the moments when they need to be involved.
When we criticize some of the rituals and bureaucracy that has become associated with the word "agile" most of the time we are criticizing group meetings. That includes the daily standup. I've run teams successfully by having quick one-on-one 10 minute meetings with everyone on my team, everyday. But I rarely feel the need to get the whole team together. In fact, when I'm leading a large team, I never need to get the whole team together, it is always some smaller subset that I pull together.
If you are the team leader, then you need to think carefully about who you need to talk to for any given purpose. If you have the discipline to only talk to the minimum set of people you need to talk to, then you are freeing up a lot of people to keep working on their real work, since they are not in a meeting with you.
I’m not a fan of ceremonies/rituals and always have to laugh when looking up the dictionary definition of ceremony/ritual which implies something that gets because “we’ve always done it” but is actually pointless. Pretty much on the nose. If something is useless, then change it!
Group meetings, the next exponent of that is SAFe (so called Scaled Agile Framework) where you'll see 2-day-long meetings with 80+ people. I am not a fan.
1. If the people on the team can't change the process, it ain't agile
2. Sprints aren't a failure of agile, they're a failure of devops. Once you start deploying multiple times a day, nobody will care when one sprint ends and the next begins and it'll be fine
3. Estimates aren't literally the devil, execs need to have a rough idea how big it is, just don't get more granular than months
Point 2, even with CI/CD multiple times a day, with zero flaky tests and a sweet build time and automated test time, it sounds like you still need almost zero time between the final git push and merge to $BRANCH causing a deployment.
What about review? Code review? QA? Showing off to stakeholders? These things add a delay, and there is a 'crunch' around the end of sprints where things are artificially urgent. I am not sure that is optimal. I have not heard a reasonable solution to this, other than abandon sprints maybe?
I think this stuff works wonders when at MVP stage and number of users < 10 maybe. For mature and complex products, the type I typically have worked on it is not ideal.
Obviously not all projects are the same, there are certainly pieces of software that would be disastrous to do that with, I'm just speaking generally. For reference, my (very not-startup) team is ~20 engrs in 3 teams responsible for ~30 services and we deploy stuff ~25x/wk.
Also a 'crunch' around the end of a sprint sounds like mini-waterfalls.
How do you avoid mini waterfalls? At some level of granularity you are doing one thing you committed to at a time. If someone else needs to check it thats a handover.
How I have seen sprints done is you commit to X Y Z, have a definition of done like “is in production” and then developer works on X, gets X code reviewed and so on.
If the aim is that the right amount of work is in each sprint then naturally the last day will be busy trying to get stuff reviewed.
Just do away with sprint commitments. Pull a new story off the backlog each time someone needs a task, and count how much you accomplished at the end of the sprint instead of guessing at the beginning.
That sounds sensible to me! Thanks. Nice to get these perspectives as I have never heard a colleague say that before.
I guess you can then get rid of the 2 weeks? Because you can measure velocity as a continuous rolling average. Retros could be continuous too and standups already are. Grooming is easy to make continuous too.
Try 3 week sprints. Or 4. ShapeUp by Basecamp recommends 6 weeks. Sticking to the same things everyone is doing is counter agile, and you should be experimenting and see what works for your team
Sounds to me that the artificial crunch is the problem. And if you have the capability to deploy quickly, I don’t see any value in sprints they tend to artificially delay deployments, reviews or retrospectives. If something needs talking about, don’t wait for a ceremony in a week and a half when everyone has forgotten about it.
I’ve been part of teams working this way without sprints and artificial crunch on mature and large scale services, so it really isn’t a MVP/startup only deal…
Separate deployment from release (Feature flags). Then you can do QA and show to stakeholders whenever in whichever environment. You can even use it for Code Review, though I prefer pair/ensemble programming to CR (better to correct things as you go than after the fact)
I mostly agree on this - my extension to #3 is that estimates are fine as long as when you inevitably sometimes miss your estimate, you're not held to the fire for not being a fortune teller and asked why you didn't get it done last month.
I'm also not sure I agree on your take on Sprints. In my mind, the value of sprints is that they provide a barrier where you should be able to go "we already loaded all our work for the next 2 weeks. If you want something done, we'll start it in N days." Done right, sprints are your defense against "can't we just do this thing".
re: estimates: This is equivalent to saying don't confuse estimates with delivery dates, which I wholeheartedly agree with. But you also have to be willing to say "The estimate was wrong, here's a new one" sometimes.
re: sprints: You don't need a barrier against "can you do this thing". If the thing is important enough, do it; if it isn't, don't. You work on whatever the top priority is, and your sprint commitment is just a list of what the priorities were, N days ago.
(Edit: you do still need a PO willing to say "doing the thing would delay the other thing" though)
> 1. If the people on the team can't change the process, it ain't agile
Absolutely.
> 2. Sprints aren't a failure of agile, they're a failure of devops.
I don't think so, here's the timeline: Scrum (90s), Agile (2001), DevOps (2009). So sprints have been around a long time before DevOps, the way I interpret it.
> 3. Estimates aren't literally the devil, execs need to have a rough idea how big it is, just don't get more granular than months
How? Your event horizon spans only the next couple of iterations, everything beyond that is pure speculation.
A subtlety of point 2 is that not all products or services benefit from multiple deploys per day. If each deploy creates work or risk for the customer, then beyond some point, frequent deploys may be harmful. Your customers may have more valuable things to do than consume or help QA the latest increment of your offering.
Separately from deployments, in some less-than-ideally run organisations, management may whimsically change its mind about what the number one priority is from day to day. Two-week sprints can shield the delivery team from this, by acting as some kind of low-pass filter on thrashing priorities, to create space to get a few things entirely done.
Sure. I worked on a legacy (monolith, >5 years old, no automated test coverage) thing and managed the teams that turned it in to a modern (microservices, ci/cd, full test coverage) thing. So, kind of a naturally occurring A/B test.
Five years ago, we deployed once per sprint, and it was a big deal: the test pass takes this many days, the change management team needs this much time, etc. So quite often, finishing a story on day N might mean it goes out on day N+4, but finishing it on day N+1 means it'll take two more weeks, day N+18. That's where the urgency came from: that "tax" of lost time due to infrequent deploys. So there was a lot of pressure to get things into a given sprint. Hence, sprint commitments.
Now, on the same platform but with frequent deploys, we still have pressure to get a story done by a certain day sometimes, but the sprint cadence no longer affects our ability to do that. It goes when it goes. Without that pressue, we realized that sprint commitments didn't serve much purpose, so w stopped doing them, and we haven't missed them.
That's what I was referring to - it's not the sprint that matters, it's the "tax" on deployment time, which varies depending on where you are in the sprint. Take that away, and you stop caring where you are in the sprint, like a teenager in summertime losing track of the days of the week.
All you have to say is "rolling release," or "why releasing according to
lunar phase is dumb." That you need to explain it the way you did is
another example of the bs job creation mindset, aka, agile.
The motivation to batch releases was largely driven by a desire to do full end-to-end integration testing of changes between a number of entangled backend and frontend systems. It often took days to stabilise all the components in the single pre-production staging environment used for end to end integration testing, then run through the test plan. Perform batched end-to-end testing of system was the process bottleneck.
Monthly releases created similar crunch dynamics of changes being delayed by a month if they weren't stable enough to enter the monthly test cycle. Since end to end testing is the bottleneck, you don't want to feed in low quality inputs (rushed changes that are likely to cause issues and then trigger re-tests of the entire slow end to end test process).
I'm not saying this is the best way to do this, it definitely isn't, but there can be some reasoning behind it.
> The problem is that Scrum started out as a lightweight wrapper around Extreme Programming (XP) to make it palatable to management, and it is all that extra cruft that people of added that is the problem.
Look at that statement, and then let’s abstract it and make it a bit more “polymorphic.”
The problem is that process ______ started out as a wrapper around idea/adhoc practice _______ to make it include management, and it is all the other crap that is added that is the problem.
Every time management gets involved with the teams I work on, this is what happens. I hear tales of people who have “helpful” management. Some of my managers have been the nicest guys, and tried to do well by me. But because they’re not part of the team, they rarely add value, but rather burden the team with their non value adding presence.
(Kinda ranty, I know, I’m just channeling the OP which was titled a rant, I guess)
A lot of this really resonates with me as a manager of dev teams, but once again the very unhelpful advice for "how do you figure out when you'll be done?" is "refuse to do it". If an executive is asking me when we will deliver something this is a completely valid question. It's how everyone outside software development lives. If lean aglie can't give me a way to shape expectation of what we're building AND answer the delivery question we're stuck with BS agile, or we might as well go back to waterfall which (accurate or not) has always been very clear with a delivery date.
> It is not the technology that matters, it’s interacting with people. The better engineers can communicate, the better a team will perform.
This pretty neatly summarizes my ~25 years of experience in the industry! With the additional observation that most developers and managers are not aware of this phenomenon... :(
> Don't estimate
I agree.
But in my experience, the real problem is out of the developers' hands:
1. On a project, my team worked hard not to miss any deadlines. it worked, but the project was delayed for other reasons, and the PO told his superiors that the project was delayed because of the development team anyway.
2. On another project, my team didn't need to estimate or have a deadline. BUT at every meeting the boss said the project was behind schedule. WE WERE LATE EVEN WITHOUT A DEADLINE. Total nonsense.
If your boss is crazy, your job has a lot of politics, etc, nothing will work, and this kind of environment loves to embrace Agile methods.
> Recruiting people does not work by doing laundry lists of questions asked by HR drones that don’t understand context or demand 10 years of experience in a technology that’s 8 years old. Five rounds of interviews, tests and exams where people have to recite algorithms by heart are pretty pointless.
I second that sentiment.
I've worked in places small and autonomous enough that the office manager would handle the admin of posting a job ad, passing on the CVs to the person who knew what their team was looking for in a colleague - note, a colleague, not a "new hire". The absence of an HR filtering layer left hiring decisions to those it most directly affected, but with the admin side handled by office managers (or in some cases COOs).
I've also worked in far larger places with heavyweight HR departments. Frustrating and backed by corporate policy (and as declared to shareholders) regarding recruitment processes. The question "Why can’t hiring be agile?" is worth exploring, I think.
Hiring is agile in China big tech. Over-hire, do it quickly, and expect people to churn within 1 day to 6 months. Annual turnover of >100% in the places I've been.
For the majority of people, they just want to know what to do and expecting them to “become agile” is like asking them to change their beliefs related to politics, religion, etc. — AND more importantly, practice them.
If you want to become more agile, focus on improving yourself and finding others that align to what you feel is agile.
____
* Personally, to me, Boyd’s OODA model of agility is the the most adaptable. Problem is it’s not a check list and requires you to internalize is. For example, most people think being agile is always about speed, but it’s not, it’s about dynamically controlling yourself in to optimally control what is not yourself; sometimes most agile thing to do is be slow, or even do nothing — to see what happens, force someone else to act, etc. People that to me felt the most agile are the ones that enjoy being agile all the time and playing agility games.
I'm always very confused about this kind of statement: "Don’t estimate". To me, projects without estimates tend to take way too much time. It's also federating a team to have deadlines. But I agree that estimations often become a joke when your manager asks you to estimate something that you have no idea how long it will take.
How to reconcile both? Being more flexible on the estimation? Taking them as a team goal and not a business one? How to manage the lack of estimation when company planning is important? I can hardly see how "Ok everybody, we are going to release a product but we don't know when! Soon!" could work.
Estimating is a bit of a joke though.
It's dependent on too many unknowns.
Example: How long will it take you to get to China?
It entirely depends on how you're getting there and where you're coming from. I can give you an estimate, but if the only known is that you're starting in North America, and you're going to be held to your estimate, you'd better give the estimated time it'll take to walk there.
The real issue isn't the estimating, it's that once you give an estimate, you're held to it even if everything around it changes.
Estimations only has value if you're not going to be held to it, otherwise people start fudging numbers. As a rough guide it can be useful, but too often it's used for more than that.
Perhaps it could be expanded upon with: Don't estimate, solve the problem.
Solving the problem may involve estimating, but blindly being asked "How long will this take?" without exploring the entire solution space, often without even being aware of what problem needs to be solved with that information, is when you get failed results. A time estimate may not even be the right tool to solve the particular problem.
If you are the one asking "How long will this take?" you have already recognized that you don't have a satisfactory solution yourself and need to defer to the experts, so why obscure useful context from those you are deferring to? While there may be some sense of pride in being the one to come up with the solution, such emotions aren't suitable for the workplace. It's okay to give someone else the 'glory' if they have the solution and you don't.
I've found estimations helpful when it's really about breaking down parts of a task that aren't independently deliverable, but still independently estimatable. When working with a sufficiently technical manager, it gives them the best idea of your confidence function. It also helps junior members of the team understand more of your process and get some problem-solving practice without years of experience.
estimate both a time and a confidence (or fitness).
2 weeks with 9/10 fitness is a very different estimate from 2 weeks and a 6/10 fitness. The latter should result in more discussion so those you're giving the estimate to can better understand how likely you are to miss that estimate and what they can do to facilitate a better estimate. "Give me 2 days to investigate this 1 thing that's a total unknown, I can probably give you a better estimate afterwards".
The problem is very few people on either side of that table ever want to do this.
The problem with this kind of article is that it doesn't recognise that software is built in different contexts. "Don't estimate" may be fine if your product is an SAAS that gets updated incrementally. It won't cut it if your clients question is "we've booked the O2 arena for April 2023. Can you build this by then?"
The problem is these clients have no idea what they are doing, they should be working out if something will be ready before booking.
If they are coming to you telling you the deadline and asking if you can meet it you know your going to have problems with them. Invariably they won't have a backup for when its not done on time for whatever reason.
Yeah. The point remains though- even if they haven't booked yet, at some point they're gonna book and then it's a hard deadline. There are even cases where it's valid to come to you with a deadline: can't move Christmas, for example.
If you have a fixed deadline you descope to something you think will take ten percent of the remaining time build it in what turns out to be more like half of the remaining time and then add enhancements until you hit the deadline.
It’s not hard. Maintain a priority list. Always work on what is #1 priority. If #1 priority is blocked then work on #2 priority. Use the priority list to negotiate with your stake holders. That’s it.
I have had a lot of success using this simple formula in my 30+ years career. Both as an individual developer and when managing small to large software teams. It’s simple. It works.
It’s not hard.
Maintain a priority list.
Always work on what is #1 priority.
If #1 priority is blocked then work on #2 priority.
I've described this same approach as "risk driven development." Identify the highest risk to an effort's success, address it, then the next highest, etc.
If, during this process, a newly discovered risk is identified as being more than what is currently being worked, put the new risk at the front of the queue and address it.
This drives risk to 0, which drives the probability of success to 1.
If you don’t have a good idea, don’t even start. The problem with a lot of projects is they focus on the process (because it’s easy) without having a good or clear vision to begin with.
If you have a clear and compelling vision, then everything else falls into place.
Start with no process and add (sparingly) as needed. Always treat process as the least important thing on any project.
Aren’t all business models at odds with agile (agile is one but hardly taught in economics schools, or?)? It sure feels like it, as soon as a agile process gets formalized towards business, it crashes. Execs wants metrics, which gets abused from both sides of it. And to create relevant metrics you need details which just adds to the bureaucracy and slows things down. As there are deliveries to consider and business contracts, management and sales wants hard deadlines to sell. Thus you need to estimate everything to give management an idea if and how relevant something is to do in relation to what needs to be done from a sales perspective. It always ends up in bureaucratic hell because the more formalized something gets, the more data it needs to be accurate and deterministic, which is a goal from management.
Countless times I had to estimate Jira "tickets" (actually called issues in Jira terminology) in hours. To this day, I fail to understand why and I haven't gotten a pertinent answer other than "the business needs it for planning, they're paying us, so you do it."
Yet this contradicts at least 2 Agile Manifesto pillars:
1. People and interactions over processes and tools (fostering the estimation ceremony + booking time rather than actual interaction or developer happiness).
2. Reacting to change over following a plan (plan-the-work and work-the-plan over reacting to change).
How do you even "do" estimates?! Clean Agile (ISBN-13: 978-0135781869) says "Estimates are never commitments. They are guesses."
So it seems the business is trying to get a commitment out of an estimation, at hour granularity.
Businesses want to be able to forecast and goals are often set aggressively and ahead of knowing whether they are achievable. There are three basic levers to wiggle at that point. Money, time and project scope.
Money can only really be applied early by staffing up, adding more people to a late project will invariably make it later. Thanks Mythical Man Month.
So for anything in flight that looks like it will miss a desired goal you can only really change the delivery date or reduce project scope.
But you also need to know you’re going to miss so you need some idea of how much work is left and how quickly you’ll get through it. Detailed estimates are a reaction to this along with the idea of measuring ‘velocity’. I’ve never seen either work well but people seem to cling to the idea that next time it’ll help them get it right.
One of the funniest Agile meetings I’ve been in had a producer explaining the state of the project. Slide 1: Velocity doesn’t mean anything outside of the team it’s for, comparisons don’t make sense. Slide 2: A line graph plotting each teams velocity against one another. Cue any useful discussion derailed because people want to know how we can get everyone’s velocity up to what looks like the best teams.
There’s a lot of dysfunction in business and a lot of winging it without really trying to understand why things are being done. You get a lot of odd practices spread around because project success is not necessarily because of “good leadership” but often project success is the measure used for leaders. A couple of wins and you can gracefully fail upwards believing in your own bullshit.
> And at the end of it, the estimate has to be thrown out anyway because the devs who picked up the story didn’t have the same experience as the devs estimating.
I thought the "rule" was a single core team, not multiple sub-teams of devs.
If devs with less experience are also at the planning poking table to voice their concerns; the more experienced devs can take that feedback on board to come to a consensus estimate.
Individual tasks may be over or under depending on which dev picks it up but the idea is to get the average as close as possible to being accurate.
I understand that 30 years of experience has to count for something, and maybe some people are fortunate enough to be able to work in an environment that is devoid of all the politics that "Agile" leaves in its wake.
My own years of experience have taught me that estimates will always be sought by the customer.
They may even shop around for a better price with a different software vendor/low code solution etc.
If the development team doesn't know the estimate, how is management expected to know? Simply put, management will be held to whatever number (of person hours) they provide, and are also expected to keep it low enough to close the deal. Deadlines are part of doing business, no matter how much we try to dance around that fact. This is why management will immediately turn around and convert story points into hours. These guesstimates will then become deadlines. I hate having deadlines for creative work, and would rather not be bothered by them, but they exist nonetheless.
Until you convince your customers to accept "NFC" and still do business with you, Agile will remain a quixotic idea for the corporate world
Can we please not fool the next generation of software developers into believing all this bs?
The problem is that all too often do customers think they know what they want but actually have NFC what they need.
Unless there is a constructive dialogue between the people who build it and the people that want it, it will end disastrously. Just think of all the failed white elephant projects that get “negotiated” on the golf course.
Until there’s an acceptance that being agile isn’t just something developers do but has to be adopted by the whole org, it’s not going to work. I know it sounds a bit dreamy but I’ve worked in big orgs that did agile rather well, so it is possible…
When it comes to money, the person footing the bill will get pushy rather fast. So I am curious how such a process works closer to the business side of the project, even in a big org
I disagree with “don’t estimate” part. If it’s hard, it doesn’t mean it’s wasted effort. And it’s very valuable to business to know with higher degree of precision how much things cost to develop.
It depends on many things, but from my experience, it’s possible, we went from 40% to 10% margin of error per quarter. When there’s a will, there’s a way.
Agile is what managers who have no clue about the agile manifesto do after they attended a mandatory, 6 hours long SAFe training. "Doing Agile helps the coding monkeys output three times as much as before? Gotcha!" When you prescribe the same sprint start day across the entire organization, that is Agile. If you have 6 weeks sprint commitment, that's Agile. When all teams need to estimate stories using Fibonacci, that's Agile. When the dev team needs to commit how much time they need for a feature that is planned for next year Q4, that's Agile. When the company needs an certified coach, that is Agile. When I need to create a Jira ticket and got it approved by three people to change the name of a small class, that's Agile.
I use "agile", when a company or product team understands that when writing software, you need to adapt to the things you find out. You don't know what you will do in 27 weeks and there is no point in story point estimating that stuff a half a year in advance. The company is agile when one team can use Scrum, another team can do Kanban, yet another can do whatever they come up with that works for them.
Everybody else does it, we have to do it, don't question anything even though you think it's not really that important or even detrimental. It's emperor's new clothing all the way down.
I agree it's helpful that there's a common theme on how "we" do things, but the point about making it organic is spot on. Nobody does things so they do scrum. The goal is to deliver something useful, how that gets done might be very different in different contexts. So mold it to how you do things.
Nice enough article, but I don't know where the author found such a poor definition for the word "ceremony".
A ceremony ain't just going through motions, it is going through motions to reconnect with elders, identity, re-confirm to all the position of some in the local structure of power, show symbolic values.
As such, that word is the most precise one in the Faux-Agile lingo, and clearly shows the purposes have nothing to do with delivering better software faster...
My own view is that estimates are useful, but should be treated less as a fixed point-estimate prediction, and more as a single step in a kalman filter, with the understanding that the output at each step is probabilistic, and the number of steps is unknown.
Buildings do not have moving parts. Their core structures are almost all fundamentally the same. There are problems to solve, but again, not dynamic systems, and always variations on very well known themes.
Programming is usually like building a new type of interdimensional alien spacecraft engine that interfaces with some other alien artifacts. There are usually a lot of unknowns, new concepts, many moving parts, unsolved problems, to build a new invention.
Depends, really. Most software is doing basic CRUD and efforts can be estimated reasonably well. Some other software, like your example suggests, is literally rocket science. But even here estimates are typically not too far off.
> Programming is usually like building a new type of interdimensional alien spacecraft engine that interfaces with some other alien artifacts.
If programming indeed was "usually" like that, then there shouldn't be much software getting produced at all. Since building "a new type of interdimensional alien spacecraft engine" is something that's very very unusual, to say the least. :)
The difference is you're hiring most other trades-people to do a generally fairly routine bit of work and then be done. A lot of programming is about doing unroutine work that has no clearly defined end-state.
I've been on teams where estimation is easy - you've got years of roughly the same set of devs doing fairly similar feature/bug work on the same code base to look back on and give an educated guess on sizing. Right now I'm on a team that's mostly doing integrations with third-party systems where our skill levels on the team vary between 20 years and 6 months of experience in the field, let alone this team.
$COMPLEX_REQUIREMENT: 12 days
BACK END - 3 days
FRONT END - 3 days
UNIT TEST - 2 days
...
Which is almost certainly gonna be wrong, so it useless. But "12" is a number and number can be added.
We have 274 man days this quarter, and of course we allow time for bugs and leave and stuff, so sum of estimates is 263, so that's our plan for the next 3 months.
Never works.
Estimates are useful for making branch decisions though.
So knowing is it 1 hour or 1 day or 1 week or 1 month or 1 year (which is closest) is useful for deciding if it is done (hint if it involves code at all then not 1 hour!)
For each category, accept it might take up to 5 times as long. So 1 day = 1 to 5 days. 1 week = 1 to 5 weeks and so on.
Is it a disaster if the thing took 5 weeks?
Id there another "1 week" tasks that is more urgent than the one you are thinking of doing next?
That is the level.
Then don't track too closely how long it takes .. BUT ... maybe have a team leader flag at 2.5x estimate to discuss in depth if it is worth carrying on. In addition to standups/whatever to get unblocked or report the estimate was way off.
In a nutshell treat the estimate as a decision heuristic. But not as a developer skill measurement tool, or a timeline delivery.
If you have a real deadline, then presumably you agreed to it (some contract) and hopefully when coming to that agreement technical people were consulted, and loads and loads of buffer was included. It is like agreeing to host people on Christmas day - it will probably need say 20 hours of prep, but arranging it 3 months ahead of time and slowly working on it the whole time is a surefire way of success. Starting the project on 23rd Dec is a surefire way to fail. Common sense in real life (because we use common sense not trying to "optimise" every last second).
Also in real life if other things mean you can't host Christmas ... you don't agree to the deadline. Let's book a table at the pub. That is what you do. Companies tend not to do that, they agree to all kinds of silly things. And you get silly deadlines, and bullshit estimates as a result.
An idea I’ve begun to understand better is the the higher the formality.. often the lower the average velocity, experience, capability and trust in and among the team.
Standards, structures and processes are still something I wouldn’t do without, but the configuration or extent can vary based on the team.
Problem I found with this journey is that Scrum concepts pollute Kanban and beyond. We now just do what works, as little process as works for us - and we implicitly review the process with each commit.
I think what’s missing from a lot of agile teaching material is how you build trust with your stakeholders/management. Without it whatever process you implement will still morph to have to include deadlines and estimates.
Sort of. I find real businesses with real needs who hire me as a contractor to design / develop software. They have no extra time and money for propaganda. I work either alone or hire subcontractors for help. How I actually do development is my internal business. They do not go into that area.
What a crock of poo. No security as ever. Fowler now bothers to mention Threat mOdeling. You engineers are rubbish. You are so lazy, and hate security so much. Even in healthcare and medical devices. Security is EASY! but alas, most of you and the management drones do not have a clue.
Agile has killed more people than Waterfall. Terrible. You should all be ashamed of yourselves. Do try harder! lol
Waterfall is not practiced, its an ill-defined strawman meant to be the "other wrong way of doing things". At best its a self-deprecating measure meaning the more agile you are, the less you know where you are going, and the more waterfall your project management appears the more you know what you are doing. Is that supposed to sell me on agile? I want the strawman.
Nope. Waterfall is still used and used correctly in situations where there are large and important contractual or compliance obligations. You wouldn't use an agile methodology in construction, you waterfall the hell out of that. Same for hardware projects where fab times can be very lengthy.
I don't blame you for coming to the conclusion you have, but you'd do well to signpost when you suspect something is true vs have reason and evidence that something is true.
Likewise, American manufacturers by WWII had already gotten very good at building complicated machinery based on relatively light levels of blueprints and documentation, with the detail-level designs being added to the drawings used on the factory floor.
In many ways we have to blame the introduction of computerized project management for waterfall... it simply wouldn't have been workable before that to even attempt a real waterfall method for projects.
I won’t wade into the intricacies of defining Waterfall or whether it has merit, but it definitely should be mentioned that Kanban—a capital A Agile technique which most capital A Agile advocates find unstructured and unpredictable (as in tends towards lowercase a agile)—was popularized by an auto manufacturer, where safety and liability are at least as much a concern as construction.
I love kanban. It's not perfectly suited for knowledge work, though. Great for smaller projects and for places where precise scoping and predictability is less important than just making progress.
Waterfall was not the name given by Royce to the multi-stage model of software development but it's easy to see how it got called that. Boehm called it "Waterfall" and used it as a foil for his own ideas in the spiral model, which was a pretty good description of a "middle-out" sort of process with an emphasis on feedback and customer support.
The Royce paper saw the one-shot waterfall as unrealistic even with the project alternating back and forth between phases. It ends with a complicated-looking process of writing two systems. Estimation of time to completion or cost is not really solved there.
But really what people mean when they refer back to Waterfall is the idea that you can put a reasonable upper bound on cost and time to completion of a larger project while keeping a maximum lower bound on functionality/scope.
Even in the 1980s, people were looking at business software systems and thinking they were pretty well defined in terms of complexity. A database, a bunch of data entry and data retrieval screens, some online functions, some batch functions. Maybe you could specify these like a builder specifies a concrete sidewalk. (A literal example from a software engineering book). Then estimation should be possible.
The trouble is that the requirements of a multi year project are often out of date before the project is started and many specifications are underspecified until the customer has seen something like the final product. So why not plan around change? Bound the budget, allow some schedule time, and see how much functionality can be built in that time. Let the customer prioritize parts of functionality and the developers work smoothly, and the whole team can stay in a mode of delivering functionality frequently.
Unfortunately it is still practiced in many environments, including most Navy software acquisition efforts, which are frequently run under a "systems engineering" process intended for hardware like ships and airplanes.
That's super interesting and I didn't know this history. However, misinterpreting someone's ideas doesn't make it not real if people are actively practicing them.
The link is the man who coined the term. Would you say waterfall today is strictly adherent to NOT testing code before delivery of subassembly? That's the coiners definition.
No, I'm saying that the article states that waterfall, _as is used today_, was misattributed to the coiner, and that's about all it says. That doesn't mean that waterfall doesn't exist and it certainly doesn't prove that the agile folks invented waterfall to have something hate on.
I used to work in banks where you had teams of architects create high level design (HLD) docs that get turned over to the design team to produce low level designs (LLD) that the get turned over to developers how are meant to write the code, then to be turned over to the testing team. With massive handover meetings at every stage and forests of paper for milestone documentation that had to be signed off.
Never had my soul be destroyed as completely and I seen such a massive waste of time and money.
Waterfall absolutely was practiced. Budgets always overrun and developers had to work overtime to meet deadlines completing all the tasks that weren't mentioned in the plan.
This still happens under agile too. It’s not like a process magically fixes things. Most of the issues I read from SWE in regards to agile is that they are often neglected in both decision making and the ability to hold leadership accountable.
No amount of agileness is going to save you from a miss managed team.
Well I wouldn't count mismanagement against agile then. At my work we practice SWE by managerial dictatorship and also "agile." I'm certainly not holding the failures against agile.
If you are implementing a set process from a book you are by very definition anti-agile.
When I implement agile at companies I don't even call it agile. And the only initial practices are:
* transparency -- you can't fix what you can't see
* improvement loop -- you need to learn from your mistakes and feed the results into next cycle
* process building fundamentals -- once you learned you need mechanisms to preserve the results. Checklists, templates, documentation, automation, infrastructure as code, etc.
Everything else is result of the learning. As we go, we spot problems or improvement opportunities and we decide how to fix it together. A lot of these fixes come from agile literature. Nobody said you have to reinvent everything. But now that we have implemented it in response to our actual problem we know why it is there which is way better than just implementing it without understanding.
Along the way people learn they have power to change things in a realistic way and not only they are welcome to do so but it is actually their job. Which is exactly the goal of all of this and yet completely missing when you put forward a book and say the process has to be implemented exactly as in the book.