Hacker News new | comments | show | ask | jobs | submit login
Scrum is the new waterfall (adambourg.com)
127 points by snow_mac 768 days ago | hide | past | web | favorite | 177 comments



I've been at a few studios, all expounding how they're "Agile! Scrum! Fast-Moving!" And I've seen true "agile" horror shows.

Stand-ups consisting of everyone either A) trying to prove they actually did work yesterday in as many words as possible or B) taking this as a chance to have a nice chat with everyone. Hour-long daily stand-ups are not agile.

The worst though was the planning meeting for next sprint, where no one had a #$%^ing clue what they needed to do, what the priority was, where the dependencies lay. A day every two weeks you dread.

I think the big take-away is not to do Agile for Agile's sake. Do it to solve a problem- if developers don't know what each-other is doing, there's solutions. Working on a product with changing requirements? Work in sprints to be able to respond to that. But don't do it because it's what all the cool kids are doing.

EDIT: The inverse is also true, people. It's not supposed to be a religion, you can pick and choose practices and processes to fit your requirements. Do what you need to do to achieve the goals you want to achieve.


At the university where I am, we were discussing today in one compsci class how agile is something that only works if you don’t care about consistent quality and only care about always having the newest, shiniest, greatest.

And how, if you are working on software for ESA, or working for one of the largest internet retailers, or if you’re writing a control software for railway switches, waterfall is the only possible option.

Even if the target is moving, agile is never able to provide quality.

Some of my professors were hired before by large internet retailers to spend weeks trying to understand even their business processes, as the retailer had built their infrastructure without oversight and every employee was working on whatever they wanted.

Just this "trying to understand what even is going on" helped them improve their processes and greatly increase their profit margin, as they were able to make many things more efficient with rethinking and rewriting.

EDIT: I’ll add my definition of quality here: Your software is able to interact with third parties without them ever having to rewrite their software, your software can run for decades without or with little maintenance, and your software is well documented. In my opinion, Agile is very ill suited for these requirements. One example is the Internet of Things, where you can be sure no one is ever going to update their smart washing machine


That's all just horribly inaccurate.

There is no aspect of agile development that means that 'quality' (whatever that means!) cannot be delivered.

Forget all of the fancy words and meetings and tools and techniques. At the core, agile development is about breaking work down into manageable, deliverable, testable parts, and being able to flexibly respond to changes in requirements when they inevitably happen – the goal being to ensure you are always building the correct thing.

You're right that this might not be appropriate for some systems – particularly those where there are unavoidable complex, cross-system dependencies that define acceptance. That's fine – traditional engineering approaches are suitable for these cases.

But the idea that agile development is anything to do with "without oversight and every employee was working on whatever they wanted." is a complete nonsense.


I’ve commented twice on this before, I don’t know if I should comment the same again, but:

The type of development written down in the manifesto, and the type of development used by companies as Google or Facebook, is not in any way acceptable for critical systems. Or complex systems.

The worst part is when that interpretation of agile [1] is used for complex systems, like it is by Google or Facebook and similar companies.

Agile works for web services, for small apps, but that’s it.

Anything larger, or more complex, needs planning, thought, and design. And often needs refactoring.

It’s rare that you can build a system where you don’t have issues with concurrency (and need to do timing analyses) or with interactions between multiple different systems.

I’ve seen these issues even when writing tiny projects like an Android app for an IRC bouncer.

And in any serious development, where your code is going to run critical systems for decades, you can’t just work agile.

    ----------------
[1] "as long as it works, documentation doesn’t matter", or "always have the newest, shiniest, greatest – no matter about plan or software design" – paraphrased from http://www.agilemanifesto.org/


"Agile works for web services, for small apps, but that’s it. Anything larger, or more complex, needs planning, thought, and design. And often needs refactoring."

I hope you discover the flaw in this logic sooner in your career than later.

I'll start with a hint: You are misreading the Agile manifesto. All successful teams and companies - Agile or not - do planning, thought, design. And the Agile community (which came out of the Smalltalk community) also invented refactoring (in the disciplined sense, along with the first Refactoring IDE).

Here's another hint: very few actively run systems in financial markets/brokerages were written with a waterfall method since the mid 90s. Not all were developed with great agile either, but let's just say they were "incremental and iterative". These are complex systems. They're not designed for 8 months on paper.

"And in any serious development, where your code is going to run critical systems for decades, you can’t just work agile."

This is somewhat true, assuming you have stable requirements. But almost no software, even for complex systems, runs for decades without an update. It just doesn't happen anymore except in the most constrained environments (space exploration, signal/control systems in dark territory, etc.)

If your career trajectory is working for the military building weapons guidance systems, or an FDA manufacturer building blood pressure monitors, you also may come close to this ideal.

But note even companies in mission critical healthcare, like Philips, are adopting cloud, open source, and agile in their next-generation health management systems that integrate with their MRIs and CT scanners.


And I can tell you that even some cloud software, like the Elektronische Gesundheitskarte was developed with Waterfall.

Often you have to design software interacting with dozens of other entities, all who are going to write software interacting with your system.

And your system is supposed to be able to run for years or decades with minimal maintenance.

Agile is far more suited for web apps, or small mobile apps, which can be constantly updated.


Actually that is incorrect. The Elektronische Gesundheitskarte was developed with "V-Modell". V-Modell, now V-Modell XT is the mandatory process for projects run by the german government.



If you read the description of v-modell carefully, you would have realized that v-modell isnt a strict process and does not define in which order certain steps have to be made.

Before tailoring (which can be used to create an agile process as much as waterfall btw) it just tells you WHAT has to be done and not HOW it has to be done. The order and iterations of each step is defined during the tailoring step.


Healthcare.gov was also a healthcare application developed waterfall style and was an enormous failure on launch day because of it's interactions with 3rd party systems.


And now imagine Healthcare.gov was Agile and its API would change every month completely.

And then imagine how every insurance would complain and how, after 2 or 3 months, everyone depending on healthcare.gov would have stopped supporting it because after the third rewrite of the parts of their software interacting with healthcare.gov’s API they’ve gone mad.

Every time you interact with third parties you have to provide stable interfaces.


"And now imagine Healthcare.gov was Agile and its API would change every month completely."

This is not required by any Agile method that I have experienced. You can create a stable API and stick with it forever. If you know what you want/need.

"Every time you interact with third parties you have to provide stable interfaces."

Yes. Case in point, the Cloud Foundry APIs at http://apidocs.cloudfoundry.org are backwards compatible and stable behind a dozen or more agile projects. Changes are made continually on the codebase, but the API remains stable, so that interoperability is not broken as new features are added.


It's not required, but the average Agile project ends up with update support like Android, and with a complexity like the Facebook Android App (which had to rewrite dalvik at runtime because they didn't plan about the limits of the VM they were working on).

Having a plan is the first step to perfection, they say.


Agile has never said you don't plan, you just don't fully trust plans made without all the information. You make your plan, and its great as far as up front plans go, but it WILL be wrong in some ways, and you will update it.

Who ever said you don't plan in agile and scrum? Where does the magical backlog you start the project with come from?

The point is not to pretend you know things that you don't. You have a severly flawed interpretation of what agile means.


Agiles definition seems to get redefined every time a discussion like this appears on HN.


> And now imagine Healthcare.gov was Agile and its API would change every month completely.

Most methods that get the "agile" label attached to them focus on managing change by organizing work in such a way that, outside of externally driven changes that the organization can't control (which have the same effect regardless of development method) the parts that are done don't change much after they are released, to avoid widespread breaking changes.

So I don't see the point here, or how it connects to Agile.


If the parts that are done don’t change much, it’s not Agile.

Agile is if no piece of your software can ever be called "done" because you are constantly changing things to be able to react to the market, as the market always changes.

If your software has clear requirements, has a point when it is done, and only requires minimum maintenance after that, you aren’t writing agile.


> If the parts that are done don’t change much, it’s not Agile.

Most agile methods focus on prioritizing the parts that are both most certain and highest value for the cost as a way of managing requirements uncertainty.

> Agile is if no piece of your software can ever be called "done" because you are constantly changing things to be able to react to the market, as the market always changes.

One of the aspects of uncertainty many agile methods address is change in market, sure; that doesn't mean that every piece of the software is changing all the time, it means that those methods focus on having mechanisms available that are intended so that when market changes create a need for changes in the software, that change can be implemented on a timely, efficient basis.


"You keep using that word. I do not think it means what you think it means." - Inigo Montoya, The Princess Bride


I am using the definition from agilemanifesto.org, which argues that "change" is more important than planning what you actually want to do. Or that "it works" is more important than documenting it.


I think you're taking Agile as defined in the manifesto out of context a bit too much and making it sound more like a religion (which, if you did, you would probably not be alone in doing).

Agile only makes complete sense when you take it in its historical context as a reaction to other methods, not as a standalone methodology with no history that sprang wholesale from the ground. It's not "this sounds good, let's try this", it's "we tried the other way and it did not work, we need to do something different".

Saying that you have something that works (an MVP, for instance) is _more important than_ documentation. That doesn't mean you shouldn't have documentation. Documentation will always lag behind actual code unless someone is full time updating documentation, which I have yet to see happen anywhere.

And even if you have documentation - it is an agreed communication of intent. The problem is, not everyone is necessarily reading it in the same way, or communicating their intent very well, or realizing inconsistencies and hurdles in what they've communicated. So when you actually have to write the software - you find you can't do it the way it was proposed, the way it was proposed is less efficient or fault tolerant, conflicts with other parts of the design, or is just ambiguous in a way that causes the developer to code up something other than was intended.

In a waterfall design, you wouldn't find out about the ambiguous bits until you present it to the client - at the end of the cycle when it's quite possibly already too late. In an iterative development cycle you can get constant feedback and more easily react to change. That's what agile is really all about.

Change is inevitable - agile attempts to recognize this fact and work as quickly as possible to create a product and immediately and incrementally improve upon it, keeping things moving forward, rather than spending huge amounts of time fussing over details that maybe won't make it into the final product while missing huge discrepancies or 'unknown unknowns' that aren't well understood until something is actually written.



The problem that you argue is not a problem in reality.

When breaking changes are made to an API, the version number is simply incremented. It is up to requirements to decide how long older APIs will be supported.


you seem to be mistaken about what Agile actually is.

i suggest reading the Agile manifesto [1] to get an understanding.

[1] http://www.agilemanifesto.org/principles.html


I commented on it before in a sibling comment.

Also, please link the actual manifesto, not the intention behind it.

And that can be paraphrased as "People over tools", "Documentation doesn’t matter as long as it works", "Who needs a contract? Just collaborate with the customer!", "Plans? Why? Just follow the latest trends!"

And I have to disagree with them in every single point.

Documentation > Working Software (as others then can write their software interacting with yours before you are done),

Contracts > collaboration (if you want me to stay longer to do something, expand the contract and pay me extra),

Plans > change (if you just follow trends without ever structuring your design, you’ll end up with lots of code and no single person understanding all of it)

http://www.agilemanifesto.org/


That is not Documentation that is Specification. How do you Document something that isn't built yet unless you have perfect foresight? Last I checked humans could not yet perfectly predict the future. Thus, the value of Working Software that can be interrogated and probed vs "Comprehensive Documentation" that may or may not be based on wishful prospection.

The Manifesto values Collaboration over "Contract Negotiation" leaving off the Negotiation part is a straw-man argument.

Similarly, "Responding to Change" it is not the Change that the Manifesto values it is "Responding to Change". What it values "Responding to Change" over is "Following a Plan" not making a plan or doing any planning.


Usually, you can decide on structure of code before implementing it. You can even have tests up and running before implementing it.

And in many cases this actually works.


You can do agile test first too.

"hey here's a feature we're pretty sure we need:

As a foo, I need to bar, So that baz

okay, write the test, make the test pass, refactor"

What about that isn't possible in agile?


Agile works for web services, for small apps, but that’s it.

That's plainly not true. Many large applications are written by agile teams.

Anything larger, or more complex, needs planning, thought, and design. And often needs refactoring.

Yes – none of these things are banned by agile development. Refactoring is positively encouraged!

It’s rare that you can build a system where you don’t have issues with concurrency (and need to do timing analyses) or with interactions between multiple different systems.

Absolutely. This has no relation to the development process used to build those systems.


> Absolutely. This has no relation to the development process used to build those systems.

Yes, it has.

Agile encourages people adding features without previously planning every single interaction of the feature with every other piece of software. Which means, Agile encourages writing code without being sure it will interact safely with other parts.


No, it doesn't, you are making things up.

There are plenty of agile/scrum teams who run continuous integration that won't allow code that breaks a build to even be in production. They write integration tests and acceptance tests and all of it is done with sprints and iteration.

Agile does no encouraging of adding features. It simply gives you the freedom to choose which ones get implemented based on the current knowledge rather than be forced to make those decisions based on 18 months ago knowledge.


Sure?

Because last I remember, Google is constantly shipping their Android Compat Libs with constant integration issues.

You’d assume that at least Google gets it right.


My experience with taking such classes at multiple schools was always that any CS class talking about Real World Software Development turned out to not be all that accurate. Which perhaps makes sense since computer science isn't really about software development, real or otherwise.

My favorite was being chided by a phd student teaching assistant that I needed to understand how things worked in industry if I wanted to pass. Problem was, I'd worked for multiple companies at that point and he'd worked at 0. What he was looking for (what it was exactly I don't recall) hadn't been my experience anywhere I'd worked.


The issue here are systems like Internet-of-Things devices.

90% of the people buying your smart washing machine will never ever update it. Never.

So change is non-existent in the code you are delivering.

And we’re going to have more and more IoT devices.

Already today we’re seeing Smart TVs with preinstalled apps for online streaming sites that don’t even exist anymore.

We have to be able to separate those pieces of code that will surely change and require constant change from those that will never get updated.


Absolutely none of this matters to agile vs. waterfall.

You can accomplish this with agile development:

1. People meet at the beginning of the project and say "heres what we think our washing machine needs"

2. They create a backlog based on those features

3. They start working and implement some features.

4. They find out they don't actually need the washing machine to wink and smile and say its done in a fancy british accent. It was a silly idea marketing came up with that adds no value. They drop that feature from the backlog.

5. They find out making the machine text you when your laundry is done is a desirable feature, they add it to the backlog.

6. Management says the competitor is launching a month earlier than us so we need to accelerate the time table. Engineering gives them the hard news that they need to sacrifice some features. They look at what they have left to do and prioritize the important items and drop the feature for the washer tie-dying your clothes if you want from the backlog.

6. Project enters qa, all tests are passing but testers found a bug. They write a test to confirm and reproduce the bug, they squash it and update the build.

7. Software is working, launch happens, everyone celebrates.

Nothing here says the software has to change anymore; its done, its shipped, its good to go, and done using scrum/agile.


That’s not the definition of "agile" I meant here.

The definition of "agile" is what actually happens in reality: You depend on an API from Google, and they, thanks to their so "agile" team, constantly drop or add features every few weeks, constantly move everything around, and then drop support for the API a few months later completely.

And they argue that it’s "necessary to have moving open standards for everything to be able to react to change".

That’s pretty "agile". Change! Change! Change! Who cares about consistency?


Maybe it's "agile" but certainly not "Agile".

You seem to have a total misunderstanding of how Agile methods work in practice. Not saying that Scrum is great but the points you are raising make no sense in this context.


All smart TVs these days are updatable.

My Nest thermostat is updatable. I don't have to do a thing for it to update. I would bet my Smart washing machine will also be.


And we see how well this updating works with the "agile" developed Android... Not.

A phone has a lifetime of over 5 years, but support is dropped before the legal minimum support of 2 years, even for Nexus.[1]

A device like a Thermostat shouldn’t even ever need updating. Everything it should do is record, transmit, and accept settings for temperature. Everything intelligent should be in a separate device.

It’s like we’re learning to separate the world into microchips and the interfacing processors all over again.

Also, I argued this before, if people can’t even update their game consoles or are still running Vista SP0 this month then they sure as hell won’t update their Smart Washing Machine.

You’re maybe living in Silicon Valley and all around these shiny new toys, but reality is that people are still using fax.

    ----------------
[1] By the way, this opens an interesting legal opportunity, I need to get updates for my Nexus device, as it’s less than 2 years old, and the vulnerability is a manufacturers defect.


That's not true. Quality and agile process are almost orthogonal. It is entirely possible to develop extremely high quality software within an agile environment and very possible to develop extremely poor software using waterfall.

Agile doesn't mean "every engineer working on whatever they wanted."

Ultimately quality software results when developers place a priority on writing correct software and when management is truly committed to creating high quality software and all of the expense that includes.


> Agile doesn't mean "every engineer working on whatever they wanted."

I totally agree. In my experience, it is very dependent on the group. When you are unlucky, you are in a group where for example one person always grabs the interesting parts. Because the group decides, it is very much about group pressure and clashing interests. A bad agile group can be similar harmful as a bad project manager.

One of the worst project managers, I experienced, insisted to divide the work "evenly" to the people in the group -- totally disregarding experience and abilities of the people. In a bad agile group, similar effects can happen.

In non-agile projects, there are dysfunctional groups and in agile projects there can be -- but in the later case, the problems can be worsened.


> Working software over comprehensive documentation

> Responding to change over following a plan

These statements from the agile manifesto are directly opposite to any kind of quality software. Documentation and Code Quality are some of the most important features. Your software is not working if it doesn’t exactly match its documentation.

You can live with that in web software, but you’d end in jail if you tried to write security critical systems like that.


Whenever the manifesto says “X over Y,” it does not suggest this is a strict dichotomy. I have worked in an agile manner on software that had extremely well-documented behaviour and all sorts of documented, objective requirements.

The expression "Working software over comprehensive documentation” simply emphasizes what matters at the end of the day: Working software. If you need a certain amount of comprehensive documentation to prove that the software works, you incorporate that documentation into your process.

It’s the same as emphasizing collaboration over contracts. I worked for a decade in consulting, I assure you there are contracts with clients even when you use Scrum. You don’t refuse to have any hard, contracted, binding commitments. It’s just that you prioritize collaborating with people over setting up solitudes that communicate by throwing contracts over a wall.

As for responding to change over following a plan, if you can set up an environment where there are no changes allowed to interfere with the plan, by all means ignore change and follow the plan.

But if you actually have changes that matter, you need to deal with them too. But that doesn’t mean don’t plan, or never follow any plan. It just means, respond to changes that matter.

For example... What if you discover that the security for a piece of software need to be updated. You’d respond to that, wouldn’t you? You wouldn’t push on to finish what was originally planned with your fingers in your ears?

These are guidelines. They’re motivating forces. You are supposed to do it in harmony with the hard requirements of your business and domain.


Documentation can also be useful without working software.

For example, when you have to interface with others. They can work on the software that depends on yours before you have even written a single line of code, and it will work.

Software without documentation also can lead to serious issues – as in the "does the train continue running when the signal is broken" question I mentioned a dozen times now.

In actual critical projects, Documentation over software. The best software is nothing without documentation.

That’s the first point where I have to disagree with it.

In many other points, agile leads to teams just carrying the technical debt with them.


> The best software is nothing without documentation

You know, it’s not enough to restate your own points, you have to read and absorb the things other people are saying.

Once again, “Working software over documentation” does not mean “Working software instead of documentation.”

It means that working software is more important than documentation, but both can be important. And it can be that documentation exists to make the software work.

If I said “Working software over software patterns,” that wouldn’t mean “Never, ever, ever use the Command Pattern, or Adaptors, or a Facade.” It just means that you use those things to make the software work. And if they are essential to making the software work, they are essential.

So you tell me that software is nothing without documentation, and I happen to agree. But software is also nothing without binaries. But that doesn’t mean “binaries over working software.” It just means, “build binaries as part of working software."


Does the fact that you haven't read all the detailed technical specs for your car prevent you from using it? Do you know the detailed specs for the actually critical braking mechanism in your vehicle? Or is it impossible to use your car because you don't know the tolerances of every nut and bolt in the system?


These statements from the agile manifesto are directly opposite to any kind of quality software

No they are not. These statements are guidelines and principles to follow, and are not absolutes.

If the requirements (i.e. acceptance criteria) for features you are working on include comprehensive documentation, then this can be delivered in the context of an agile framework.


The argument is:

Agile is impossible.

You have to document every feature before implementing it, because someone else will write software depending on it while you implement it. If you already document and plan every feature before implementing it, you aren’t doing agile, you’re doing waterfall.

What you are suggesting – to plan, document, implement and test every feature one after another – is just waterfall for every problem separately.


What you are suggesting – to plan, document, implement and test every feature one after another – is just waterfall for every problem separately.

Well, isn't that the core of any software development process? We're ultimately delivering features, and they all require planning, documentation and testing. The development methodology used doesn't change that. 'Agile' is the key word – change will happen, and being agile helps you to respond to that.


I responded to your sibling comment with the definition of Waterfall.

And every single comment defending Agile here was just "well, you can still document stuff before implementing it, and then verify it, and then maintain it". Which, effectively, is: Even if you’re agile, you can still do waterfall.

Great...


Obviously I'm not making a particularly convincing argument…

At the end of the day, agile development is primarily about ensuring that you can respond to change quickly and with minimal disruption. Change occurs in all software projects, and agile tries to avoid some of the pitfalls that one can fall into by architecting an entire system in advance.

"Even if you’re agile, you can still do waterfall." is not quite what i'm trying to get across. Rather, it's that agile doesn't mean you can't provide documentation in advance if that is a specific requirement. Bear in mind that external APIs that must be stable are quite distinct from the rest of the system; indeed, it's a horribly design failure if you can't completely swap out your implementation without changing those APIs!


I'm very curious, what is your definition of waterfall?


The classical definition:

1. You research the actual requirements, including by testing some possible alternatives via mockups, etc. 2. You design the concept of your system, design APIs and interfaces. 3. You implement it. 4. You verify it matches the design and fulfills the requirements 5. You release it and maintain it.

Most importantly, after Step 2 your APIs don’t change anymore. This is especially important as third parties might rely on them.


I appreciate that's the definition, but the problem that agile tries to deal with is that this almost never happens as you describe in practice.

In reality:

  1. You research the actual requirements
  2. You design the concept of your system, design APIs and interfaces.
  3. You implement it.
  4. You discover that the requirements are unimplementable because of some unforeseen design issue
  5. You make changes to your implementation, documentation and requirements to deal with this
  6. You discover that documentation regarding a third-party system is inaccurate and requires design changes
  7. You make changes to your implementation, documentation and requirements to deal with this
  8. You verify it matches the design
  9. You discover previously unknown acceptance requirements
  10. You make changes to your implementation… and so on.
  ??. You release it and maintain it
Even with skilled, competent developers and architects, no system ever turns out exactly as expected. The goal of agile development is to accept that and try to incorporate the concept of continuous delivery and adaptation into the core of the process – since it's required anyway!


And now imagine a big company develops a software interacting with yours at point 2, and finishes it at point 5.

You’ll have to add backwards compatible wrappers that emulate an older API in 7 and later.

Or you can actually test with proof-of-concepts, with mockups, with other tests, before you even write the design.


I think you missed my point there – I'm not describing an ideal agile process. It doesn't matter how many proof-of-concepts, mockups, tests and design documents you produce in advance. Change still happens.


Matthew already highlighted the concerns. So, hypothetically, what happens when just after you finish step 3 it is found that one of the core security mechanisms that the security of your life-critical product depends on and that was part of your published API is suddenly found to be insecure due to some new breakthrough in a research lab somewhere? Do you throw it all away and start over from step 1?

Or what happens when, again at step 3, you discover that one of he approaches advocated by the design, and tested by the mockup in the small is unacceptably non-performant when scaled to the real load? Or some other constraint that was not fully tested by the mockups?

You can't say "well, we do a better job with the mockups." If the mockups covered 100% of the functionality and requirements of he system they wouldn't be mockups, they would be the product.

And if your answer is, "well we do this kind of discovery in the testing and mockup phase, changing the mockups to reflect the issues we encounter," congratulations, you are doing Agile development.


Okay, then let’s think about it.

You write your software, you deliver it, you discover that your software, running in pacemakers, is insecure.

What do you do? How do you send out security updates? How do you expect to be able to constantly update that software with agile?

There are many cases where updates are literally impossible.


I'm not sure how a 'waterfall'-style development lifecycle will help you here. Even if you front load all of your design, documentation and implementation, you will still find bugs and security flaws in your software. As an example, see every software product ever shipped!

I think you're maybe labouring under a misapprehension of what agile development means. In particular, you don't have to 'constantly update' deployed software in order to be able to develop in an agile fashion. You're free to follow the guiding principles to make sure you are able to cope with changes when they inevitably happen.


> That is, while there is value in the items on the right, we value the items on the left more.

Agile does not say that documentation is not important, nor does it say that a plan is bad. Documentation is worthless if the software doesn't work. The plan is crap if it no longer addresses the actual real-world requirements of the product.


A software without perfect documentation is worthless.

Without documentation, you have to assume things. Does the train, if the signal is broken, stop or continue running? Does the radiation treatment machine, if you have a race condition, and two commands are executed at the same time, block, or just execute the last command?

Software without documentation kills.


> Documentation and Code Quality are some of the most important features.

Code quality is an important attribute, sure, but those don't say anything about it.

Comprehensive documentation of the bureaucratic sort of process documentation that I think motivated that line is not a deliverable, its an artifact of process; when it becomes understood as an end to its own rather than as something that should be used to the extent it is useful to achieve value in the end product, it becomes extremely problematic (I've experienced the kind of bureaucratic software development that this reacts to, and I can assure you that it does not contribute positively to quality, it contributes to management -- and staff -- spending lots of time worrying about and measuring and optimizing metrics that are unrelated to quality.)

> Your software is not working if it doesn’t exactly match its documentation.

Your software is not working if it doesn't deliver the expected behavior. Documentation of requirements is important to provide a consistent method of communicating between people (and across time and space) what those expectations are. And to the extent that it is necessary for that purpose documentation is a good thing.

But its important to keep the purpose in mind. Otherwise, you end up with excessive effort going into polishing documentation rather than building software.

> you’d end in jail if you tried to write security critical systems like that.

Sure, bureaucratically comprehensive documentation is extremely useful (and sometimes essential) for demonstrating compliance with process mandates. Such mandates, and the incentives they create, are also very often not helpful to the actual quality of software, including the aspects of software quality which motivate the mandates.


How many lines of code have you written that are in production with actual users right now?


I’ve worked on some agile projects from others that I picked up and had to "fix", where we, now, after 2 years of trying to work with them, have decided to completely rewrite them from scratch.

Every agile project I’ve seen yet was just a huge ball of technical debt. It worked, it had few user-facing bugs, but adding features was impossible, and the documentation was "whatever the software does". Completely matched the agile manifesto "working software over documentation" and "change over plan".


That isn't an answer to the question.

Your response is selection bias. Every project that you've seen is a ball of technical debt. There is no argument to how documentation alleviates technical debt. If you document a poorly designed and scoped piece of software it doesn't make it any easier to change.

If the software was impossible to change and that change was a requirement then the code bases you were working with did not represent "working software"

The agile manifesto says, "working software over documentation." It does not say "working software and an absence of documentation."

The fact that you needed to add features is exactly the reason agile exists. If the creators had a complete feature plan in the beginning and faithfully executed it without deviation from the plan then there would be no need to ever add features and the quality of the code in terms of "changability" would be moot.


As compared to the waterfall projects I've witnessed, most of which never made it to completion, the rest of which were far away from what the client actually wanted once delivered.

Unfortunately, outside of a small percentage of companies, most software written today is just bad.


Well, except most government-scale projects that work out well are Waterfall or V-XT.


"Your software is not working if it doesn’t exactly match its documentation."

Not true. Documentation is not always updated. And many times, if not every time, the requirements documents the clients give are often flat out wrong. So you could build the software to the requirements, and end up with something that doesn't meet the client's needs.

"Documentation and Code Quality are some of the most important features."

However, these take a backseat to working software.


Your software is not working if all the third party services that depend on the API you provide are broken because you changed your API, or because your documentation is wrong.

You can work without documentation or without stable implementations if you control all the endpoints and ensure they always have the latest version (like mobile apps, web apps, etc). But if you can’t, you have to remember that every API version you’ll ever provide will have software using it, and you won’t be able to EOL the support for that version.


Ill reply to the root of all your comments, but this comment by you below sums up the problem:

> If your software has clear requirements, has a point when it is done, and only requires minimum maintenance after that, you aren’t writing agile.

This is simply not true. All 'agile' projects Ive worked on have had a complete-as-possbile analysis phase where we figured out the scope and the domain up front. This is not anti-agile at all, but is necessary on any project anywhere you are working on. (Agile is largely about avoiding 'big design up front', not 'big analysis up front'. There is a massive difference between analysis and design.)

Agile is about changing your plan when the _requirements_ change. Your API or whatever should change if your requirements change no matter what methodology you are using. But with waterfall you will not be able to and you will end up with a useless API.


The issue is that there are projects where changing your API is impossible, which means that using Agile is often a hopeless concept. Because if huge insurances already depend on your API, no matter how Agile you are, you can’t change it anymore.

And there are many cases where your code will be frozen at one point. Even if the requirements change.

Especially for Internet-of-Thing devices this can be very problematic, as no one is going to ever update them.


I dont really understand this. "NF_REQ_00: API must not change"

Add verification tests to ensure API remains as documented. Every time someone checks in code your tests are run, break if something changes.

Every project has functional and non functional requirements, you write tests for them, your project is in a failure state if the tests are not passing.


How do you update the code on a Smart Washing Machine? A Smart Pacemaker?

In the world where everything is digital, we’ll have a huge technical debt of un-updatable software.


> The lifetime for a washing machine is 30 years. Your software on that will last 30 years. Using a development method designed to make quickly changing requirements easy is stupid when your code will be "write once, never change".

If the requirements change, you must update your software. How does waterfall handle changing requirements? It doesnt.

To repeat: agile _handles_ changing requirements. It doesnt _provoke_ changing requirements.

As someone who spent the first 6 years of my career doing waterfall, the only way to combat this is by treating the functional requirements as an immutable contract between you and the customer. When the requirements change you blame them, and refuse to change your software. Then your software never gets used.


> If the requirements change, you must update your software. How does waterfall handle changing requirements? It doesnt.

How does agile handle change? By assuming everyone will update and has no issue with all third-party accessories constantly breaking.

If you can’t ever change your code after you’ve written it once, then Agile isn’t useful to you.


You seem really hung up on this idea of clients having to update code that is already out in the wild when requirements change.

Maybe I've had a twisted experience of Agile, but it seems like its most useful when you are working on a greenfield project, with a customer that maybe doesn't even know what they want, but they know they want something. So you get some requirements, and build a prototype. Then you show it off, and they make some comments, (generally like: "Can it run in the Cloud? Is it Social?", "Could that icon be more of a cornflower-blue?"). Sometimes you get useful feedback also... Then you go back and refine the prototype. Rinse and repeat, and eventually it does enough that they are happy.

Even when you do know "exactly" what you are building, I would still prefer starting from a bare-bones version, and building out from there. Unless you've built this exact thing before, planning it all out beforehand will invariably miss some Rumsfeldian unknown-unknown technical detail, which might tip up the whole apple-cart of the carefully, laboriously, expensively, laid-out plan.

It's not like its 1995, and releasing a software version involved burning thousands of CDs, printing manuals, boxing it up, and distributing the boxes to retail stores. And most of us are not launching the software we write out beyond low-earth orbit.


you mean firmware updates? What is the problem you are talking about?

Plus its a regular problem ensuring that an API is stable. People do it all the time - aren't you wondering why you are the only person arguing this point? Many people in this thread deal with these problems on a daily basis...


The problem is that no one is going to do firmware updates on their smart washing machine.

Your software can’t be updated. You have one try to do it right.

It’s what next to all of the modern startups don’t get right. They build fancy software, but then in a few years your smart house doesn’t work anymore because the services it connected to have changed APIs and the house itself encountered a few bugs?

The lifetime for a washing machine is 30 years. Your software on that will last 30 years. Using a development method designed to make quickly changing requirements easy is stupid when your code will be "write once, never change".


You are buying much higher quality washing machines than I am, apparently. If you get 8-10 years out of most brands now, you're doing quite well. Planned obsolescence is just the best...


Mostly quality Miele machines. They have 10 years warranty from the manufacturer, so most actually survive for 20 years. Not what they used to make – they used to survive a lifetime – but it’s okay.

Same with stoves or fridges.

We already see how hard it is to keep mobile devices updated. Android is the nightmare example, but even Apple drops devices after 4 years. In 20 years, your smart fridge will have tons of malware on it if it’s connected to the internet. If it isn’t connected, you won’t be able to get updates, so the software has to be perfect.

And the point we made in Uni was that agile is suited for situations where your requirements change after deploying. In all other cases you can do waterfall – provided you actually find out what you’re supposed to do – better.


I'll have to look into that brand the next time I'm in the market - I've been hearing a lot about them on here lately.

Personally, I want my appliances 100% electromechanical. I don't really get the whole IoT buzz. Besides simple reliability and repairability, I have no desire to control my stove or washing machine with my phone. I can get up and punch a button.


Then you’re the perfect customer for the German market! Next to no modern hip bullshit. Next to no IoT, next to no "Agile", or "moving target", or "constant updates", instead technology like it used to be – buy once, use a lifetime.

Warning: Expensive. Really expensive. Their washing machines start at 1300 USD.


If you are writing integrations for API features that are fantasy at this point, you are digging your own grave. You have extended your risk and when you find out six months from now that you bet the farm on a useless feature, oh well; thems the breaks.


Why? What that is written in the Agile Manifesto precludes quality?


> Responding to change over following a plan

Which, in reality, leads to everyone working on random things, just adding features and somehow patching bugs.

But often refactoring the system, rewriting subsystems, etc is necessary.

> Working software over comprehensive documentation

If your software is working, but there is no documentation, this might work well for a website, but as soon as you go into control systems for the heating of the ISS, or go into railway control systems, or systems that measure the optimum way of packaging a package at an internet retailer, documentation is everything.

    ----------------
The agile manifesto works kinda, but, to quote a professor, especially the "as long as it works, we don’t need to know what we are doing or have documentation, leads to you carrying all that technical debt around with you all the time".

One of the largest european internet retailers fixed their code quality problems by forcing teams to release all code shared between two teams to be released under Apache License on GitHub. Suddenly developers started caring about code quality, reusability, and maintainability.


Which, in reality, leads to everyone working on random things, just adding features and somehow patching bugs.

No, it doesn't. Bad management and process leads to that.

But often refactoring the system, rewriting subsystems, etc is necessary.

Totally acceptable and something that can easily be included in agile development.

If your software is working, but there is no documentation

This is not a prerequisite of agile development.


>> If your software is working, but there is no documentation

> This is not a prerequisite of agile development.

What I often see in "waterfail" projects is documentation that's inaccurate and reflects what once was the plan someone wanted to build.


As an aside; I've been playing with rust and it's built-in documentation solution has an interesting quality to it. Code snippets in documentation are compiled and can be tested at compile time.

It's one excellent way to ensure that updates that necessitate a change in documentation actually get documented.


I love using doctests in my Python code. They can't replace proper tests, but its great to be able to guarantee your docstrings are correct.


This is so often the case, I agree. I can't count the number of times I've received a design document that details to the level of database field names – and of course, never bears any relation to the final product.


> > If your software is working, but there is no documentation

> This is not a prerequisite of agile development.

No, but agile says that software is still quality software, even when documentation doesn’t work. http://www.agilemanifesto.org/

It’s pretty clearly written down.


You are ignoring what I am saying – these are guidelines and principles to follow in development processes, not absolute requirements.

If a requirement for the system you are delivering is comprehensive, complete documentation, then you can deliver that. The point is that "comprehensive documentation" without "working software" is useless.


No, it is not. Your software might be working later on – and at the same time, someone else can write software depending on yours, based on just the documentation.

Additionally, software without documentation leads to issues like "does the train continue running when the signal is broken?", and similar ones.


No, it is not. Your software might be working later on

It might be, but it almost certainly won't be. See the errata to any set of documentation. But this is a digression – if the acceptance of your product or feature requires documentation of interfaces in advance, then that's a requirement that can be met regardless of the development process that you use.

Additionally, software without documentation leads to issues like "does the train continue running when the signal is broken?", and similar ones.

No, that's what incorrectly designed, implemented or tested software leads to. But again – if documentation is required, using an agile process in no way restricts your ability to meet that requirement


But when you have to document and plan everything you write before you do so, isn’t that again waterfall?

Are you just telling me that waterfall is also agile?


Are you just telling me that waterfall is also agile?

No, I'm obviously not communicating effectively.

Documentation is a deliverable component of a software system. As a result, it can be tracked and managed just like any other deliverable component.

In any real software system, the initial documentation will not reflect the finished system. That's inevitable – it's why we have errata. The difference between 'agile' and 'waterfall' in this case will be whether those changes to documentation are delivered at the end of development, or as they are required by changes to other parts of the system.


The issue is when other software is built upon those initial documentation of your system.

Third party services depending on you having a stable API, for example.

You can’t just break support for them, so what do you do in agile?


Don't break support for them?

There are still requirements, there is still design. If you have a customer depending on your API, that is a requirement. Agile doesn't mean requirements go away.


And if your implementation never changes, but you just do minor changes – isn’t that the maintenance phase of Waterfall again?


I don't like trying to make direct comparisons between Agile and Waterfall as if they are two separate mutually-exclusive approaches to development.

A product developed in an Agile process still has a 'maintenance phase'; the idea of software maintenance is not a Waterfall concept, it's a Software Development concept.

One thing I think you're having trouble with (maybe) is that this article was about Scrum, but you're attacking Agile. Scrum is an implementation of Agile, or maybe better characterized as a process facilitating Agile development, but it is not uniquely Agile. There are other Agile implementations/processes (Kanban, XP, etc.). Agile is more about attitude than implementation.

What Scrum is attempting to do is break down big ideas into small workable units, and help teams organize, assign, track and complete those work units efficiently. It separates the development cycle into several sprints: a team works a selected amount of items per sprint based on their capacity vs. the 'weight' of those items. If the development process using Scrum incorporates accepting changes to requirements or the design along the way, it's Agile. If it doesn't, Scrum can look a lot like a series of tiny waterfalls; you can absolutely have a Waterfall process where your teams organize and execute work like Scrum.

The most important thing to keep in mind is that neither Agile nor Scrum dictate what you consider to be a finished product, or a successful product. The standard of quality you added to one of your earlier comments is absolutely achievable using Scrum or another Agile process - in fact, it is the stated goal of Agile to turn out a high quality product which meets stated requirements.


The issue is that Agile works far better when you have constantly changing requirements, and when you will constantly work on the code. Agile works best when "development" and "maintenance" are the same. See: Chrome, Android.

Disadvantages: 0 backwards compatibility, annoying for users (I’d love to force the people who made these decisions to try and use a 5 year old Android phone with modern apps, thanks to the constantly breaking APIs that’s not possible), there is no backporting of fixes.

Advantages: Always the newest, shiniest.

When your requirements are stable, and you write


I think we can resolve this debate through a map. I think our problem is a disagreement on what is actually industrialized and predictable vs. What is more exploratory and uncertain.

Most software - but not all - deals with uncertain requirements. These are suited to agile methods.

Other software is in a well understood industrialized domain. Here, I would not recommend waterfall per se, but an approach like Six Sigma and Formal engineering methods certainly makes sense.

Simon Wardley has explained this well here: http://blog.gardeviance.org/2015/06/why-agile-lean-and-six-s...

I think the disagreement is

A) is how industrialized IoT applications actually are.

It will be 2 decades or more I believe before they are. This will require to lots of disposability and change in the meantime. Hence appropriate for agile.

B) how much indistrialized software will there be in the future? Things like railway systems are well understood and industrialized. And don't need agile. But my point has been there isn't a lot of railway software out there to write anymore.

But at some point, will the newer IoT domains be eventually so industrialized that it requires formal engineering methods? To this I say - sure - but not for a long while. The applications are still too fluid.

C) how many people actually work on various ends of the spectrum of agile vs industrialized ? There has been an explosion of activity in using software in new domains, which is agile by definition. Industrial software even by your own definition only needs to be written once every 80 years. So it makes one surmise it will always be a minority of software development.


Well, whatever Agile means in theory, in practice, it almost always leads to situations like the Android Update situation. Of the Google Chrome Update situation, which is only better because everyone is forced to update.

A situation like the Android update situation, though, is completely unacceptable for smoke detectors or other household devices.

No matter how good predictability is, the demand is that these devices are "buy once, just works™". So we need to write stable software for an unstable use case.

But in no case should we end up with serious devices having a support situation like Android, or the SAMSUNG smart devices.

Yes, it will only be written every few decades, but that is fine. It doesn't need to be a huge industry. Just like Washing Machines are mostly made by smaller enterprises.


I think that the products you're complaining about are in the state they are in because of vendor choices independent of the software development process. If total stability and backwards compatibility are not design requirements, neither Agile nor Waterfall will automatically implement them for you.

I think your class or professors are assigning behaviors and outcomes to development processes that should instead be assigned to the companies or people implementing them.


"Working software over comprehensive documentation"

over != instead of: "That is, while there is value in the items on the right, we value the items on the left more."

And no, you probably wouldn't use agile to manage the project of heating the ISS. That is a very static problem space with unique requirements. Using the same framework to plan and build anything related to heating the ISS to build an HR system would be overkill. Right tool for the job, etc.

I've worked on both good and bad agile projects - all have involved managing change. The good ones didn't involve throwing a plan out & doing what you want. They merely accepted that when you start building a solution, you lift rocks and uncover requirements that you never envisaged having. In an agile environment, you can deal with that - bring those things onboard, reprioritise and rescope, giving an accurate idea of when a product is going to be delivered. In a traditional waterfall environment, these same requirements get left until the now-woefully inadequate product is built, and useless to anyone.


Well, that’s why you start testing what you can do, what the issues will be, etc while still working on the plan.

You don’t decide to build a bridge out of a specific steel without testing in models which kind of steel is ideal for your case, or if you might have to change the shape of the bridge.

But in the moment that others will build projects depending on your implementation being stable – in the bridge example, this would be others building the pillars for the bridge or the bridge deck – you have to keep the implementation stable. (In software, this is usually third parties basing software on your interface)


But it's not just simply the "correctness" of a solution that changes, that 3rd party API you're having to interface with might not return data in the same way you were expecting, the other company who were going to deliver the grid component can't do it in time anymore, half the dev team have fallen ill and are out for 3 weeks. No amount of planning up front can ever be sufficient, agile gives us devices to manage all of this that the traditional waterfall strawman lacks.

And then there's the solution itself not being suitable. Have you ever shown a solution built completely to spec in a waterfall-esque environment, only for them to say "oh, but it doesn't solve x"? or for them to start using it and suddenly realise there's some fundamental flaw in it? This is where agile comes in - you can regularly demo the software and make adjustments and reprioritisations as it comes together. It enables that conversation to be had and the change to be made when it's cheapest - early on, rather than when a huge app has been built on top of it.

Also - agile doesn't preclude stability. You can manage this with test suites (the top layer of the current project I'm working on's test suite is built against our documentation), we know if and when we've broken something because we're constantly emulating the third parties in your example using our software and asserting that it remains correct.


Well, sure.

But that’s the issue: What if you are writing the third party API, what if you are writing the firmware of a Washing Machine which is sold with lifetime guarantee*? (up to 99 years or death of buyer, whichever happens first), what if you write feature where every change has to be reviewed and costs millions?

People expect from quality software what they expect from a quality washing machine, meaning that they buy it once, never have to touch it again, and it will always work and never change. Except for some minor features added at some point.

But removing or changing features? Never.

This is true for users (remember how many people complained about Windows 8 unusual interface?) and for other developers (How many people go all crazy every time Google kills yet another service, or removes another API, or changes their API syntax?).

Sometimes it works, sometimes it doesn’t. But, effectively, I’m asking for perfect backwards compatibility. And that is rarely possible in systems where the software defines the documentation, instead of the other way round.


Never allow your school to interfere with your education.

> in reality, leads to everyone working on random things,

No. It's just that we acknowledge plans need to change and draw new plans. If we feel something need refactoring, we refactor. If it needs a complete rewrite, we do it. We can do it because we have tests for everything. The tests define what the parts do. They are part of the documentation, one you can run and be 100% sure it is correct. When we acknowledge something needs to be redone, we plan for it.

> If your software is working, but there is no documentation,

See above.

> to quote a professor

Judge the argument, not the person making it.

> Apache License on GitHub. Suddenly developers started caring about code quality

A GPLv3 variation could have been a smarter move, but yes. Having people able to look over your shoulder is a fairly decent quality driver. All my recent work is open source and, when I cut a corner, I always document the corner, why I cut it and when and how I will fix it.


I added in the first comment my definition of quality.

And in projects like "ISS heating control", or "railway control system", or "electronical ID card for 82 million people", you want a software that is backwards compatible so third parties interacting with you will never have to rewrite your software, you want a software that runs with minimal maintenance, but is easy to extends, etc.

And Agile is ill suited for that. Agile is well suited for cases where no one ever depends on a your software providing a specific interface and where your target moves quickly and where you can ensure everyone always runs the latest version. In other words, small applications and web apps.


"you want a software that is backwards compatible so third parties interacting with you will never have to rewrite your software, you want a software that runs with minimal maintenance, but is easy to extends, etc. And Agile is ill suited for that."

I really think you need to build a lot more software before you can make a claim like that. You're being fed lines from your professors.

The vast majority of software, period, is not backward compatible, has to be rewritten all the time, and costs billions to maintain, and is hard to maintain.

Agile vs. Waterfall is only one factor among many in such as discussion. The difference with Agile is in its ability to deal with uncertain requirements, something Waterfall cannot handle by its nature.


Well, if companies build their software depending on yours, before you have even written a single line of code, and if your software is supposed to run for decades, agile – which is better suited for constant change – is not ideal.

And this piece of software exists. Governmental projects interacting with lots of insurance companies, for example. Like the Elektronische Gesundheitskarte – a concept of giving every German an insurance card issued by the government, with all medical data stored in a cloud with insurances, doctors and pharmacies having access to it.

Before the software was even finished, PoS software was able to interact with the API and insurances had sent out millions of cards.

Breaking anything at that point costs millions.

EDIT: regarding your edit: Agile is far more suited for constantly changing environments, where you also can just break support for any third parties depending on your API. But obviously, that’s not really quality software when every software depending on yours has to be changed every few years.


Like pointed out before, you need more experience before you can make claims like the ones your teachers make.

They probably need it too.

What you describe is a problem stated in great detail and an API that's needs to be kept backwards compatible. There is nothing in agile that precludes that. Quite the contrary: if you build your tests well enough, everyone who did not participate in the original design can use them as examples of how your system actually works. More than that: you can always prove your system still works as specified, all the time, every time. Agile is good, very good, for dealing with a changing world. And, while the APIs in the system you describe can't change without major investment, if the rest is also difficult to change, whoever did it did it seriously wrong.

You can tell that to your teachers.


Well, we aren’t in that class to learn how to write software, but how to apply engineering principles to software.

And if you buy a washing machine with lifetime guarantee, you expect it to run forever, but the requirements are mostly static, too.

For a smart TV, on the other hand, you separate the pieces of software with static requirements, and those without. The internal signal decoder, etc will be static. UI might end up rendered by a local web browser, and especially media content will be online.

So if, in 10 years, Netflix defaults, the online content can immediately remove Netflix and replace it with its successor. This UI might make sense with Agile.

But the code the user expects to be reliable and fast – like the code for decoding and displaying a TV signal – will not change, and should be written with waterfall.

    ----------------
In our world, people are constantly complaining about change. People don’t want to buy a new phone every 2 years, they want to buy one phone one time, and be able to use it for 10 years. At least.

People want to be able to buy a fridge or washing machine and use it for decades. Do you buy a new fridge every year?

We are going to be living in a world with internet of things. And with the internet of things we will have shoes and houses running software. More and more software will have to be built with the expectation never to get updated.


My Mercedes-Benz has software firmware for my transmission that is updated annually, at the shop.

The software can be updated over the air in a Tesla every few weeks.

Mercedes is working with my company to adopt agile practices and server deployment to disposable cloud environments for their connected car IoT applications.

Industrial and Appliance manufacturers such as Philips, and GE (a 10% investor) are also adopting this same software and methodology for their IoT platforms and applications.

I don't know what else to tell you - there are certainly areas where you correctly point out you need to get it right the first time. But you keep making projections about the future of immutable software that are contradictory to all the evidence.

The world is a big place and even in Deutschland the big industrial manufacturers are adopting lean and agile. Cars at Toyota and Honda (and many others) are developed with Lean Product development principles - not a big up front design plan, but test-first, concurrent, iterative, incremental engineering.

One can buy hardware that lasts a decade - the software can and will evolve except in constrained environments. A washing machine is not a constrained environment.


No one is going to update their washing machine. No one.

Out of the people I know, most never updated their phone software or the firmware of their consoles. My mum was, until a week ago, running on her PC Windows Vista with the last update being downloaded in 2006, when we moved into this house and I had run Windows Update for her.

A washing machine is probably the most constrained environment in any home. No one’s going to update it. When you turn it on, you want it to just work. Same with the smart fridge.

These devices have a lifetime of decades, and no one’s going to update them.

Same with cars, most people don’t want them to be updated.

Remember, this is Deutschland, we still use Fax for business communication.


"Before the software was even finished, PoS software was able to interact with the API and insurances had sent out millions of cards."

Yes. There are hard constraints in many projects. If you know exactly up front what you need, you do not need Agile approaches to handle the uncertainty.

However, there is plenty of uncertainty to be managed behind this API.

"Agile is far more suited for constantly changing environments, where you also can just break support for any third parties depending on your API. But obviously, that’s not really quality software when every software depending on yours has to be changed every few years."

One doesn't have to change an API to evolve the software. Old school HTML and HTTP from 1992 still works on an evolving web.

"But obviously, that’s not really quality software when every software depending on yours has to be changed every few years."

APIs can be made to be backward compatible or versioned as they evolve.

Again, you presume the people asking for the software know exactly what they want. IF they do, then you are fine. It is very rare in practice except for simple or well-understood predictable domains. If the stakeholders do not know or agree on what they want (most of the time), something like Agile is appropriate to manage change.

Most APIs evolve a lot as parties learn more about the problem space, but this does not necessarily mean older versions automatically break. This really has little to do with Agile vs. Waterfall, it has more to do with the design decisions made for evolvability.


> Old school HTML and HTTP from 1992 still works on an evolving web.

Actually, not. Try

    <blink><marquee>TEXT</marquee></blink>
Most browsers stopped supporting that.

And many other features, especially in CSS, constantly change and evolve, often older variants being dropped. Especially Chrome is dropping support for older code all the time. (Also, soon CSS newspaper-like column flow will be dead T_T )

Even if the people you work for don’t know what they want, the part of "determining what they want" should be part of the "find the requirements" at the begin. You might even want to work with mockups or proof-of-concepts at this time.


"we were discussing today in one compsci class how agile is something that only works if you don’t care about consistent quality and only care about always having the newest, shiniest, greatest."

My employer (Pivotal) uses XP across the company as a way of building consistently high quality software products at a sustainable pace. We have something of a reputation for doing this over the past 20 years with Pivotal Labs.

"And how, if you are working on software for ESA, or working for one of the largest internet retailers, or if you’re writing a control software for railway switches, waterfall is the only possible option."

This is partial nonsense.

Firstly, unmodified waterfall was never an explicit software development methodology, it was an example used by Winston Royce in 1970 about how NOT to run a software project. http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/wate...

You think Amazon delivers software in a waterfall? Amazon has dozens of teams building microservices in parallel with thousands of releases to production happening daily. http://joshuaseiden.com/blog/2013/12/amazon-deploys-to-produ...

I was the General Manager of Operations for a Class 1 railway, and know something about how railways systems were design and built. Control software for railway switches is written around once every 25 years, so yes, one can do that in a more deliberately planned manner. Financial and billing software for railways is mostly Mainframe or packaged (SAP or Oracle), so that too, was built in a waterfall, which was by no means a guarantor of quality in practice, with many problems being discovered only after deployment.

Yard management systems and equipment tracking can be built in an agile manner, though mostly were built in the 80s and 90s so only add-ons and additions are being built in this way. Much of the surrounding demand management, BI, order management, and forecasting systems are built (at worst) in an ad hoc manner or (at best) in an agile manner, because the level of change required is too high to warrant the lead times of a waterfall-planned approach.

The point is that most companies that develop new products do not do so in a waterfall manner, they do it in a manner that allows for continuous feedback, because they've learned that quality in the face of uncertain requirements requires such feedback.

"Even if the target is moving, agile is never able to provide quality."

The most polite thing I can say to this is, "citation needed".

Agile methods are an attempt to use Lean product development principles in the realm of software. I think you may want to learn more about them before drawing such conclusions.


1. I was not talking about Amazon.

2. The issue with the ESA code, like with railway systems, is: You write the code one time. You get a review. If you try to change just a single line later on – for example, to fix an issue that leads to the heating system on the ISS overheating – it will cost millions.

Agile is a system that is not optimized for taking one version of software and running it for decades or centuries unchanged. Which is often a requirement.

Agile is useful if you can always push the latest version to all users, as in web applications.


"The issue with the ESA code, like with railway systems, is: You write the code one time. You get a review. If you try to change just a single line later on – for example, to fix an issue that leads to the heating system on the ISS overheating – it will cost millions."

I can guarantee you that 90% of the code in the Positive Train Control systems (the first real "new" control/sensor system in freight in decades) were not written with this methodology.

"Agile is a system that is not optimized for taking one version of software and running it for decades or centuries unchanged. Which is often a requirement."

"Almost never" a requirement, in my experience. And... centuries? Really?

"Agile is useful if you can always push the latest version to all users, as in web applications."

Yes, or you know, mobile applications, or sensor network applications, or... basically pretty much most software that isn't in highly constrained environments. The revenue of the former is in the trillions; in the latter is the billions.


And I can guarantee you that the software controlling the switches for the high speed rail and commuter rail networks in my state was written with waterfall.

The requirements were pretty clear, as there was a legacy system to be replaced and the changes to be made were not too hard to be found, the design was done in a DSL specifically written for the task of representing this project, the software was implemented, verified, and by now is under maintenance.

The requirements for a railway system rarely change, so agile is not really useful there.

Regarding "centuries": Some German companies stopped using the original Zuse pre-war computers in 2011. That’s over 70 years of software running constantly. Some banks still run code from the days before C. In the future, we’ll have more and more situations like that. And just like washing machines are sold with a lifetime guarantee, we’ll expect the same from the smart washing machine we’ll buy in 2025, too.


I believe you're fundamentally mistaken about the direction he future is taking: the world is not regressing to things that never change, it is progressing to continuous, perpetual change for most things in active use. There is plenty of evidence for this.

I empathize with a world view that prefers stable, planned, predictable, and long lasting. But I have rarely seen this work in practice.

I work with the banks (including in your state), and as far as I can see their aim is to move towards faster change in their IT systems, and disposability.


The banks do, but the people don’t.

No one wants to have to buy a new Smart Fridge every year because the manufacturer stopped updating the old one. Hell, people won’t even update at all.

We can’t even get updating software on Android correctly. And instead of releasing bugfixes for all releases, Google instantly abandons a version of Android. There are 0 maintenance branches of it.

Applying the Android "Agile" situation to the rest of Software and especially Hardware is the worst nightmare possible. If that is going mainstream, I’ll quit compsci and start teaching kids in a kindergarten instead.

Android is actually the perfect example: Always having the latest is more important than it being thought out well! Documentation is less important than it somehow working!

Android’s update situation is the exact outcome of applying Agile development not just internally, but also externally. And it’s a horrible nightmare.


I feel you, if the world is moving towards Android updates, it won't be a great world.

One can design systems for usable and even pleasant continuous operations AND updatability though - even in an agile manner - eg. erlang and Ericsson.


Sure, one can.

but almost always Agile leads to this issue. Even the previously linked NASA document says their spaceflight software should have a release and update schedule with feature updates similar to Google Chrome (aka, no support for old versions at all, always the newest, shiniest)


This. It is so important to separate the practices associated with Agile apart from Agile. One practice I use a ton is retrospectives and really it's useful just as a way of codifying how much and how to experiment with and change team process.


The best project manager I ever worked with just wrote everyone's names on the board, asked what they were working on today, writing it on the board next to their name. You can imagine how the next day went; "So Andrew, how did X go yesterday, any blockers? Is it done? What are the issues, can we break this down further?"

It's all you need.


I don't really agree with this - Scrum isn't really easy to do but in many situations it is the simplest approach to take.

If you need to monitor your progress in 'real' terms (i.e., whats actually completed) Scrum is pretty much the minimum ceremony you can get away with in my experience.

If you don't need to do that, say if you don't have a deadline that you need to know you wont hit ASAP, then Scrum is likely not a good fit.

> It’s boring, its old, it leads to pointless meetings run by people who don’t write software, who don’t understand the technical process behind writing software and don’t always care

I dont think these people are meant to be running scrums.

> Adhere to the sprint commitment, even if you have to work over time.

Yeah, and you will be if you dont adjust your commitments as soon as you realise youre not going to meet them.

> Agile is about moving fast, building working software today and delivering with changing requirements

Yes it is. And scrum adds time tracking on top of that.

It sounds like you dont really know what the purpose of scrum is. And this is fair enough, because I dont think they had a very good explanatory service - it was pushed as the 'next big thing' like XML and SOAP was in the 90s.


It sounds like you dont really know what the purpose of scrum is.

Unfortunately, every response to "we tried process/technique X and it failed" in this space is ultimately "you didn't understand it" or "you did it wrong".

So no true scrum would have...

No true Agile shop would have...

And where does that get us? It appears that the success rate of people understanding and implementing these things is very close to 0%. Maybe the problem is with the processes/techniques after all.


It appears that the success rate of people understanding and implementing these things is very close to 0%

I don't think this is the case. You're likely only hearing about the failures.

Here's a counterpoint – we implemented Scrum. It didn't cause any real issues, increased the speed with which we delivered features to customers, and increased the visibility of everything that was happening both within the development team and within the wider business.

Part of that was having a dedicated individual responsible for managing and implementing that process who made sure that it was actually achieving things we wanted to, rather than being a tick-box exercise, and who isn't afraid to make modifications such that the process better reflects the needs of the team.

In fact, I'd argue that the problem is completely the opposite to what you imply – it's not that people don't implement 'true' agile or 'true' scrum, but that they try to do so, rather than using these systems as a basis for building a development process that works for their team. No true agile team is slavishly adherent to rules that don't work.


You only hear about the people struggling with agile because the people not struggling are busy working on stuff.


well allow me to put a note here as someone who uses it successfully regularly.

Also I explain several times why he doesn't seem to understand it - having non-technical people running his scrums, working overtime and screwing up their velocity measurements, not being able to cope with changing requirements. These are things that scrum was built to _directly address_.

I say again, if you're criticising scrum as being too onerous, you are probably either using it wrong or using it where you shouldnt (where you dont need to track your progress.)


At the same time, many stories have been of people who have done it horribly wrong, to the point where you really cannot say anything but, "You did it wrong."


edit: I just noticed they said 'sprint commitment', and that they should adhere to it even if it runs overtime.

This is probably the most wrong part of the article - you get done what you can, and let it affect your velocity. It is this velocity that is the central indicator of how much you can realistically achieve


I read "adhere to the sprint commitment" as being an undesirable aspect of Scrum that the author was complaining about, not something that Scrum is lacking.

It's certainly one of the bad parts of Scrum as commonly interpreted; c.f. http://www.peterkretzman.com/wp-content/uploads/2014/10/Aski....

(That's why newer docs tend to substitute "commitment" with "forecast", to clarify the Manifesto author's intent).


interesting - I recently heard someone call it a forecast and thought 'man why don't they use that term in scrum?'

Nice to know that they actually do (I havent read up on it in some time, probably should)


I agree with you. "Scrum is the new waterfall" is true in this case in that both have been built as total strawmen.

Scrum says nothing about unit tests, nor does it require sticking to the commitment (although it used to)

> I dont think they had a very good explanatory service

The most recent documentation on "Core" scrum is very simple and easy to parse.


Yup, this guy sets up a total strawman which doesn't accurately describe how I've seen Scrum run at two major telecom software companies.

We change mid-sprint (when we need to), we don't have non-programmers running scrums or writing user stories and if you don't complete a story in a sprint then you roll it over to the next one.

If you have deadlines and contractual commitments to meet his suggestion of "use Kanban" strikes me as somewhat absurd. Running a team off of a Kanban board has its place (some of our support / platform teams use it) but I'm pretty sure your business owners want a better answer for when something is going to be delivered than 'when my Kanban board is empty'.


This isn't a straw man. This is exactly how Scrum gets implemented when management is running the process and not the development team.


+1


What it delivered soon? Put it in the top of the Kanban board and set a rule that the stuff in the top of the defined list gets worked first.


Having worked on a team using Scrum, I agree with this: it's pretty much the least process you can have while still fairly accurately predicting when work will get done and when any given feature will show up in the product.

But it can be done well or badly: the team I worked on rigorously adhered to the process, and only cautiously diverged from it after careful consideration. Scrum done badly will rapidly become micromanagement instead.


I feel like every time I complain about scrum and my frustrations with it to people who believe in it, the response is always "well you just don't understand scrum," or, "you're just not doing scrum correctly." If the process is really as good as it is supposed to be and espoused to be by scrum evangelists (including many of the coaches my companies have hired to help us implement it), then I wonder why it's so hard to do correctly.

I have worked with great engineering teams that have hummed along in informal processes that were similar to Kanban (though not defined as such). As soon as we were forced to switch to scrum, out productivity absolutely tanked, and we couldn't get the things we needed done because we were either in planning meetings all the time or we had to spend longer deciding whether to change our priorities mid-sprint as things would come up or lessons were learned, because that meant a lot of time and energy spent communicating up the chain why things did not get finished or the priorities changed.


> If the process is really as good as it is supposed to be and espoused to be by scrum evangelists (including many of the coaches my companies have hired to help us implement it), then I wonder why it's so hard to do correctly.

I'm quite skeptical of "agile coaches" and similar.

But in general, I'd suggest that it's easy to break by either swapping out or eliminating one of the defined roles, having someone unsuitable for those roles doing them (e.g. a manager or program manager serving as "scrum master", which instantly turns scrum into micromanagement), having an excessive number of mid-sprint changes (if changing a sprint's work mid-sprint happens every other sprint, something is very wrong), or not having any acceptance from the broader organization for getting work done at a regular cadence without constant "emergency interrupts".

You should not be spending any significant fraction of your time in planning meetings; those occur once a sprint at most, and the sprint-planning portion should mostly consist of quickly doublechecking priorities and grabbing the top stories by priority. The other significant effort lies in breaking down and "sizing" stories (which is the job of the development team, not the person providing requirements).

I do agree that Kanban can work as well; it just doesn't (in my opinion) have good predictive power for when work will get done. (On the other hand, some of our teams ended up switching to Kanban because Scrum doesn't work at all with a distributed team.)


> It sounds like you dont really know what the purpose of scrum is.

Agreed. People lose sight of this being a "framework" far too often. Make it fit your needs and run with it.


I think we can just summarize the entire thing now and forever as "Management leading developer meetings always leads to tears".


As I experienced Scrum in practice, it is said that people should be "empowered". In reality, something totally different is happening: People are not empowered, but the management just delegates the responsibilities for the project to the developers and lays back, assured that they just have to wait for the "agile fruits" to be reaped. They have done everything, they could do, now the "empowered" employees have to see ...

Those poor employees might feel "empowered", but in reality, they are "impoverished". They are now responsible for the project, but often without amble resources needed. For example, when they need additional hardware or rooms or .... they are not empowered to get it. They are still beggars. Instead, they are empowered to work longer hours to reach the project goals they have set themselves. Why did they such a stupid thing? Because they wanted to look bright or because of group pressure or because they want a raise ...

And after they set their goals themselves, they are on their own.

This is not "empowerment", it is not about people -- it is just a good excuse for the management to get rid of the responsibilities and to "motivate" people to work longer hours. It is the newest "silver bullet" process of a project management that already lost control.


This. 1000x. This is definitely one of the pitfalls of poorly implemented scrum. For bigger companies, scrum tends to be a decision by the higher ups and implemented by the lower ranks keeping middle management is disarray over their role in the change. Moreover as the author mentions agile is about people first, but followed by processes. If scrum teams don't have the power, or empowerment, to change the processes, to become more agile, to remove blockers/impediments, the team burns out / self destructs for the precise reason you mention about spending longer hours achieve the committed (not to mention the committed is affected by the deadlines which are manipulated by management/or worse, external stakeholders) in order to uphold their word/project. To me there is a flaw here.

I do believe scrum can be an effective tool for a few use cases, but people often underestimate the warning signs. "Scrum is easy to learn, but difficult to master." - https://www.scrumalliance.org/community/articles/2011/may/sc...


This is really the heart of it, I've really only worked in environments that have been top down as their implementation of scrum. When the engineer team got to decide, we picked Kanban. The lightest process for us.

Scrum top down. I buy that. Bottom up? I'd give it a try, but the team needs the ability to say "no".


Scrum works fine when it's just programmers. Scrum and Agile become shackles when management gets involved and sees it as a status-checkup for their top-down view. The latter scenario seems to be 90% of situations, unfortunately.


The fatal flaw of any process - scrum, kanban, waterfall, etc, is when engineering teams try to use them as a replacement for communication rather than as a communication aide.


Yep. Process is compensation for lack of talent, which may sound cruel or arrogant but it's not. If you're great, you can self direct, design, and implement what needs to happen in your project with no help or management at all.

It's totally okay not to be great, but when you're not great you need help, and help comes in the form of management, and management needs a way to set and manage expectations. Agile is one way, waterfall is another.


I don't understand why this is on the top of Hacker News. It seems to be one person's opinions and experiences about complex issues that face hundreds of thousands of engineers in thousands of companies across the world.

Nothing in the article makes me trust this persons perspective over any other. It's not thoroughly investigated with metrics over many case studies. There's no empirical evidence to back up his assertions. He doesn't have a reputation for other work that would make me trust his informed opinion over any others.

There seems to be hundreds of articles touting every possible side of this issue, why did this one strike a chord?


Your right, I presented no evidence. Honestly it was a rhetorical post that I did not expect to strike the cord that it did.


Whats wrong with waterfall? Surely mis-using waterfall was the culprit. I'm guessing same goes for Scrum. I've never followed true scrum so my opinion might be jaded however the devil generally lies in the implementation of the process.

You cannot fit the same tire on a truck and a kid's tricycle, same way you cannot fit the same process for all development cycle. If you team needs clear direction and everything well defined (written in stone), follow waterfall. If your team needs formal process but can do some things on their own (say a team filled with new hires and one lone experienced dev), scrum might not be so bad. Kanbaan is good when everyone knows what they're doing and you really just want to know which stages do you need to focus more on.

You're best bet is probably to try all three and whatever else exists, mix and match until you can successfully deliver your products without a whole lotta overhead and then keep tweaking the process when conditions change (new hires, folks leaving, change in goals/features).


Waterfall is literally a strawman methodology made up to illustrate the worst possible way to build software (short of having a million monkeys trying random programs), in order to serve as contrast to a way that actually works.


I wasn't really going towards defending waterfall[1] but lemme give it a try.

Waterfall might be not bad when you cannot have feedback mechanisms, you cannot roll out changes in phases, where requirements/scopes are fixed, tech stack is well defined and products have matured enough.

[1] : wanted to more say, use the right tool for the right thing in the right way. I don't think its easy but its doable.


It's amazing to me the lengths organizations will go to avoid the one thing that actually works: hiring a small team of really good developers, paying them a lot of money and giving them hard ship date.


That seems like laissez-faire waterfall to me. The problem with the "hard ship date" is that it usually comes with a set of expectations for what is ready on that date: If you need software X on July 1, what happens when software Y becomes more necessary on April 15?


A hard ship date and flexible requirements is the way to do it: you force the issue and require your (great) developers to immanentize the software. It clarifies the mind and, assuming you didn't start off with absurd goals anyway, you get most of what you want.

Most importantly, it gets you to ship.


I've seen that go completely balls-up in several cases. The thing is, even really good developers can struggle to define what a complete and working system looks like.


But if you have a few people develop and ship a product, what's everyone else going to do besides look stupid?


But then you're working in a bigger business where you do actually need to interact with managers and internal/external stakeholders. And they're all asking you: «When can you deliver?».

Yes, estimating is hard. It's impossible really, but everyone knows this and accept the consequences of delayed deliveries and blocking events. But they want status updates and delivery schedules all the same.

Pointless exercise maybe? But in real life and in business actually having someone thinking longer than three days ahead is usually a necessity, in my experience.

I agree though that you should minimize the amount of pointless meetings and solve blocking issues as they come - this is more a sign of a good tech lead and healthy work environment rather than weaknesses in the Scrum methodology.


Scrum at its core is about setting your team up to LEARN and improve by introducing feedback cycles.

That's it.

If your team learns after one cycle that they are more effective without one of the common ceremonies, then your team should drop it.

The problem that Adam is observing is not with Scrum itself - it's that too many miseducated managers lose sight of the point and are obsessing over the wrong details.

Source: I have been on extremely successful Scrum teams, I have seen Scrum done properly dramatically transform "stale" teams, and I have trained with Jeff Sutherland, one of Scrum's founders.


Any system implemented without an understanding of how systems work, and how humans interact with them, will be painful.

We need systems thinking, not any one prescribed process.


> We need systems thinking, not any one prescribed process.

Problem is, its hard to get people doing systems thinking, especially about systems they are part of.


It's hard, but it's the only problem remaining.


Straying away from any consideration of "how well agile was implemented", I've never been at a shop that seemed to get any real advantage by saying they are "agile". Although my experience is only 4 shops on that matter.

I would argue that since the method is deliberately left unclear, in reality it mostly profits to whoever wants to game it to his/her advantage. And if everybody wants the project to succeed as their respective primary goal, then you do not need such methodology anyway.

Do whatever seems to work best for the team, and don't name it. Don't waste time writing blog posts about how you found a new groundbreaking way to work, as these matters are subjective.

For example, now I work by asking whatever needs most attention right now, getting it done, and then asking again. No backlog, no notes anywhere, nothing. If we forget about something and it is important, it will come back up. It has no name and it works perfectly for us!


Similar to choosing a web framework, most often the issue is with the people, not the methodology/framework.


I'm at a small BigCo. We have a few pilot agile projects. Based on what I see them doing, and what I read in the article, it looks like scrums are two week death marches.


This is so negative.

Look... the process can't be, "Hey you get all the time in the world to build undefined requirements with out providing any sort of insight." The reason you let people who don't code in to the meetings is because they pay your salary -- right? So translate. Give them something that makes them feel you are spending their money correctly.

It's not all that hard to just define requirements, put a line in the sand for what you think you can get done in a two week block, and show your progress on a board -- is it? This article seems to take the stance, "Devs are the only ones who matter and everyone else is someone who is going to make me sit in a meeting I don't want to be in." It's really immature to think this way, and short sighted.

You need some sort of process, some sort of roadmap, so your sales and marketing team can drum up interest for the product and the team can be successful growing the business. This isn't an academic exercise, it's a job, and you work on a team that requires communication to operate.


If the primary way information flows between teams in your organisation is by throwing everyone in a room on a bi-weekly basis, you don't have much of an organisation. That's not professionalism, it's bad management under the heading, 'better than nothing.'


If you got a bad development process it doesn't matter how you label it SCRUM, Waterfall or Kanban.

The main is issue is to implement SCRUM properly. If I have seen it both ways. The difference is like night and day.


The problem is that Scrum is rather lukewarm agile. It's mostly about toe-dipping, not making big changes in how you think about how you work. It can lead to this, but it just really often doesn't.

Firstly, Scrum too often becomes the waterSCRUMfall - you don't deal with the problem end-to-end, you insert a version of Scrum in the middle of a broken organizational context.

Secondly, Scrum also mostly has a lot to say about team organizational structure, reporting, and process (which managers like) but not so much about software development practices (which help you do your job). This can lead to busy work and misinterpretation from those who "manage" software rather than those who "do" software.

Thirdly, I have a theory that a lot of the fallout from Scrum and agile not working is due to people trying to find something easier to digest than Extreme Programming, which includes nearly all the development practices that scrum teams pick and choose. Once you look for something easier to adapt to, meaning you don't have to change everything, it becomes easier for agile consultants to keep modifying the already lukewarm process to be a 10% improvement of the mess you already have rather than a leapfrog replacement.

And the few times a leapfrog is OK, it requires a lot of organizational air cover and buy-in, top to bottom, because you can't "force" people to work in a very different way without it being rejected.

My view is - if your approach to building software is broken, and you want to do something different, you have two choices:

a) build a process organically from the top down and bottom up simultaneously - perhaps a mix of Kanban for change management, and individual development practices that the software teams & senior developers want to do.

b) if you have the willpower and lack the knowledge or patience, adopt a "more extreme" agile method like Extreme Programming, and complement it with organizational change management techniques like Kanban. And ensure you have people or partners that can really help you with it (at all levels - not just as "coaches").

Finally, Kanban is not a software development process. It's a change management process. You an evolve a software development process out of Kanban and picking/choosing from other ideas from XP or others. Kanban is good in that it allows you to tailor an approach that deals with "big picture" organizational change issues first -- trying to do too much and piling up requirements inventory or work-in-progress. But it doesn't say anything about how to build software.


Where do I send my money! Scrum is the worst! Kill Scrum


This made me laugh. Send money to adam dot bourg at gmail




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: