In a moderately large (say, 500+ employee) organization, deadlines are essential to get anything done. Suppose, for example, you're overhauling a billing system, making international tax calculations simpler. The finance team is making a hiring decision: they need to get tax stuff calculated before the end of the year; do they need to find consultants to help them with this, or is the software going to be ready in time?
If your answer to them is "I can't tell you, we just do stuff on the kanban board and don't think about deadlines," they're
(1) going to be less than thrilled with you, decreasing trust, and (2) unable to make their decision effectively.
I don't think "there's only "stuff" to be done" is an adequate alternative to deadlines and sprints if it doesn't address the reason organizations use those tools. They're primarily tools for predictability and scaling outbound communication about progress; I think the author mistakenly believes they're supposed to be motivational tools. They're sometimes misused that way, for sure, but that's not the whole picture.
Either the deadline is too short and your team has to work overtime, or it was too generous and you lose efficiency.
The project manager, of course, still needs to be able to make an estimate according the his team's velocity, which he can then communicate to other business divisions.
“One of the most controversial updates to the 2011 Scrum Guide has been the removal of the term “commit” in favor of “forecast” in regards to the work selected for a Sprint. We used to say that the Development Team commits to which Product Backlog Items it will deliver by the end of the Sprint. Scrum now encourages the Development Team to forecast which Product Backlog Items it will deliver by the end of the Sprint. It may seem to be a simple wording change, but in fact there are strong reasons behind it.”
Pretty sure the Scrum folks are aware of needs of enterprises and their deadline addiction.
In my opinion, it needs be more like drawing a (rough) "this is how far we will get at the current velocity" line somewhere in the sorted backlog, rather than a fixed set of tickets that you expect to get done.
Popular tool-driven "agile" workflows where you move tickets into a particular sprint, followed by the admission of defeat at the next sprint planning meeting when you failed to complete all of them, reinforce the idea of "commitments" rather than "forecasts".
I think the issue stems more from the prevalent confusion between deadlines and milestones in part of the IT world.
I used to work on ship command system. The ship would be stopped for maintenance for a couple of months and available for installation and qualification only during this set period of time. Anything not ready for this window would not be delivered. That's an actual deadline which the project had to meet.
As a project manager, it was my job to keep track of my team milestones to ensure we would be ready for the deadline and find solution when we had issues. It's not only about overtime by the way. You might need a temporary increase of the size of the team, access to external resources, time with an expert or a scope adjustement. Sometimes it's just impossible to meet the deadline and you just want to know as soon as possible to reschedule at minimum cost.
You do need to make an estimate in order to plan your milestones but you also need to commit or you will never know if you are on the right course.
PMs who can successfully and reliably handle this kind of analysis and forecasting for their team are more rare even than high-end software developers (in my experience in nyc startups, may be different elsewhere). And they can be far more expensive, because these skills apply heavily to things outside tech as well, so you're competing in a larger employer pool than developers.
The reason, I think, that (immature) companies use these sprint/deadline frameworks is that it puts the entire burden of everything complicated onto the engineers (estimation, forecasting, figuring out how to meet deadlines, incremental delivery, progress communication, GANTT charting, road-mapping, etc). They focus all their budget on hiring "good engineers" and then try to localize all the "hard" stuff there within engineering, and then they decide they can skimp and hire underpaid former consultants to be "product managers" where all they do is check boxes on a requirements list, and play the telephone game between deployments and engineering. It's literally creating the role of that guy in Office Space - "I'm a people person." These people will also never be able to gain enough control of product direction to challenge anyone in leadership over anything.
Company leaders seem to have some vague notion that this will somehow all work out in the long run by framing this whole thing as "being agile," and "not sinking too much resources into things that will change," yet all it is is lack of proper project planning abilities and a fear of owning risky decisions. It's a culture utterly lacking in leadership accountability. If everything is "wait and see," or, "ask the engineer for a commit- cough I mean 'estimate' for when it'll be 'done'," then nobody can ever be held accountable for bad leadership decisions or for improving their decision making skills over time as they learn and grow in a leadership position at the company.
I know this comes off kinda conspiracy-theory-esque, but I have a feeling that many of these systems develop in order for the less-skilled-but-scrappy initial engineers/employees who were friends of the founders to retain control as the company grows and develops the resources to hire far more skilled (but more risk averse) employees. Far too often you have someone who is perfect as a founding engineer and VP of Tech for 10-15 initial developers, but does not grow their experience to match the exponential growth of the company, so they do NOT have the experience to be CTO of a 100 engineer tech team several years later if the company takes off. But now this person has been through "war stories" with the founders/C-Suite, and the other execs can't bring themselves to realize that the person is now a huge burden on the company, because the other execs are also mostly in over their heads as well at this specific point in time (high growth).
One good way to judge if this is the case at a company is to see if they have an "engineering IC career ladder" and see what skills they decide apply to "senior" IC engineers. It very often ends up being things outside engineering that appear at the higher rungs of the ladder, even though it's supposedly an "IC" ladder, because they can't figure out how to hire the proper roles for those things, and can't do them themselves. It's crazy to me how many people seem to think a very experienced engineer should be working on GANTT charts and roadmapping instead of systems architecture design & maintenance, and high-complexity-scale technical change management, not to mention R&D and onboarding/training.
I've spent most of the first few years of my career working for small startups, or teams where I was the only dev, so I've seen lots of engineering projects "managed" by people who don't understand engineering. I was quite capable of self-managing, so it was often frustrating being straightjacketed by someone who didn't really know what they were doing.
I wasted a lot of time and probably would have been better off in a larger start-up or company with proper engineering management and a defined career track.
Giving an estimate should not come with the whiff of punishment for probabilistic inaccuracy, potentially due to factors out of one's control.
That's what deadlines do. That's why they distort behaviour.
I can deal with hard technical deadlines and project milestones - the former is a fact of life and the latter is often required to help focus larger projects. What I emphatically dislike is sales guys making business decisions before checking with the engineering teams to determine how much effort is required to deliver a project.
The solution is to not let this problem end up on your side. "We will see what we can do, but since we didn't make any estimate on this, the risk of failing the deadline is on you. If you want to set a proper deadline next time for my team, consult me first so we can do a proper estimation. I hope it works out for you this time."
I also had plenty of cases where we had to reach an imposed deadline. Then 2 weeks later you ask "did they find issues?". "No, they didn't deploy it yet"
I've found it's better to work with them on a compromised phased released / MVP so the customer still gets what they asked for and you are still seen a miracle worker albeit without actually having to do anything unrealistic in the end.
Unfortunately though, there are some businesses that place their sales team too far ahead of the health of their engineers for any compromise to happen and there isn't really much you can do aside taking the overtime pay and move on shortly afterwards.
I always worked in small companies < 200 people, so this might be different in bigger ones.
If you are the competent software engineer that always delivers quality software, and is always pushing the company towards a better and more progessional way of working, you will not easily be seen as an obstruction. I never was, and always had good to excellent reviews.
A sales person that sucks a deadline out of their thumb is in no way to be seen as professional. Questions such as "who estimated this? Was anyone from software development involved? Did you account for testing? Do you expect the custommer to find all our bugs?" Etc, should reveal that there is another way of working.
I had a sales guy respond to me "I've been doing this for 30 years, so I know". This guy was fired half a year later
Sales people setting deadlines without consulting the people that need to do the work, is very unprofessional, and there is no way that anyone can defend that. You just have to make this very clear, in a diplomatic but clear way.
But as I said, the best thing you can do in those sort of places is take the money and move on because you aren’t going to change the CEO’s attitude.
If you are seen as a replaceable work horse, you lose a lot of leverage indeed.
Here in Belgium there is a huge shortage of software developers, which places the programmers in a stronger position than the companies.
I can leave a company and instantly get the same or a higher pay somewhere else.
A company on the other hand, first has to find somebody, and then still overpay the first few months before the new employee becomes productive. From that point it's a bad position to be in.
My thinking is if a CEO comes from a sales background then they will run their company in that mindset. Much like how some CEOs who are developers sometimes struggle to release products because they're focused on the technology rather than the product (gross exaggeration there - but I hope you understand the point I'm making)
I think the point is that if you do make those commitments you are just making things up, in reality those commitments are worth pretty much exactly as much as not making the commitments.
Where with reality I mean the actual physical reality. I realise that social reality can be different, and even a commitment that you make despite everyone knowing that you don't actually have any basis for making that commitment (no track record, no reasoning, no nothing...) can be valuable.
Well, reality is not that harsh and there is a middle ground. It definitely is possible to complete things on schedule. Assuming "cost, time and scope - choose two" as a reasonable model, there are trade-offs you can make to get the thing out in time.
What the OP argues, and I fully agree, is that it's unsustainable for people to work with deadlines all the time. But I think that discarding deadlines entirely can be competitive suicide. There is a way to put in place deadlines in particular strategic circumstances, assuming (again as the OP implies) you have some control of either scope or cost.
There are some deadlines that are too short that can't be hit, but if you have a project your team has done before in 4 weeks and give them 8 weeks, it'll take 8 weeks. The reason is it can always be improved. There are always more edge cases and more automation that can be added. Plus, many people would gladly work at half speed rather than full speed if the outcome was the same (daily compensation, praise, success, etc.).
I have $1,000,000 in the bank. I have 10 engineers. If we don't make at least a break-even income in 6 months I have to fire 10 engineers. So you have 2 months to build something so I can spend 4 months hyping people up to buy a bunch of it or we close down. And I know your engineer asses ain't gonna write bug-free code.
For instance, buying a home thinking that a new highway will be done around the time you move in could lead to multiple years of disappointment. Voting to allow your city to provide substantial financial incentives could enable the builder to go to extraordinary lengths to get the project done at the appointed time.... but even then, only within reason.
...to expand a bit: you're more likely to benefit from correct forward looking estimates if you have sound alternatives available when those estimates to be less than completely accurate.
Sprints are timeboxes and not deadlines. Stuff is done at a "normal" pace and it either fits (or does not) in that timebox.
Fundamentally, it does not really matter in itself. Noting down whether the estimation was correct or not, though, is important.
The whole point is that a team is supposed to get more _predictable_ over time. They learn how to estimate better, and having a fixed "box" to fill is a very easy way to do so.
So: Scrum is a "scientific" way to reach a sustainable pace ("scientific" meaning: based on objective, repeated measurements).
This is not to say that Kanban does not work. It's just way way harder to do right, because measurements of how good a team is at having a sustainable pace are not as intuitive and are easy to ignore for teams -- and ignore it they do, unless they are very well disciplined.
But all of this is well know in the world of Agile, I remember talking about this ten years ago. Perhaps an agile coach would truly help this person.
My own understanding of Scrum was, that you can commit to a deadline for the whole project (say, we'll have this application ready by the end of Q3), and then use sprints and backlog prioritization to manipulate the project scope. So, when you hit the deadline you always have ready to use product, just maybe with less functionality than intended.
Most people who say they are doing Scrum or think they are doing Scrum aren't really. Scrum is far from perfect but it's better than most of the bastardizations people invent for themselves when they claim to be doing it.
If your proposal for fixing it is for people to try harder to do it correctly then you might have a bad framework.
The point is that deadlines are bad for the people doing the work when they can't influence the amount of work. Not that deadlines should be removed completely, but they should be moved to a level where the amount of work can be influenced, namely project/product management. They should then, completely independently from the development team, adjust the backlog and/or deadlines in case the project schedule starts to slip. If the deadlines are needed because there are doubts that the dev team is not working as hard as they reasonably can then there are bigger issues that any project management method cannot fix.
The reason for deadlines in software, they way they are used today, is simply to get more hours out of a week from a software developer. Unlike in construction, software developers do not get paid hourly, nor do they get overtime pay. Abitrary micro deadlines coerce developers to work extra hours which makes their cost per week for amount of work to go down.
In construction, if they made up an arbirary micro deadline that was too short, the workers would get overtime pay at 1.5x their normal rate. Plus the additional hours. So for them, a too short deadline makes their cost per week for amount of work to go up.
We debate endlessly about all these estimates becase we're all pretending its about things that its not. Its almost completely based upon getting more hours per week per developer, as it makes the labor less expensive.
To find a solution within the constrains of this problem-space there need to be made tradeoffs that require creative thinking at every level.
I used to focus on technical work until I realised we where spending millions on projects that go nowhere.
Like in the Dilbert comics its ok for management to wast the time of the developer but a developer should not be allowed to wast his time. It may sound like a cop-out but its much more productive to be able to say, ever so politely, "no this is not something we are going to do".
Sprints are about accountability. Estimates are supposed to change based on new information, the understanding being that things change- priorities, desired outcomes, designs, sick time, and so forth. Short sprints are designed to surface new information as quickly as possible, rather than waiting until the project is 90% "complete" timeline wise but only 40% effort wise.
The Dilbert comic shows a false equivalence; management's expectations are simultaneously deadline, cost and feature driven. There is no management technique that can account for all three of those; not even Kanban.
This, so much. If you're missing sprint deadlines, its not necessarily a personal failure as much as a process failure. The solution is to commit to less, to refine your items of work so they can possibly be met in the time you expect. For this to work though, there has to be buy-in from the team members: you can't force it on them.
I've also found that the somewhat artificial sprint goals do act as a good motivation for the team to help each other if they see that the goals might be missed.
There's also the part where you have to be continuously getting feedback and tweaking the ways in which you run the process based on the feedback. If team members feel that this is just another unresponsive management tool designed to get them to work more, it will fail miserably. Instead, if they see it as a construct to provide accountability to stakeholders, they will strive to change and improve the process until it works well.
Ultimately though, the key feature seems to be: the people. If you're not working with a team that's genuinely interested in accountability and helping the product succeed, this process will not work. These are high-trust, highly nimble processes which are completely inefficient if they are actively sabotaged or the promises of good-faith expectations are violated.
Is this true?
I was watching a talk from James Coplien the other day and he claimed 50%. That seems extreme.
1) The task was not correctly estimated; it is discovered during the sprint that more effort than originally thought is needed.
2) Work that was not accounted for during sprint planning reduced your capacity; things like ad-hoc or unplanned meetings, sick time, peer and code reviews, HR processes, bugs from previous sprints, and so forth.
In both cases, the plan that came out of "sprint planning" does not match reality. I don't consider this to be a failure at all. How people react to it can be wrong, however. "Just get it done anyway", failing to inform team leads / managers when the change occurs, etc are unhealthy, because it creates a disconnect in expectations among the team due to either conflicting understanding of the work being done, or changes in expectations about capacity required.
So this isn't about failure at all: a failed Sprint is one that fails to meet the Sprint goal. I suggest taking a Scrum class to learn how velocity works, how agile works, and to learn how to manage uncertainty (and if you don't admit to this 50% uncertainty, then you are lying to yourself and cooking the books) and to learn about Sprint Goals.
Your post suggests that you understand none of these things. They are all Scrum basics.
I suppose when the product you're making is, say, advertisements, because you merely arbitrage, you can satisfy all of those at the same time.
The more interesting point of view is, "The kinds of software businesses you want to get into require no management at all."
I did in fact work at a company that was something of a dev shop for marketing agencies, and it was pretty miserable at times, because they inevitably expected contracts that held to all three. Cost was typically the one thing we could get them to budge on as new information came up (i.e. this necessary feature wasn't accounted for) though who actually ate the cost depended on negotiations.
I don't think I would enjoy working in a business with no management at all, to be honest. There are things that I am good at, and there is a definite perspective from which I view things, and the same applies to good managers, and there is not a complete overlap between the two. On top of that, they deal with problems that, often as not, I'd rather not have to worry about.
Nice work if you can get it.
Having milestones and deadlines in whatever process you choose to implement is very important to keeping everyone's eye on the ball. The problem comes in when there is not flexibility for the necessary tradeoffs, and it becomes a death march. This is management failure plain and simple. Taking away deadlines may superficially be better for people's health, but it doesn't help the company be more effective which in the long term could potentially lead to everyone losing their job which is ultimately way more stressful.
A set of experts (including the author of the book) sat down to complete an education project (like a program or something). They estimated it would take them 1 year, at the most. It took them 7 years.
It is extremely difficult to estimate the needed resources to solve a problem. Especially a problem you have never solved before.
Sometimes you have things to do with high estimates (many days) that can't be reasonably split into smaller pieces (the "you don't know what you don't know, and what exactly has to be done" situation). But still, it doesn't look good to put a too highly estimated task in the sprint, so you instead split it into fake "Do X, part 1" and "Do X, part 2", which is basically creative accounting to satisfy the tooling.
In that situation, the Scrum theory says to create a "spike" to figure out what to do, and then create the proper stories, but if often doesn't work this way in real life (in particular, if you need to discover and then deliver ASAP, this doesn't help at all; also often you're in "continuous discovery" mode, i.e. you do stuff, discover what else is needed, and so on, until you're done).
A solution to overpromising is to create "bonus" tasks that are kept in the backlog and not put in the sprint, but this lowers the incentive to take them since they're not formally in the sprint (and also who's taking big out-of-sprint items while there's unfinished small stuff in the sprint?)
Adding all the unpredictable things happening during the sprint, basically the sprints are creating more problems than they're solving.
The only good things about them are:
- SLAs for other teams for higher level planning
- a way of saying "no" to incoming requests ("sprint content full, sorry")
You can adopt the product delivery point of view and track progress on something like Kanban board. You track actual deliveries of actual items when they get done and on each time box (eg. 14 days) you open the board and try to understand blockers by looking at kanban columns where items are piling up. If a column grows beyond certain size (work in progress) you should start investigating the reasons evsn sooner. This way you either keep delivering or not and in that case you use periodic sprint meetings to understand why not.
Unfortunately what happens in organizations is that over time the organization based on tracking progress of delivering items or products shifts to tracking work or time workers spent working.
So on periodic 14 day meetings instead of asking questions 'why hasn't this item been delivered yet?', 'what is blocking the progress in delivery?' or 'why are these items piling up in that column?' you get question like 'what is everybody doing and what they plan to do?', ie. typical status report and micromanagement.
The focus shifts from items delivered to work done by people.
As the team then loses focus, managers start to impose deadlines.
It is another case of treating symptoms instead of looking at the causes.
This misses the key power of the dev team in Scrum - to say "no":
"Can you do that quicker?" No.
"Can you cut corners?" No.
The dev team may not be able to change features or scope, but they absolutely can tell the product owners to either change things, or accept what the dev team says they can do in the allotted time.
Who would have a better idea? Probably someone who knows how to paint.
It has been like that since I studied at uni and it continued like that throughout my career.
So to say they are bad for me seems silly. Different people work in different ways. :)
> Suggesting that meeting a sprint by cutting corners is a thing.
Yes, in bad scrum implementations.
SCRUM and KANBAN have both pro's and con's.
If you can't implement SCRUM in a good way, you will also have a problem with KANBAN.
There some good discussions about on online like https://fenix.tecnico.ulisboa.pt/downloadFile/3779576751814/...
> How much effort goes into each task?
"how large is the story compare to other stories we had before?" is a better question.
Refinements are a crucial meeting to keep the team aligned and make sure they can collaborate on user stories and help each other.
> How many sick days are there going to be in the sprint?
you don't plan for that.... you only plan for things you know in advance (vacations etc. )
Capacity planning takes less than 5 mins in our sprint planning. not sure where the problem is.
> Is someone leaving or joining the team?
Yep, very important to discuss. in any context (regardless if you do SCRUM or something else).
> people end up confusing sprints to deadlines
End of sprint is a deadline according to definition:
> the latest time or date by which something should be completed.
Because the team thinks it can complete the sprint's stories by the end of the sprint.
It is important to deliver value to customers in a timely manner. This is to lower the risk and prevent multi month/year projects that never go to production, etc.
Sprints help you with structuring your process to achieve this goal. Please keep in mind, this doesn't free the team from thinking how to get the value to the customer as soon as possible.
(You can have sprints and never go to prod...)
> trigger the fight or flight response
If you have people in the team that have that kind of reaction ? Talk about it in a retro, and improve.
SCRUM requires a "save" environment, if you don't have one, almost no sane process will work.
if you don't meet your sprint goals. talk about it in the retro and improve as a team.
Consider, pulling in experience good SCRUM master's the good one's can help how to improve your procceses.
One thing to keep in mind, if the SCRUM is only a thing of one particular department and/or is not fully accepted by the CEO and board, you have some side effects where situations can be quite disappointing and stressful.
however, the scrum master and PO can help sometimes in these situations. If they can't the SCRUM process will be bypassed by people outside (like the CEO) and hence it undermines the process and makes it less effective.
If you've been working as a developer for any length of time, you should know roughly how long a particular task is going to take you.
if you work in an inverted pyramid where the team has autonomy (within boundaries) - scrum doesn't become a reporting tool it is for the team. since there is nothing else "above" the team.
The story points are a measure. That help your team to plan a sprint nothing else.
even some devs to themselves. that's usually totally counter productive. working on getting better is usually the way forward.
That's what matters.