Hacker News new | past | comments | ask | show | jobs | submit login
How big tech runs tech projects and the curious absence of Scrum (pragmaticengineer.com)
275 points by yarapavan on June 11, 2022 | hide | past | favorite | 353 comments



My take as a TPM and certified in Scrum: the better and more skilled the team members, the less you need Scrum and other frameworks. Scrum is great for teams composed of new developers who don't yet know how to work together, or teams at companies with poor culture. But the best teams are self-organized and don't necessarily need the guidance of Scrum. As Gergely mentioned, team autonomy is a big factor in job satisfaction (and performance).

But, it can still be worth doing in high performance environments if specific things are needed. Part of being Agile is adapting processes to teams, and some Scrum processes can be useful (relative estimation for example) even when not doing Scrum itself.

As an aside, Gergely has a newsletter[1] (of which this is one of the free articles) that is absolutely fantastic. It's $100/yr IIRC and well worth it. Gergely deep dives into topics around tech including hiring, structure, comp, etc.

Gergely also has a book[2] that helped me get hired in SF when I moved here a couple of years ago.

[1] - https://newsletter.pragmaticengineer.com/about [2] - https://thetechresume.com


All big methodologies and big application platforms (e.g. J2/Jakarta EE,.NET) are attempts to make average developers predictably productive and to ensure long-term consistency in and across large projects. These things are more important to "enterprise" than raw productivity/development speed.

Highly talented developers don't need this structure and process, but if given the freedom, they are also more likely to create software that only they understand, using flavor-of-the-month languages or frameworks that they have personally selected. When their work is done and they move on, nobody else can figure out what they have done or why.


I disagree. There's nothing interesting about pursuing frameworks and languages of the month. The highly talented, intelligent developers I know don't care about that stuff - their actual work is way more interesting.

I think what you're describing is average to slightly above average developers who are working on boring, already solved problems. So they have to find something to make it more interesting.


I agree the best teams have been more “keep it simple and focus on making cool stuff” than being focused on trying random new frameworks. But Scrum is not a popular choice for keeping it simple because it comes with a lot of baggage.


Scrum is pretty small. A lot of the threads on here are mentioning all sorts of Agile tools that Scrum does not require. Those extra things are optional if they help you but defined outside of Scrum (like planning poker). And if you dislike one of the few things in Scrum such as the end of sprint retrospective, you can take it out. So a lot of people think they’re doing Scrum when it’s really Scrum plus other stuff, and they think they hate Scrum. Experienced developers may not find it helps them all that much, but to me, the important other side of the coin is that when you try to teach the powers that be about Scrum, it helps educate them enough that they don’t mismanage the developers too much. To management, Scrum tries to say don’t come in and add requirements every other day during the sprint so the developers can at least have a few days of focus to actually ship some code. I’ve definitely seen overeager product owners come in and hover over developers more than is healthy for the project even AFTER being educated on how Scrum is being followed and they should pull back and give a little breathing room! If I had a nickel for every time management pulled people aside to tell them more stuff to build and then shortly after ask why something they asked for weeks ago isn’t done yet…………Scrum tries to give developers a way to communicate what needs to happen for them to ship! Maybe in some purely tech companies the folks in charge already get it and don’t get in the way so the developers can just have at it, but those are rare amidst all the types of businesses employing software developers.


There are lots of ways to make a project more interesting. These eager beavers don’t have a broad enough world view to channel that energy into other things. Some of which are maintenance neutral, others which are maintenance positive.


You are incorrect. In my 18 years in dev, I've seen quite a few of those fabled 10x or 100x devs, and oh boy they can deliver if unconstrained by processes and bureaucracy. Rest of the team then scrambles to figure out how things work once they move on to next cool thing.

Always some cool new framework (or even better, one developed just for this situation because its fun). All of them, no exception, loved exploring another framework du jour (and java has/had tons of those) for each project. None of them were 'finishers' - ideal for them would be prototype something quickly, and then move on, daily project grind was universally hated.

Few had to be treated like babies by managers - finding new topics hey can play with to maximize contribution to the project (which often paid off, ie once we got 10x throughput increase compared to default ActiveMQ installation, the guy did some proper debugging/perf testing only possible due to open source nature of the product... brilliant guy, but hardly a team player, complicated personality).


I have no doubt such people exist, but you're generalizing needlessly.

I've met plenty of senior devs that stick to target and idolize maintainability.

No point in insisting that your anecdotal evidence is universal (by asserting "you're incorrect").


Isn't the post he's replying to doing the exact same thing?


Maybe it does, I'd still disagree that's its main point however.

Obviously claiming extremes is always wrong. But I'd still wager that most people who are indeed senior devs -- myself included -- are long past the "let's try this new thing because reasons!" thing.


> Highly talented developers don't need this structure and process, but if given the freedom, they are also more likely to create software that only they understand, using flavor-of-the-month languages or frameworks that they have personally selected.

I think you might be using a slightly incomplete definition of “highly talented.” My highly talented colleagues take maintainability and minimizing tech debt into account when making technical contributions.


I do, and I consider myself "highly-talented," as well as "extremely experienced" (over 30 years of software development -shipping- experience).

I'm also an excellent "team player." I worked for a Japanese corporation for almost 27 years, and you don't last long at Japanese companies, if you don't "team" well.

But I also don't shy from using advanced techniques; usually in language idioms, as opposed to ungrokkable algorithms. Most of my algorithms are fairly basic. I tend to go after difficult problems, though, and it's ... difficult ... to solve difficult problems, simply. Sometimes, I have beautiful, low-code solutions, sometimes, I have solutions that would send Rube Goldberg into fits.

They all work, though; and they ship.

I expect the folks that maintain my code (usually Yours Troolie) to be at a level similar to mine, and I won't "dumb down" my code, so someone fresh out of a JS bootcamp can understand it. I have an insane level of documentation, and supporting artifacts. If someone is able to understand the code, they will get a lot of help.

I'm militant about Quality. My code needs to work well, be performant, be flexible, and to have clear, unambiguous boundaries. A great deal of my work consists of reusable, standalone packages (I use them heavily, in my projects; Most of my dependencies are ones that I wrote). This affords an intense level of Quality. It's abstraction with teeth.

I'm used to owning the entire project lifecycle, from napkin sketch, to shrinkwrap. This gives me a great deal of insight into what is necessary for "the long game," without getting caught up in "bikeshedding."

My experience is that most companies that I've encountered, don't want people like me, working for them, so you get what you get.


How often are they around 2 years later, though?


I've written SDKs and infrastructure software that has lasted decades. The secret is usually to use boring tech, and inline documentation.


I meant the devs, not the software :-))


This is true. I think it's a huge problem, but everyone seems to be happy with the situation, so what do I know?


> Highly talented developers don't need this structure and process, but if given the freedom, they are also more likely to create software that only they understand, using flavor-of-the-month languages or frameworks that they have personally selected. When their work is done and they move on, nobody else can figure out what they have done or why.

IMHE, not so much. You can't tell which code they wrote, except it's easy to read, and there's a lot of it. The loudmouths, they write inscrutable, flavour-of-the-month code. The talented coders aren't as bothered by this as much as everyone else is.


Highly talented developers don't need this structure and process, but if given the freedom, they are also more likely to create software that only they understand, using flavor-of-the-month languages or frameworks that they have personally selected.

Micromanagement processes and resume driven development are both problems but they are very different problems. IME good developers aren't going to be the ones pushing for all the shiny new toys all the time and writing inscrutable code. Good developers value stability and simplicity and they choose tools that help to get the job done. Good developers also write code that is relatively easy to understand.

The kind of developer you're talking about above isn't highly talented. In fact they're exactly the kind of dangerous mediocre developer that many of these overly controlled development processes evolved to contain. But the best ways to deal with them still aren't micromanagement. You can either teach them to be better (if they have a positive attitude but don't yet understand why what they're doing is often a bad idea) or fire/not hire them (if they're the mercenary type who only wants to exploit your salary payments until they can boost their resume at your expense and jump to exploiting someone else for bigger salary payments).


>or fire/not hire them (if they're the mercenary type who only wants to exploit your salary payments until they can boost their resume at your expense and jump to exploiting someone else for bigger salary payments).

Instead, most companies incentivize this very behavior. I wouldn't call them "mediocre developers" when they are playing the hiring meta game correctly, even if it leaves a wake of destruction behind.


I agree. That’s a net negative outcome, I would expect mediocrity to have a moderately positive outcome.


In my experience, documentation dodging is practically institutionalized in Scrum. Sorry, the sprint ended, I’ll do that later. I’m not saying there are good processes for guaranteeing documentation, but I am saying Scrum is amongst the worst.


People basically used Scrum or that part from the Agile manifesto as a get out of jail free card for not doing any documentation. Probably the single worst thing that ever happened in the history of software development.


Documentation task could be included as part of sprint / or task Definition of done.

In true Scrum spirit, the team that has problems with lack of documentation, should figure that out from (1) retrospective, and start (2) including that work in Sprint explicitly, and (3) to explain to business owner that without these tasks, features are gonna get delivered 3x slower.

Benefits: documentation/testing will be explicitly required as a requirement, time will be allocated and accounted for, business and stakeholders will be aligned on that.

This doesn't happen because scrum leaders never care enough about the product, but at places where they do care it is well documented and structured


What works best from my experience is to include documentation in the DOD. Most of the time if you build a feature there are emails, tickets, notes from meetings etc. To write the docs is just a quick editing job in many cases.


I've heard that suggestion a few times - just plan for documentation in your sprint. It's a nice idea. I've never seen it done though.


I write inline documentation. It's usually headerdoc-style stuff. Tools like Jazzy, DocC, and Doxygen make it very useful[0].

I'll often write the documentation before the functionality. When I'm done, I run Jazzy, and I have an indexed, navigable, relevant, up-to-date document set. I point GH Pages at it, and voila! You have a well-documented codebase.

It also encourages brevity and keeps the focus local.

[0] https://littlegreenviper.com/miscellany/leaving-a-legacy/


Acceptance criteria include documentation. Don't document, don't close the task, don't pass go or collect $200.


Really unclear on why you would have this perception. Most good devs I know appreciate the importance of a stable toolchain. I’ve worked in nearly a dozen languages at this point and it’s pretty rare that someone uses a language for the sake of using a language.


> Highly talented developers don't need this structure and process,

Indeed. But in most real world cases, this is because they’ve internalized good process and shaken off the categorical constraints that more junior or less disciplined developers still need to rely on.

While the archetype of the idiosyncratic graybeard poring out inscrutable, magical code has exemplars, it’s not the norm that many of us see when we look around.

During periods where software engineering becomes more collaborative and corporatized (as we’re in now), it’s hard for these types to find a home. Yet there are hundreds of organizations stocked with many highly talented developers doing “good” work well.


I am pretty sure only dunning-kruger/kinda smart but very inexperienced developers write unmaintainable software like that. The best developers I know prioritize ease of shipping + maintainability and only make things hard when they need to be.

The type of person you're describing I have run into a few times but usually they are not considered highly talented at FAANG, more like the type of person to quickly hit career wall because they overly focus on coding instead of business objectives.


I don't think Agile/Scrum provides any guard rails against "bad" high-level technology or architectural decisions. Which is fine: no pre-canned process really does that.


I wouldn’t call these talented.


> Scrum is great for teams composed of new developers who don't yet know how to work together

I couldn’t disagree more strongly. My experience is that scrum teaches junior devs bad Han it’s around blame shifting, focuses them on process, prevents them from learning about the business, and bogs them down in meetings that don’t help them learn.

> or teams at companies with poor culture

I would argue that scrum ossifies bad culture. It has a habit of giving bad leaders metrics around things they shouldn’t be measuring.

There’s a lot of software engineering research about processes and the case in the literature for scrum is very weak.


That sounds like you weren’t doing scrum right. It’s also possible I wasn’t doing it right and was doing a healthy version of it?


Every scrum proponent always responds to any criticism with a no true Scotsman claim. I’ve never personally seen or heard second hand of a successful scrum implementation, and the SWE academic literature doesn’t support it either.

I’d argue that spending time on activities like scrum poker or sprint planning are actively harmful for most kinds of teams. The points games are inherently adversarial, only add value for scrum masters, and waste time that could be spent understanding requirements/business problems better. The incentives are inherently perverse and the whole exercise encourages and rewards dishonesty.


Well I feel it was successful on my old team. We weren’t micromanaged and owned our own process. We story pointed and those metrics were only ever seen and used by us. We got tons of value out of retros (had loads of difficult conversations with high levels of trust). We didn’t have a scrum master—everyone knew how to run all meetings and process. We worked closely with our customers doing rapid prototyping as to not waste time building features they only thought they wanted. We showed progress to the org via demos. I feel like it worked really well and really enjoyed it.

As far as no true Scotsman, I didn’t mean it like that—I don’t care if people don’t want to use scrum, but I do take light exception when people shit on it an go on to describe a process that is alien to me. Though I guess what my teen was doing was a bit more of a mix of scrum and XP.


> We story pointed and those metrics were only ever seen and used by us

That seems to be the exception rather than the rule in my head experience. Even still, what concrete value comes from assigning fairly arbitrary effort estimates in this fashion?

> but I do take light exception when people shit on it an go on to describe a process that is alien to me

Respectfully, I’ve seen a bunch of scrum implementations and attempts in that direction and they have a lot of overlapping pathologies. I’ve also gone to the source material. IMHO scrum simply wastes time on meetings that give a false sense of visibility and create a lot of negative incentives. For those costs, I’ve never seen a return that was worth the trade off.

It sounds very much like you were doing some flavor of XP with a few scrum meetings sprinkled in. This isn’t the process scrum evangelists and consultants are pushing. But I’d still argue that most story pointing meetings are wasteful.


There another issue I've seen with all the forced estimation that gives questionable values to the business.

When devs overestimate they then tend to take the extra time to overengineer the solution, and when they underestimate they tend to take on technical debt to make the very arbitrary deadline.


I think it also incidentally discourages the trade off conversation by artificially time boxing work.


> what concrete value comes from assigning fairly arbitrary effort estimates in this fashion?

This sounds counter-intuitive, but I’ve found in many teams the value of story pointing is to instigate healthy conversations about requirements and surface assumptions in a team that might be reticent to engage in those kinds of conversations naturally. The actual point values are probably the least important outcome - any inaccuracies tend to average out over time anyway.

I’ve gone in to many teams in my career where “planning” amounts to 1-liner tickets describing a month’s worth of work, and the team has a few dozen of these in the backlog. Conversations about what is actually involved in delivery don’t happen, and execution is chaotic because everyone has a different understanding about what we’re building.

This sounds like a dysfunctional team because it is. A surprising number of teams are dysfunctional, and many of these frameworks and tools come out of attempts to solve for dysfunction.

In these teams trust and morale tends to be low. The abstraction of story points (and associating them with complexity, not time) opens the door to having conversations about what the work actually is rather than how long everyone will take to do it.

It’s a lot easier to hear “why do you think this is a 5-pointer instead of a 3-pointer” instead of “why do you think this will take 5 weeks, I think it should only take 3.”

Scrum isn’t a universally-applicable tool, and introducing story points without also leveraging their ability to stimulate conversations about the work is a waste of time, but applied well they can help break through to teams that aren’t ready (or don’t have the muscle memory,) to have these conversations without some kind of trigger.


Having been in a team that needed to have these conversations because many were ramping up on the work, these conversations were godsends for catching inaccurate and problematic assumptions, making sure people understood how their work supported the business, and enforcing consistency with how the work was organized and documented which helped as the team grew and turned over.


That’s fair. Honestly I haven’t gone too deep into the source material. Most of my knowledge comes from two former coworkers and current friends who were well versed and experienced in XP and agile and had a huge impact on the engineering culture at the company I was talking about.

I actually don’t disagree that story-pointing feels like a waste of time, but I do find it useful to talk about what’s involved in a task and figure out how long is might take. I don’t think it’s unreasonable for management to want a rough idea of when something will be delivered, mostly so that they are able to plan themselves. Of course, it all depends on endless variables as to how much process a company needs. In larger companies with multiple teams, I strongly believe in self-organizing teams with a well-defined point of contact and freedom to implement as much or as little process as they want.

Edited to fix horrendous typo as a result of typing on my phone. I likely didn’t catch them all.


Yeah, I think periodically talking with product folks about the nuts and bolts of a task is really useful. It’s even quite useful to think about when tasks are becoming large. That sparks a lot of good conversations. The specifics of how to best do that will vary a lot team to team.


We do it pretty lightweight as a way to estimate the whether there’s the right amount of work to fill up ~2 weeks. Senior engineers get a lot more leeway to reprioritize on the fly.


That’s not entirely unreasonable but also not the prescribed method I’m criticizing.


I get that. In general I find scrum/agile to be unhelpful when run by people who don’t adapt it to the team/business needs.


> what concrete value comes from assigning fairly arbitrary effort estimates in this fashion?

My team asked for this. I dont know why they need it but apparently it helps them understand how much work theyre agreeing (and committing to shipping) in a sprint.

I think if you have a good grasp on what each issue will entail, you dont really need it but if youre less experienced maybe it helps?


But that just begs the question, why sprint? Two weeks seems arbitrary, and meetings on that cadence are pretty noisy. Trying to deliver in that frame can either lead to dead time if things go faster than expected or can lead to artificial failures if you miss it. Why set yourself up to fail that particular way? I don’t think it’s especially useful for organization or accountability and adds a lot of noise and waste.


Of course its arbitrary, but you gotta pick something. We optimise for progression. Failing a sprint doesn't actually have any real ramifications. What we use failure for is a signal to indicate problems. Missed? I guess we have some talking to do in retro to figure out what was misunderstood. Missed three sprints in a row? Is there a problem we need to resolve?

Similar to misses, we don't really get much from shipping on that cadence either, other than progress and building confidence in upper management to base their timelines on us delivering what we expect.


planning can be useful if stories entering the queue are refined in real time and the team knows where they are going. best case, it's a ten minute "here's the backlog; who needs work?; what are we shipping this week?" conversation...but this assumes a high-performance team of go-getters where every story is vetted well enough for anyone to pick up

(that convo can be had on slack, btw, just like standups.)

the problem is "scrum masters" waiting to refine/reject stories during refinement (or not refining at all), or engineers not updating their cards frequently enough, or nobody guarding the backlog and allowing anyone to dump stuff into it. when these things happen, you get three-hour-plus-long sprint planning that wastes everyone's time

the other thing I absolutely hate are capital-r-Retros. retros are supposed to be the meeting that everyone looks forward to, but they often get skipped because it's an hour of people sharing platitudes because the room isn't safe enough to REALLY say what you feel. if i ran retros, they would be at a nearby pub or chill restaurant, over beverages and food, with no real structure other than "at some point we should talk about how this week went."


OMG yes, the retros at my current company are just full of platitudes (and that’s what people look forward to) and I hate it. People do bring up difficult topics but because want to get through every. single. topic. the difficult convos get rescheduled to yetanothermeeting so there’s time to talk about all the positive stuff (which, again are just platitudes, nothing constructive). I tried to change this and got shot down.

Otherwise (and I’m forgetting which thread I’m in here so maybe repeating myself), I have experienced incredibly valuable retros. They were structured but still fun.

And very much agree with the first paragraph. I also agree it could just be done on Slack, but I personally like having a daily meeting where I get to see my team all together… so long as that meeting is short (max 5 mins on a good day).


Genuinely curious about this: is there a specific white paper (or collection) to which you're referring?


I’d have to dig back into the literature, I don’t have anything at hand. Basically for every pro-scrum paper you’ll find a criticism. A few people have done meta analyses and found no benefits. There should be something in IEEE but I’m on a plane right now.


the good thing is that when you see that Scrum process is broken and is not going to improve, that is as clear signal as it can get for a developer to quit to a better place.


That would be my recommendation.


Scrum (stand up and all) is a waste of time and only lower tier companies are still using it.

That doesn’t mean that you don’t have a process at all. You might have once or twice a week sync meetings, and a biweekly planing meeting.

But scrum as it is traditionally thought (Daily stand ups, scrum master, agile coach) and all that hoopla is thought to be out of date.

If a company is spending 500k for an engineer the last thing they want is pointless meetings wasting their time, or have a non technical lower level agile coach try to dictate things (which I have seen it happen).

At that level the expectation is that the engineer is mature and will know how to unblock themselves and doesn’t need infantile hand holding.


I remember working at companies where we'd have meetings once a week to discuss what people were working on and plan out the next week. Daily "standup"-style meetings were absolutely unheard of. If you had a "blocker", you'd check in with a coworker or boss, discuss it, and work it out, like a normal person. You're not sitting on your ass to announce "you're blocked" in a daily standup.


I agree so much with this that my blood pressure just went up.


Thing is that scrum is for kinda hostile enviroments where developers need some safeguards and peace agains adhock requests and constant ops works.


Yeah; the thing to remember with formal development processes is it's to try and force something upwards, not downwards. If the team is being forced to adopt Scrum, and the rest of the business isn't willing to adopt the contract placed on them (for Scrum the biggest is the business can't interrupt the sprint or -nothing- is a reasonable expectation from the sprint any longer, and any estimate is null and void and they don't get to ask why the team missed it without a finger pointing back at them), it's not Scrum that is the problem.


There are plenty of sub-500k TC engineers getting around, and the vast majority of companies are “lower-tier” by your implied definition.

These companies won’t succeed if they try to act like they’re working with mature engineers, because they’re usually not.

Skills like knowing how to unblock yourself, how to identify systemic problems in a team and correct them don’t come naturally to a lot of people.

If you’re not encountering teams like this I’m happy for you, but lower-performing teams often benefit by adopting a rigid methodology to kick-start the process of self-improvement.

In an ideal world, they further transform their working style into something that works for them by reflecting on their experience on a regular basis.


I've seen agile coaches be amazingly helpful...generally explaining to the VP why "pinning all three points of the PM triangle" is impossible, and why they have to pick either date or scope, not just throw headcount at it, if they want to continue to claim to be agile (let alone be successful.

That said, every agile process needs to be treated for what it is; maybe a decent starting place, and probably a collection of good ideas for the context it grew out of, but not directly transferable; the whole point of a retro (the one meeting that the agile manifesto actually stipulates) is to modify the process for the team.


Frankly, I think you're already fucked if you need an agile coach to explain that to a VP leading a software development project. Either the VP is grossly ignorant of what they are doing, or they have structural incentives that make building good software of secondary, or tertiary importance. I've never seen mere coaching fix this.


Is an agile coach a PM? Or do they function more like a (lifestyle, executive) coach does?


I feel like that is misunderstanding scrum. For example, standup is not supposed to be a status report, yet so many companies use it as one. In my mind it’s just a daily ritual where the team comes together and very quickly discusses the day. It should last no longer than 5 minutes. I understand that not everyone even wants that, and that’s totally cool, but standup gets a bad name since most people have experienced it as a status report.


Isn't the big point here that the daily stand-up is meant to be for communication within the team, not for reporting to management or stakeholders?


Indeed. But lots of places, including my current employer, don’t follow this.


I think the reason people end up using it as a status report is that nobody understands what to say otherwise. I can either say only "I'm not blocked" every day like a broken record (because if I were blocked why would I wait until the next morning to say so...), or I can say, "I'm not blocked and here's what I'm up to today", which is just a status report.


I don't see the problem with the status report, as long as it's the culture that you usually only need a sentence or two.

Imo in bigger stand-ups, there's more pressure to sort of "show off", which means talking more, and combined with having more people there already it takes forever and is really painful.

But if it's just a handful of people who work together saying a couple sentences each on average, unless they're blocked or something interesting happened, it's fine.


Yeah I mean I prefer not to do it, but I don't hate the "small status report" kind of stand up. But I just question the "it isn't a status report!" line of explanation. If it isn't, then what is it? I often hear in response to this "it's just for blockers", but I have always found that particularly weird, because who just stays blocked until the next morning?


Ya, the "just for blockers" doesn't make sense. As far as I'm concerned, it's just time at the beginning of the today to come together to figure out what needs to be done for the day. The issue I take with "status report" is that I don't care, at all, about all the little things people did yesterday. I usually just tune out and I'm most others do as well.


It’s supposed be more of a quick planning meeting. But yes, I would often just say “pass”. Although really as far as I’m concerned it’s all about the ritual of coming together every day.


I "quick planning" is an oxymoron. If anyone says "I plan to do this today" and the response is "actually you shouldn't, you should do something else", that one thing is going to be at least five minutes of discussion. If that happens twice or thrice it might as well be a half hour meeting.

And I actively dislike / don't understand the value of that sort of ritual...


It’s about coming together as a team as far as I’m concern and get energized for the day. It doesn’t work for everyone and that’s ok. “Quick planning” is like, “Hey, I need with this today. Is anyone available?” But really it’s about coming together for me.


I don't want to diminish your experience, I recognize that this makes sense for you, but it's a "to each their own" situation and I want to give you a perspective from the other side: I find stand-ups to be the exact opposite of energizing. The way I get energized is when I have something I'm so excited about working on that I've spent the whole night or weekend having to remind myself to stop thinking about it and be present with my family or friends, and then I sit down at my desk and get to dive straight into it. An hour or two later when I start following up on emails or doing chats or a stand up, that's fine, but it's not energizing, it's just when reality sets in.

I think this is just like the remote vs. in-person thing; it's mostly a personality trait. I don't have the "daily team ritual" gene, but others do, and that's ok.


Oh ya, if you read my replies around this thread (I’m not suggesting you should) I’m saying that it’s “to each their own.” I’m only defending stand up as a concept, but if a team doesn’t find it valuable they shouldn’t do it (same goes for all rituals and meetings). I’m realizing now there is a distinction on where the discussion is “I’m being forced to do stand up and it sucks” vs “This is why I like stand up.” As someone in the latter camp, if your standup is a soul-sucking status report (pr just plain soul-sucking), then atop doing it! And yet, your company is forcing you to do it. And that sucks.


Yeah totally, I got the sense that we agree on "to each their own" but land in different places. The unfortunate thing is that it's totally possible (indeed common) for members of a team to come down in different places on this. So you could have half the team's eyes glazing over every day while the other half of the team would lament losing the ritual.


I've worked for 15 different companies, and 13 of those had a standup. And of those 13/13 were a status report.


Congrats on being an engineer that can do these things. Many cannot and that is the bulk of teams out there. For those that are this skilled, this gives a window into areas they can flag for where they need to help level up the team.

But I'd urge you to reread your post and consider that it comes across as presumptuous, myopic, and holier-than-thou.


> Scrum is great for teams composed of new developers who don't yet know how to work together, or teams at companies with poor culture.

Those are just low-immunity weakened places that Scrum is able to infect.


Three years of the best anti-agile social media comments.

https://github.com/rayfrankenstein/AITOW/blob/master/README....


That certainly explains why so many competent engineers seem to hate it so much.


My experience is aligned with this. I would add fast growing teams as well. It is difficult to grow 10->150 in a year without some form of methodology in place.


Gergely is a great writer and has a lot of accurate insider takes. His Twitter is worth a read as well.


> "Scrum is great for teams composed of new developers who don't yet know how to work together, or teams at companies with poor culture."

Hmm. I think your take is a bit like saying "Baths are only for dirty people. People who stay clean don't need baths."


The analogy I usually use is

“a Michelin-starred restaurant doesn’t have laminated recipes on the kitchen wall. A McDonalds does. Both of these are a good thing, depending on who you’re working with and what you’re trying to achieve”


That is a fantastic analogy, it captures exactly the pros and cons and the context dependency of methodologies.


Good analogy!

Here's the problem though: how to you hire for a "McDonalds". It seems good to be self aware about whether you're running a McDonalds or a Michelin star restaurant, but can you be honest about that with candidates and still hire people?


It's funny. Many years ago I did work at a McDonald's, but the manager of that store refused to allow those types of posters on the walls. He said it looked bad, he wanted clean white tile and stainless steel to be what the customers saw, not a bunch of laminated posters.

He always argued with the regional manager about it during store inspections. But since this McDonald's made a lot of money, and everyone was actually preparing the food correctly, the store manager generally got his way.


More like: long-haired domestic dogs require someone to regularly groom their hair for them. They don’t have the instinct required to drive them to groom themselves as much as they need, so a long-haired dog left to their own devices outdoors will end up with hair mats and ticks they can’t reach to scratch off. Long-haired cats do not need the help of a human groomer to avoid these things, because they do have the instinct to chew off excess/itchy/matted hair.

Scrum masters — backlog / kanban board groomers — are for junior devs, who haven’t yet developed any sense for what an approachable task size is; what their comparative advantage is vs others on their team; etc. Senior devs do have that experience — and it’s surprisingly portable between teams/orgs.


I mean a dog is loyal, can be trained to do what you say but short-haired cats don't actually need baths like dogs do simply cause they take autonomous initiative on grooming and self-care


Great insight, my key takeaway is this:

> The fewer people you need to make decisions, the faster you can make them.

This is the only way I’ve seen a “rockstar” or “10x” get stuff done as fast as they do. They only have to talk to a couple of people at most to start making big changes. The more large and stable a company gets, the more they seem to want to spread decision making power around the org. This appears to be very counterproductive, and it’s nice to see the simple truth called out.


This is really it.

There's a world of a difference in what you can do if you have free reign to make most technical decisions with full autonomy (self guided R&D, picking stuff, spiking it out, doing it for real, etc.), perhaps only reporting back weekly outcomes to a CTO and also asking their advice as needed to align on big picture goals while nothing is ever blocked.

vs.

Most decisions needs to go through someone else, which then turns into an adhoc request for another department which then has to ticket it out and implement things on their schedule. They could be a great team and do everything in the best way you can realistically hope for but before you know it, things that were literally 1 or 2 hours of dev time end up taking 3-4 weeks simply because they have other things going on too. You keep repeating this pattern and it's how something ends up taking 6 months instead of 2 weeks. It's also one of the most frustrating things to be blocked. It forces you to context switch between many things to keep yourself busy. You spend all of your time reacting to eventually getting unblocked in an uncontrolled way instead of just sitting down and cranking it out. That means you might have to jump back into something you worked on 2 weeks ago and re-get all of that context back in your head.

I know in an Agile sense every team should be self sustained and can own their work from beginning to end but from an infrastructure perspective you may run into scenarios where you need resources created by another team because in a bigger org it's company policy. The blockers could be a combo of waiting on that other team or waiting until multiple people sign off on moving forward with XYZ tech.


One of the reasons enterprise organisations are so slow to adapt and release new products is the number of people needed to make a decision. I've seen large companies try to implement agile and it generally fails because they can't shift to quick & effective decision making processes. Greenfield projects that get developed quickly due to independence and have great promise for the future get the hug of death as soon as management decides to bring the project into the same decision making as the rest of the org.


This is the exact case that I have been in the last two years or so. As a greenfield project being one of the first in the newly implemented Agile work. Everything was amazing in the beginnig with quick decision making. However as soon as we 'went live' and were forced into the established change control workflow development grinded to a halt. We now implement features at a snails pace compared to before.


I feel this in my soul, I'm not even allowed to update internal documentation in our repo without linking it to a JIRA ticket.


I mean, thats pretty normal for any stage tech organization.

The question is, who gets the right to create a jira ticket? Maybe you can do it, or just one other person. Thats not a big limitation at all to moving quick.

Edit because this is causing a stir:

Engineers within teams should have the right to create tickets themselves. Tickets should be minimal depending on the task. Creating a ticket that says 'update documentation' may take 10 seconds. Updating documentation may require a pull request. Controls (SOC compliance) may require that work is tracked to tickets.

The core questions I have is, who can create the tickets, and how detailed do they need to be?


I disagree with it moving quickly, all it does is encourage me to not update documentation and not actively improve the code in other ways due to inane bureaucracy. And surprised surprise, no on else on the team wants to document anything either; people get burnt out because they have no autonomy and just leave.

All is good tho! We followed the process and that's the only thing that matters.

I wonder what the corporate equivalent to "I was just following orders" is?


If an engineer can create a ticket and work on it, thats not reducing autonomy. Thats adding accountability.

If documentation requires a pull request / merge to update, often you have set controls that require a jira ticket. Its super common to not allow engineers to make solo commits without a pull request.

If an engineer sees the creation of a jira ticket as a limitation, or if they are not empowered to create their own tickets, then Id agree its a problem.


That ticket is just adding friction to getting things done. Why can't you track the pull request itself?

This one instance of friction might but seem like a big deal at first, but the attitude is what causes slow development and burnout. First there's this tiny process, and then why not add a few more bita of process on top, until finally your merge process takes longer than writing the updated documentation, and so people stop doing it.

Instead of asking "why can't you just do this process," you should be asking "why is this process necessary at all?"


Imagine the inverse where you have a team of engineers working on their own projects. And you dont know who is working on what, or why. Because there is no pull request yet.

Who requested this change? What is the scope? Where is the upfront documentation?

Now imagine the PR is ready to be merged. Since the engineer doesnt like documentation (as you are suggesting) then theres little context into what this is doing, who it will affect, and what types of changes can be expected.

Now imagine those PRs create a bug and causes an outage. Where do you look to determine the context into why this decision was made?

Now imagine a dozen of these teams working on the same repo. How do you coordinate who is working on what and what types of releases need to be schedueled?


I don't know what to tell you. You're describing a highly dysfunctional organization with no autonomy and extreme micromanagement. The right answer to all your questions is "leave, and try to work for a good manager in the future."

It is useful to do project planning and task tracking at a high level. But it's counterproductive to open a ticket every time you want to take a breath.

I lead a highly productive team in a challenging technical area. I know roughly what team members are working on in a given week, and that's enough to do coordination. Individual engineers write good pull request messages, and supplement it with design or usage documentation when necessary, and that's more than enough to understand what's going on.


My comment above is describing a place that is all autonomy and no micromanagement to the extreme. Its a place I worked before I understood the value of upfront documentation.

The situation Ill describe as a good place to work is, you have tickets that are created (by engineers / product / business whoever) that are evaluated ahead of time, and the team determines if it will add value to the business /end user, scoped appropriately, and set up for success with timeline and work.

As many decisions as possible should be made by the team doing the work, and they should be empowered to change their process as necessary.


The micromanagement is implicit in your use of the word "you". "How do you know ... ?". Who is that "you"? If it's the person's manager, the answer is "because we talked about it in our 1:1 or on chat". If it's anyone else, the answer is either "don't worry about it" or "coordinate with the manager". It's good to have a plan for the week / month / whatever timescale, to help teams coordinate with each other, but you really don't need real time visibility into what a team is up to if you aren't on the team.


> Now imagine those PRs create a bug and causes an outage. Where do you look to determine the context into why this decision was made?

The PR? Whatever you'd write in the ticket you can just write there instead of splitting information between two places.


Is creating the ticket really adding value to the engineering process? In your example you already have the engineer making a PR (presumably peer reviewed) and a commit, what’s the ticket really needed for?


It can add value if done right. Just as you typically want to make a design doc / rfc before working on a project, you often want to describe what you are doing before you do it.

This can protect the engineer / team against scope creep, as well as determine the context of why work was done when you may need to understand more context (in the case of a bug for example).

Also it increases accountability and transparency. It is common that teams want to know what other engineers are working on and why.


We’re still talking about updating documentation right?


I'm explicitly referring to internal docs for the engineers that no one outside the team sees or cares (unless they want an example of docs). They're mostly markdown files or adding annotations to various code. I still think it's inane and seeing all the justification for "believe in the process" it's no wonder how little truly gets done in the name of agile/scrum/control.


I think we started there but have extended into a different discussion as to 'why is jira important to engineering teams'.


I think you made that leap, but I was responding to the specific example. I don’t think anybody is contesting the use of tickets in engineering, but I’ll certainly oppose the idea that Jira is important.


It's not a lot of value but it's also not a lot of effort. At minimum it gives a little bit of visibility into what you're doing and why to the rest of the team. That's probably worth the 2 minutes it takes to create a ticket.


I don't think it is. We're all professionals, if I'm doing something they need to know about then I'll inform them.

A ticket that takes 2 minutes to create is just distracting from tickets that provide real value to my colleagues. We're here to work together and improve something, not to babysit each other's schedules.

"Visibility into what I'm doing" as a reason to create a ticket is relevant for micromanagement, not for a team of professionals that trust each other's commitment to the job.


For updating internal documentation? I think we should be careful to ensure we’re actually adding value when we add friction, and not just hoping.


What is the scope of this documentation change? Is this a quick edit, or is this a whole new tutorial for new engineers getting their stack up? How long is this going to take?

Do you want an engineer every standup saying 'im working on documentation' without any accountability?


You really want to add friction to completing those tasks? How long does it really take to complete any one of those things, and how is a ticket going to change the fact it probably needs to get done anyway? Why am I going to write a ticket for docs I can write in an hour and be done with? Why would you want the docs to stay out of date until a ticket is created, rather than just fixing them?

Frankly I think you missed the point of the article. If you don’t trust your engineers to prioritize their own time when it comes to writing documents, you don’t trust them to do anything. You’re exactly a part of the problem that Will is talking about.


I dont really think you are grasping what I am saying. I want engineers to make tickets, and prioritize their time themselves. Creating a ticket to update docs is fine because it allows the engineer to set the scope, and define the audience.

That said, I dont want an engineer spending days on a task that will not bring value, nor do I want work done that will fall through the cracks, or have duplicate work done.

Asking an engineer to create a ticket to create documentation is not the hurdle you think it is.


I'm a successful engineer. I hate JIRA. I like keeping docs up to date, but if you're gonna make me deal with JIRA's BS just so I can do a quick doc PR, guess what, no updated docs for you.

Where was the value created again?


They want full visibility into what everyone is working on all the time. Some managers, especially non-line managers, really like this idea. I can understand why, it sounds really good! Transparency can't be bad, right? In practice, it works a lot better to create fairly small teams, let them run independently and opaquely to the outside, and then coordinate between them periodically (bi-weekly, monthly, quarterly, whatever is found to work best).


I think one key thing those folks miss is that willpower is a limited resource. For me personally, dealing with bureaucracy drains it much quicker than my actual work product, writing and designing code. Making me track everything is thus a strong net negative on my output.

I'm senior enough to know what I'm doing, so don't make me waste my (and my company's, they're paying me after all!) time.


At this point you’re just strawmaning.


The original scope that started this thread was "a quick edit". The claim is (or seems to be) that it is not too much friction to require a ticket for everything, which would necessarily include these small edits. That's stupid and will discourage small edits, which is bad, is what people are saying here.

> Do you want an engineer every standup saying 'im working on documentation' without any accountability?

Yes? It isn't any of my business, unless I'm that person's manager, in which case I'll certainly use my 1:1 with that engineer to understand what the documentation changes they're making are all about, and if it seems like a poor use of time then I'll give them a nudge. But if I'm anyone else in this situation, I'll certainly mind my own business.


I'm sure talking about having to update internal documentation has value. Can you perhaps elaborate on why 2 minutes to write a ticket is such a burden?


Let’s first open a ticket to have a discussion about the ticket opening process, and from there we can get a ticket open to open your proposed ticket.


Can you fill out this form in triplet?


Can you explain why it’s necessary I need to write a ticket to update documentation?


I think he did, and the answer is SOC compliance. I learned something about this compliance from his comment, and it'll encourage me to write these tickets next time (my current org is going through this process right now).


Which comment? The one in this thread doesn't seem to explain this. Does SOC require a Jira ticket for every change or does it require an audit trail? If the former, that's stupid, I'm happy to comply with stupid regulations, but I'm not going to gaslight myself into thinking they aren't stupid. If it's the latter, then that makes perfect sense as a regulation, but the commit history of the repo or whatever other versioning system you use for your documentation system can meet that requirement.


I did. To give your team visibility into what you're doing and why.


Why does my team need to know that I spent two minutes fixing typos in engineering documentation? Also, if they do want to know this, they can subscribe to the changes being made to the documentation repository / system and then they'll see what I did.

Is this really for visibility within my team? Or is it for some kind of external visibility?


>Is this really for visibility within my team? Or is it for some kind of external visibility?

Both.

>Why does my team need to know that I spent two minutes fixing typos in engineering documentation?

Nobody said typos.

>Also, if they do want to know this, they can subscribe to the changes being made to the documentation repository / system and then they'll see what I did.

How is that more efficient than you taking 30 seconds to write a ticket and 15 seconds to talk about it on stand up?

If it was important enough to go update the docs why isn't it worth the 1 minute to tell your teammates about it?


Nobody outside the team needs a single iota of visibility into my efforts to keep our internal documentation tidy. Anyone who is keeping tabs on this is wasting their time and needs to find something useful to do.

> Nobody said typos.

The claim in this thread is that a ticket is necessary for all changes, and fixing typos is a change.

> How is that more efficient than you taking 30 seconds to write a ticket and 15 seconds to talk about it on stand up?

I'm so confused. The "that" here is making the change itself vs. making the change itself and doing two other things. The way it's more efficient is that you do just the one thing, instead of that one thing and also two other things. Maybe I don't understand your question...


>Nobody outside the team needs a single iota of visibility into my efforts to keep our internal documentation tidy. Anyone who is keeping tabs on this is wasting their time and needs to find something useful to do.

This is naive. Someone above your team is superficially watching how much work your team does. If you don't document it then it doesn't get counted. You want it to get counted if you care about promotions and raises.

>The claim in this thread is that a ticket is necessary for all changes, and fixing typos is a change.

Not in this thread. The original complaint is opening a ticket for updating internal documentation.

>The "that" here is making the change itself vs. making the change itself and doing two other things.

No, the "that" is watching a repo instead of you taking a tiny amount of to tell the team you wrote some documentation.


> This is naive. Someone above your team is superficially watching how much work your team does.

It is not naive. Top down micromanagement is certainly common but it isn't inevitable. I have worked in multiple businesses with healthier management cultures. I'm sorry if you haven't.

> The original complaint is opening a ticket for updating internal documentation.

Yes, the rebuttal to which was "that's stupid don't do that" to which the claim was "it's normal to require tickets for every change", which is what I said the claim is. I wouldn't be pushing back on a claim like "nearly every change requires a ticket, but not minor edits to documentation like fixing typos". But that's not what people are advocating here.

> No, the "that" is watching a repo instead of you taking a tiny amount of to tell the team you wrote some documentation.

What thing are you saying is "telling the team"? Filing the ticket? So the team is subscribed to the feed of new tickets then? Why not just be subscribed to the feed of new changes to the repo? What do you see as the difference between those two things? Or is "telling the team" the standup status update about it? In that case, can't I tell them about the change I made directly? How does having a ticket help?


>It is not naive. Top down micromanagement is certainly common but it isn't inevitable. I have worked in multiple businesses with healthier management cultures. I'm sorry if you haven't.

Again with the unjustifiable leap to an extreme. Nobody said micromanage.

> So the team is subscribed to the feed of new tickets then

Yes. Scrum teams do a quick rundown of the board on standup. That's your opportunity to tell them what you did and why.


What you are describing is micromanagement. It is not necessary for you to say that's what it is, nobody ever thinks what they are doing or that what they favor is micromanagement, and yet that's what lots of people are doing. And that's what you're describing.

> Scrum teams do a quick rundown of the board on standup.

This is a major anti-pattern! What's funny is that other people I've been debating with in these threads have made the point that stand-ups aren't so bad because it isn't about status reporting, it's just a really quick get together ritual. And yet this is what lots of teams actually do, pull up the sprint backlog and have a planning meeting every single day.

What you're describing is a micromanaged process of very dubious value. I recognize that this is what lots of organizations do, but (to circle back to the article) it is very much the opposite of what companies like Google do, and not because they don't have tons of compliance and auditing requirements; they do.


And the PR doesn’t do that?


No, because not everyone reads PRs. And at this point you're complaining about the 30 seconds it takes to open a ticket and copy & paste. You still haven't provided a reason why that tiny effort is not worth the bit of value it provides. And frankly it just sounds kind of ridiculous at this point.


There are plenty of comments that do a much better job explaining why a context switch isn’t just 30 seconds in this example.

Besides if all I’m doing is taking 30 second to copy and paste something what value is that really adding? If nobody says anything the docs stay incorrect. But when I want to fix an error I need to context switch and open a ticket and copy and paste what I already wrote once? I’d rather just do nothing.

Really though, I’d rather just have up to date docs, so let me update them instead of insisting people do things your way. Fix the problem with people not being able to see the PR, don’t make me do the same thing twice.


>There are plenty of comments that do a much better job explaining why a context switch isn’t just 30 seconds in this example.

I'm a developer and 30s is how long it'd take me. There's no real context switch if you do it immediately after the PR.

>Besides if all I’m doing is taking 30 second to copy and paste something what value is that really adding?

Are you a GPT-3 bot? I've answered this three times.


Your answers have been absurd, which is why people keep trying to figure out what you're actually arguing.


I’m really grateful we don’t work together.


That's too bad. You seem like a treat to deal with.


Hahaha far fewer members of an engineering team are reading all the tickets than all the PRs. Who is this really for? Seriously?


Oof, if this really needs to be explained, boy I just really don't know...


What is normal about this? Engineers should be empowered with ownership. Adding. friction or barriers to enabling engineers to make what they own better is backwards and counter productive at best. At worst, it discourages proactive willingness to make an improvement.


If it's in a repo, it's normal to need a PR to change it. And, it's normal for a PR to need a ticket.

Alternatives to creating a new ticket could be:

1) Link to the ticket number that was used, for creating whatever you're documenting now. Use the "blame" report.

2) Link it to a catch-all ticket for documentation or code quality. Look in your ticket system for a low ticket number, with recent activity. If this doesn't already exist, and you really do need to make a new ticket, write it as generally as possible to enable this style of reuse.


> If it's in a repo, it's normal to need a PR to change it. And, it's normal for a PR to need a ticket.

In my experience, the former is true but the latter is extremely not.

Tickets are for tracking concerns for stakeholders. A documentation fix, to use the example at hand, is an ad-hoc improvement for customers/consumers and should have as little in the way of it as possible.


It is standard practice for technical writing teams to link a ticket to all pull requests. Writers, like developers, typically work on multiple pull requests in the same sprint.

If you're not familiar with Docs As Code, I highly recommend giving this a look for some of the current trends and ideas circulating among the technical writing community: https://www.writethedocs.org/guide/docs-as-code/

An example: would a developer publish code without knowing what ticket relates to that code? Nope. Same with current crop of technical writers.


> An example: would a developer publish code without knowing what ticket relates to that code?

Yes? I don't track a pretty significant amount of my work via tickets because I'm working on it largely solo and I have a direct line to all stakeholders to update them.

And I am not a technical writer, but I do technical writing, and I find breaking flow to juggle tickets a great way to get me to not do that documentation work because the opportunity cost of the context switch is high.


Those ad-hoc doc updates are the exception vs the norm.

It is also common to have controls in github, where its required for a jira ticket to be attached to a PR.

In that situation. Its literally impossible to merge into the codebase without a ticket.

Again it comes down to 'who can create the ticket' and 'what is the required scope of the ticket'


> Those ad-hoc doc updates are the exception vs the norm.

Where you work, maybe. Not where I do, or have. If I'm in something and see that I can make it better without losing my stack, I'll do it. I encourage my coworkers and, when managing, my reports (when they have proven that they won't rabbit-hole) to do the same.

What is often pejoratively called "boy-scouting" in a codebase is the only practical way by which many organizations will maintain that codebase beyond feature requests.


They are exceptional the harder it is to make them, and the result is documentation full of minor grammar mistakes or unclear things.

As an example for where it works great, I have made a small documentation fix PR to the Rust standard library 2 days ago and I didn't have to create a github issue for it or anything (but I had to notify a maintainer to review it because the PR fell through the cracks it seems). That maintainer told me they have reviewed 5 other similar pull requests that day. Over time, you get a really nice code base that way.


I think you just nailed it. With the PR that fell through the cracks.

The danger of having a ticketless fix is, you end up with PRs that are not on anyone's plate to fix.

The nice thing about tickets is that it creates a clear line of succession for adding value. If done properly, you will not have anything fall through the cracks. The cost for this is, you need to create tickets.

Reduce the friction to as close to 0 to create a ticket, and you can net all the benefits of tickets without a lot of cost.


The rustc repo has a system to assign PRs to reviewers automatically upon filing. On github, PRs and issues are very similar, and one can assign people to them.

There is a bot which checks which components the PR is modifying and based on that it assigns a reviewer to it. It predates github's builtin support for specifying component maintainers, so is a bit different than that.

However, as the Rust project is mostly made up of volunteers, sometimes a PR gets assigned to someone who is less active. This happened in my instance, where the PR got assigned to a reviewer whose last approval that ended up in a merge was in February 2022. The PRs assigned to these maintainers still get reviewed eventually, by other reviewers, or maybe them themselves. The assignment is more of a suggestion than a strict requirement and often maintainers with a desire to review some PR assign themselves. I just wanted to speed up the process so I talked to a maintainer who I knew likes to go through the list of open PRs and approve them if they are trivial. That's what I mean by "fell through the cracks".


Perhaps you've worked in notably functional organizations, but, with respect, your claims throughout this thread read like theorycrafting. I have been a Jira administrator, I have had extremely positive feedback from all parties based on the workflows, the automation opportunities, and the minimization of overhead involved when I did so, and it still required a context switch. And context switches always and without exception suck.


Ive both worked in and led highly functional, and highly non-functional organizations.

You are exactly right, that much of what Im pushing is the vision of what Agile / Scrum is supposed to facilitate, and what the benefits can look like if done correctly.


Why are PRs more likely to fall through the cracks than tickets? I think the opposite is the case...


The best jobs I've had didn't have any of these control issues.

The worst ones required tickets linked to PRs with multiple layers of approvals, on and on. It could take days to get a few dozen lines of code merged. No thanks.


For me worst system that I had to fix and I had to save the company from losing only paying customer didn't have any of controls.

It was just bunch of freelancers doing what they wanted and paid in gold.

Right now as company is growing I consider leaving from time to time because people we now hired start creating bureaucracy that I would never go into.

Well I definitely introduced process into company - but as I see it there must be some absolute baseline tickets/repo/CI/CD for standards and at least weekly meeting.

I also agree that for a lot of software requiring unit testing (a lot of people writing bs tests just to pass review), requiring 2 or 3 people to sign off PR is just something I would not recommend.


Some process is necessary, for sure. However, if they only had one paying customer, sounds like they had bigger problems than no controls.


> It is also common to have controls in github, where its required for a jira ticket to be attached to a PR.

Maybe that's common, but it does not strike me as a good practice.


That's assuming a lot about the work environment. If your have a team full of professionals, trusted by stakeholders then you don't need any of that.

If my teammember decides documentation needs updating he should go ahead and update it. Tickets and PRs are for thing that need planning and reviewing. I trust my colleagues to know when that is the case and decide accordingly. If any of us messes up we talk about it and improve.

In reality this means that many, if not most, things still get tickets / PRs because it helps but the fact that it's not simple ceremony makes sure everyone knows their importance and saves time when they aren't needed.


> And, it's normal for a PR to need a ticket.

Why?


Some orgs have strict change control processes. "Why is this change happening? What is it supposed to do?" At one place, I used to just submit my own ticket,a assign it to myself, then open a PR linked to it. Totally pointless. At some other companies, it's more open. "We trust you! Just get it done." There is more risk, but it's a better environment for some. Possibly not for all.


I understand requiring such documentation and auditing history, but there is no good reason (or at least nobody has presented one here yet) to do it in both a ticket and a PR.


Have you worked in a regulated environment? It can be a regulatory and legal requirement to do this, since it's part of the software development lifecycle processes which are required to be followed.

At its core, this is about ensuring that the software is not changed without a documented requirement to do so in the form of a formal change request. The PR alone does not do that. The PR is the software change. It is not the request to do the change in the first place.

Some projects and company processes can be sufficiently lax to permit developers to open PRs and change things without any additional accountability or oversight. In these cases, you don't need that extra level of work. But many projects do need this, and it can provide value even outside a regulated environment. Many open source projects require a corresponding ticket for every change as well.

You might also need to make sure that the work meets the "definition of done". In a stricter environment, that can mean making sure that every change ties back to the design documentation and the system requirements, as well as having all of the necessary test and documentation coverage to match, and more. That gets tracked on the ticket, not the PR, because the PR is only about the software change, not the higher-level stuff.

Another factor is that the ticket is what has the visibility. Not everyone in the company is necessarily directly involved with the nuts and bolts of the version control system. When it comes to the higher-level aspects of project management, particularly on large projects with multiple teams, you aren't going to be doing that with GitHub, but you do need to coordinate activity and account for where the development effort is going for planning and resourcing purposes, even if the metrics are only used within the team.

None of this means being hidebound by process. For a trivial documentation change or bugfix, there's nothing to prevent keeping this overhead to the bare minimum. It takes 30 seconds to create a new ticket before opening the pull request, with a title and possibly a sentence or two description. That can greatly help the other people in the company see what is being worked on, what's been fixed, what's in the backlog and where effort is being spent. Ultimately, we aren't writing software just for our own amusement, we're doing it for others, and the tickets are part of communicating and coordinating our activities with other people, even if it appears superficially onerous.


Can you give me example regulatory language that would require a Jira ticket in order to change internal documentation? (That is the only thing we are discussing in this thread, despite many attempts to broaden it.)

An internal engineering documentation change is not something that needs visibility in this grand project management effort. Project managers that are tracking such things are doing busywork that is not adding any value.

I recognize that all of this is very common, but it's not actually valuable. I have absolutely zero problem abiding by rules that are not good, but what I see a lot of here is learned helplessness and rationalization. It is not necessary to convince oneself "actually this is all good" in order to merely abide by rules that it is not your responsibility to change. You can merrily follow the rules and then also, if anybody asks you, you can say "these processes are not valuable for the following reasons".


The documentation itself can be regulated, and changing it requires more than a PR code review by another software developer. As an example, the instructions for building and releasing the software could come under the controlled SDLC process.

I see where you're coming from, but you're not correct that this is misguided, nor that this is just busywork for project managers. If you're doing safety-critical stuff in the automotive, aviation or medical industries then process faults mean lives could be put at risk. A drive-by PR to update a document could have serious implications, and that's why change management processes are in place. They are a safety-net above the level of the code review and activity in the VCS, because you have a different set of eyes on what's going on.


I know there is some fallacy or something around asking for evidence (is it "sealioning"?), but are you aware of anything that demonstrates that this kind of change control process successfully reduces risk in life-critical systems? I'm very skeptical that requiring a ticket requesting an internal documentation change in order to improve that documentation is saving any lives. You might say "well it can't hurt", but I don't think that's obviously true; I think it creates a perverse incentive to leave documentation as is rather than improving it, which I think is more likely to be negative sum than it is to be valuable. That is why I think it is misguided; I think it is more likely to result in poorer documentation, for no gain.

Please understand that I really am just talking about this one case; I see no reason to question a requirement like this for changes to code or configuration or interface documentation or specifications or anything else.


An example is when all hours spent need to be traced back to a requirement that has been signed off on by stakeholders. There needs to be a recordkeeping system to support this requirements traceability, so Jira it is.


Is that a regulation? It sounds like either an internal process choice or a contract requirement.

Edit: But point taken. I can see where this would be a (misguided, counterproductive) requirement in a probably-government contract. I would just suggest recognizing that it is a necessary evil when doing that kind of work, but not a good practice.


At a previous org, I discovered most of the product management folks did not have access to github. So if someone (not an engineer) was trying to figure out why Joe changed feature X, they couldn't do it with a PR. They only had access to Jira.


Sounds like a good thing to fix.

Edit to add: But also it seems like there are two things going on in this thread. On the one hand, the reason for tickets is (supposedly) for a compliance audit log. And surely the auditing system could read from both Jira and GitHub. But this question of what product managers can access is a different one. I think the more likely reason for these rules is for "product managers" who are really acting as project managers and are trying to micromanage through tickets.


What is normal about this that Ive seen is work and tickets is often written into controls that are required for things like SOC compliance. And documentation often requires a pull request.

And again read what I said, that engineers can and could be empowered to make these tickets, which can be minimal at best.

Done right, its not a limitation at all, and instead adds accountability.


"...things like SOC compliance." This is an important point. If you work in a regulated industry or work with software used in a regulated industry you need a "paper trail". Having to document and follow a process is often frustrating to people who work or have worked in industries that don't require it. That said, a poorly implemented process in a regulated industry is even worse. As you're overloading teams with busy work and likely still failing to meet the requirements of the regulations you're following. Regulated software is not a place to "move fast and break things" that's what prototypes that never see the light of production are for. Build it without regulation to figure out how to do it and then build it again following the documentation process. It may sound odd but having a second chance to correct errors of assumption from the first time around is quite valuable, buying information when done right is a great tool.


SOC compliance is indeed a beast. I have my own theories on how valuable this benchmarks REALLY is, but if teams are in a financial org, it is super common to have controls that have the potential to really hamstring teams.

Which is doubly important to really understand the vision of agile / scrum, and not just follow the ceremonies by the book.


Could you point me to where in the regulations it says you must have a duplicate paper trail? A PR is already a paper trail...


A PR is most emphatically *not* a paper trail.

It's recording a software change, who made it, and when.

That isn't even close to the bare minimum for some of the regulatory requirements.

You're coming to this with the perspective of a typical software developer. I can sympathise greatly. But the regulations aren't about software development. They are about risk management and product quality, and putting in place processes to ensure projects deliver solutions which are safe and robust, due to having the appropriate controls in place. This isn't just about the software implementation. It goes all the way up to the specifications, designs and requirements and it requires testing and validation work at every level.

I have only in the last few years been working in such an environment. It was eye-opening. Compared with the free-for-all which is most software development, this is actually about engineering robust systems, in a very systematic and rigorous way. That does include some overhead in the processes surrounding the development work, but it's there for a very good reason.


I'm very familiar with this kind of compliance environment. I spent quite some time working on a committee focused on similar issues (privacy requirements in my case) and I certainly understand the value of auditability and the existence of requirements like this.

We are talking in this thread about a very specific subset of this: Does it make sense to require a Jira ticket for a change to internal system documentation.

I don't have experience with SOC in particular, but I would be very surprised if the regulations specified "you need a Jira ticket for every change to every kind of documentation". That would certainly be a very big lobbying win for Jira! But I suspect it is more like "you need to be able to audit and trace the purpose of all documentation changes". The trick in designing the processes used to comply with such requirements is figuring out how to balance compliance with process friction. If you are in charge of this sort of thing and your instinct is to never challenge the usefulness of busywork and ignore feedback on its impact, like the kind of feedback in this thread that is being ignored, then you are not doing your job well.


> Done right, its not a limitation at all

Context switches are limiters, IME. Smash somebody's stack and you've made them useless for half an hour.


This is why tickets can be used to protect the engineering team. You dont want people ad-hoc adding scope, or interrupting work for engineers.

You want engineers to have focus time, on a clear actionable item, that is set up for success, and will add value.

Allowing engineers to work on whatever they want, or change scope on what they want, without saying 'why', can end up creating a lot of wasted work.


Tickets can, theoretically, be used to protect a development team. I agree with you there. But they usually aren't, and most folks are living in the land of what-is, not what-can-be.


Yes I agree and thats why Scrum / Agile gets a bad rap around here.


Even not SOC compliance just liability of a company.

If stuff fails and company gets sued - they will have to prove that they "follow industry standards" because if they don't follow industry standards then all kinds of bad stuff can follow.

IF you get ransom-ware and have an insurance - see the exclusions - you have to have EDR software on each server otherwise insurance won't pay.

Not following industry standards in terms of code - the same no insurance company will pay if they catch you on just doing stuff.

So running a business all kind of BS and it is not optional and just slinging out code is not enough.


You find it normal that you need a Jira ticket to update documentation?

If while reading a document, you find a mistake, instead of changing it, the normal reflex is to create a Jira ticket?


I find this whole back-and-forth fascinating, because it mirrors my experiences in the workplace. There are some people who really like process around everything an engineer does, and some people who really don't like it. They're almost two distinct personality types, and it's virtually impossible to get them to agree because it's like asking someone to change their personality.

I happen to be one of the types who dislikes heavy process-oriented project management, and reading mr-ron's responses is admittedly making my blood boil. But I bet he reads some of the responses to his remarks and probably has similar reactions.

Some people thrive with heavy process, and some people wither. Some people thrive with light-weight process, and some people wither. I don't know how to structure an organization to support both types of people in it, but it's not an easy problem to solve. That's why there are so many project management methodologies, with new ones popping up every few years and then inevitably disappearing in favor of something new and better.


I think the solution is to do everything you can to keep the heavy process people away from anything important for as long as possible, before you get too big and they manage to embed themselves anyway, where they will commence to expand the bureaucracy to meet the needs of the expanding bureaucracy.


Reading this thread reminded about "Bullshit Jobs (https://en.wikipedia.org/wiki/Bullshit_Jobs)


What a great comment! The whole, every change needs a ticket debate is interesting but without context. Some orgs require it and you can just create a ticket and move on. Fundamentally I don't view these tickets as useful, if asked "tell me the latest state of X", or "why was Y done", the tickets rarely help (and if they do, the proper places of commit history or the code itself skimp on these)

The tickets can often go on to be "for the devs we like and trust, they can cut tickets", but for the others we require they're prioritized. This creates a lot of chafing, that quick bug fix, is it worth context switching now to create a ticket, again later to explain it to the non-technical prioritizer of tickets, worth the argument of why a "business goal" should be deferred to prioritize this ticket, the tracking overhead to finally say you are selecting ticket X, and that final context switch to actually work on it? All that compared to just fixing it, switching branch and cutting a pull request a minute later

I found I liked tickets to identify product work, so maybe 5% of my commits actually reference tickets, the rest are supporting changes that enabled that 5% to be done cleanly or are just things that should be fixed or cleaned. Coming back to it, tickets have their uses: documentation, discussion, prioritizing. But tickets can quickly turn into "this is how we keep our engineers from cleaning up the code, etc", "this is how we do programming by remote and control exactly what is worked on." Worst, it is the self inflicted burning the midnight oil, "hey, you said this was going to take a week, estimation is a skill of a senior engineer - will this be done as promised, or is your ability to estimate not there yet? Wink, the sprint is over on Monday.."


Im not sure why you are describing my example of PRs needing tickets to be 'heavy process'.

Especially since Im pushing for engineers being able to create quick ad-hoc tickets within 30 seconds.


FWIW, I agree with timmytokyo's assessment of 'heavy process'. I actually worked in a place where PR's need to be tied to tickets and updating documentation is the perfect example of where it sucks. Imagine you're trying to debug a bug, looking through the stack trace and trying to reverse engineer what the heck is going on. You find a block comment explaining what should be happening, but it's not been updated in years and not technically relevant to the final bug fix.

You've got two choices, either you sneak the comment block change into your PR, despite it being irrelevant and make the documentation update blocked by the bugfix/feature. Or you need to open Jira, make a ticket etc etc. I know you say it's 30 seconds, but it's not. It's at least two minutes, but it's not even the time that makes it annoying. It's the fact that you're doing something that you(the people timmytokyo describes) think is entirely pointless.

I ended up writing a janky git hook that would check if the branch name I was pushing had a jira-id in it, and if it didn't it would use the Jira API to make the ticket, grab the id and push the branch again with the id in the branch name. The fact that the entire thing can be automated, makes the entire thing pointless in my mind.

The thread gives me very strong vim vs emacs vibes :).


>The fact that the entire thing can be automated, makes the entire thing pointless in my mind.

Hold on, so we should stop running tests in CI too?


So the main difference is that CI tests provide useful information, whereas automatically creating a ticket from a commit provides 0 useful information, it just duplicates data.

The other difference is that CI tests do actually run automatically. If you had to go into github and press "run tests" every time you make a PR, that is a waste of time.

But sure, if there was a server side hook that automatically created Jira tickets for PR's without tickets, then the entire thing is less annoying. I still think that's a net negative since it creates noise in Jira, but it's not as bad as forcing engineers to do automatable work.

I agree that my wording was not perfect, hopefully the extra explanation makes my point clearer.


... if they aren't automated, then yes.


It feels heavy to me, because you're asking for a process gate to be put in front of something that is so trivial. It feels utterly unnecessary and demotivating. If I see a minor problem in the documentation and decide to correct it, now I have to go through an extra step of creating a JIRA ticket describing the minor problem I'm trying to solve, doing the correction, updating the JIRA ticket status, and possibly monitoring the ticket for future issues. It's. all. so. bureaucratic. And sadly it will probably lead me to thinking the fix is not worth my time.

Instead of trying to convince everyone that they should feel the way you do, maybe try to understand why others feel the way they do.


I think the answer to all of this is 'it depends'.

Is there already a ticket and are you working to update the app? Then update the documentation. No ticket necessary.

Are you redoing the whole set of docs to make the steps for new engineers to create their stack? Then please lets have a ticket to define the scope and audience.

Trust me that I do understand that there are many orgs that have arbitrary hurdles in place, like who can create / prioritize a ticket. My experience is, if engineers are empowered / encouraged to create and prioritize their own tickets, then its not a hurdle at all.


> if engineers are empowered / encouraged to create and prioritize their own tickets, then its not a hurdle at all

Many people - now including me - are plainly telling you that it is a hurdle for them. No amount of reframing this is going to alter someone else's experience.


You seem to keep trying to change the subject to "should there be tickets in general?", which nobody is saying there shouldn't. The topic is "should there be a ticket required for every change, which requires tickets for updating typos in internal documentation, because that requires a change and you propose that a ticket be required for every change. That is stupid, to require a ticket for that. These other examples are not stupid to use tickets to track them. But they aren't the subject of the thread.


Skip the ticket completely and get out of the way


Thank God we can choose where to work or I’d have to deal with psychos who think everything needs to be a ticket.

I can only imagine what software these types of organizations produce too. Probably a million little steps to do anything in the UI.


Nah, I can just make a task and make whatever UI tweaks I want. I mean, someone will probably complain about it eventually if I make some crazy change to the UI, but there's no process stopping me from doing so. Other than the handful of clicks it takes to make myself a task for the work.


It’s not really 30 seconds though is it, if everyone is making all these tickets to do trivial things, they are getting update emails on them whenever their micromanager gets around to acknowledging them. It’s generating all the work for people to look at all these trivial tickets. You already have a source of truth on what changed - the PR. Forcing a fellow human to make a ticket that is just a link to a PR is Kafkaesque regardless of whether you “empower” your engineer to open and then immediately close her pointless tickets on her own or not.


In my experience, there's a threshold where a ticket just generates net noise on the process, which is why I don't like tickets even for things like updating documentation in code.

If you multiple that "just 30 seconds" by several trivial things you encounter each day that aren't part of your assigned lot, it adds up. Those new tickets show up in your list of tickets, so now you have several line items that have to be walked through during sprint planning or daily standup so everyone knows what's going on.

It's true that perhaps other devs want to know about that ticket to update some documentation, but it may be such a minor change that it seems silly to have to do a song and dance about it during standup so everyone has full context. And often, not everyone needs to know about every little change that is made anyway.

Then, when you look at the work to be done, the large list of trivial tasks makes it hard to see the 1 or 2 really important tasks that you're really working on.

I think the dividing line here between pro-ticket vs. not is based on how much you trust your fellow devs to the right thing and how much trust you have overall in your team as an organization. If you have little trust that people will do the right thing, you'll probably want to create a ticket for everything to make sure it doesn't go astray. Unfortunately, for those with more experience and better judgment about what is the right thing to do, they'll just feel constrained by the whole thing as it removes agency.


... they are getting update emails on them whenever their micromanager gets around to acknowledging them ...

You just nailed the problem here. If the process is set up that an engineer cannot create / prioritize / close trivial tickets themselves, that is indeed true friction.


Those problems never come in isolation. If the developers need to do useless tasks for being through, the odds are really large that they also need to answer to them for management.


A ticket supposedly represents some worthwhile task that's worth specifying. If it takes just 30 seconds to create it can't possibly concern a task that's worth the cost of involving ceremony/process.


Because it is heavy process.


> If while reading a document, you find a mistake, instead of changing it, the normal reflex is to create a Jira ticket?

I would say the first reaction is to create the PR to make the fix. Then, while CI is running, I create the JIRA ticket and update the PR description to point. Best of both worlds!


Busywork is not a virtue.


It depends on the documentation. Is it within a repo? Does it require a pull request? Is it a quick edit, or is this a multi hour effort that is redoing the scope?


Cute Agilist gaslighting about not needing a ticket because that’s “asking for permission” and it’s supposed to be implicitly bundled into feature work.


”Tickets should be minimal depending on the task. Creating a ticket that says 'update documentation' may take 10 seconds”

This type of placeholder-documentation is literally the WORST type of tracking you can have. And a sign that everybody in your company forgot the reason why this type of traceability was implemented in the first place.

If I review a PR that links to a Jira ticket that says absolutely nothing except “update documentation”. What’s the whole point?!

What’s worse is when people see a link to a Jira as an excuse to write bad commit messages.

I would much rather read a commit message detailing out the background and rationale of the change, rather than a nonsense commit message linking to another nonsense kpi-chasing Jira ticket.

PRs are code reviewed anyway so it’s not like you have a bonanza of unknown changes being pushed left and right. Code reviews not only catches coding bugs but also accountability within the team, eg “why are you refactoring something that works, when our backlog has 100 other more important things to do.”


Is it? I've never needed a ticket to make off the cuff improvements. A PR and a review from other engs to make sure they agree, certainly, but a ticket for that seems like a certain degree of dev micromanagement/big brother/power trip for highly paid and skilled professionals.


At my previous gig we reduced the friction for doc contributions to just a PR but with no review requirement; if the contributor felt the need for feedback or review they would seek it, otherwise just merge the changes.


That's a stupid compliance requirement, unless documentation changes aren't themselves versioned (which they should be). You don't need to track the exact same thing in two separate places.


If it's so important to have a ticket, the patch submission pipeline should create one automatically.


In my past job, I volunteered and pushed to get the whole project, then I asked to be exempted from all scrum stuff, then got really close to PM and Designer, launched the whole thing without a single bug in 3 months and with more features than planned, and had 2 more months left.

The reason there were no bugs was because any question I had, I slacked messaged PM, and got replied in less than a minute. She was on the project full time too, and was technical so was able to test out things and apis, document correct relevant data like console log errors, browser version, json response, curl ... etc.

My takeaway was similar to yours but in more concrete terms:

- have experts available in team which can be consulted asap - give whole projects - start with minimal project - create small informal groups, POC for each area: sr eng for legacy code, designer to get artifacts, PM to clarify tickets and address new bugs, main dev to do the whole thing - have team informally occasionally input on tech decisions.


> I t’s nice to see the simple truth called out.

It’s not as though it’s a secret or anything. It’s a common point of discussion. The simple matter is that big co’s naturally have more inputs into the same types of decisions and thus need more people involved.


I don't think it's about largeness, and the autonomy of "rockstars" is of dubious value. Google is the largest company where I've worked, and also the most productive. For code that an IC owns, they only need the sign-off of literally anyone else in the company to land changes, and all that is expected is those changes are generally consistent with team goals. For code an IC does not own, they only need the approval of one owner. Pretty straight-forward.

A medium-sized company where I worked had a "rockstar" who was really just an early employee with a high title and no idea what he was doing, so every week he committed a thousand lines of brand-new unreviewed tech debt. Much of the rest of the company was unwittingly dedicated to overcoming his tech debt. Said company currently trades at about 1/3rd below its IPO valuation.

The experience of these two companies led me to the belief that hiring practices and review culture are more important than project management culture. You can get a lot done with good talent, code and design review, and vaguely disorganized project leadership, and you can run in place achieving nothing with formal project management hamstrung by bozo engineers.


> and the autonomy of "rockstars" is of dubious value.

I agree, but with your example, I wouldn’t call somebody committing massive amounts of tech debt a “rock star”. I used the term fairly facetiously anyway, because I think we all know the 10x engineer is a myth.


“Jim is so productive!”

-by design Jim is the only one allowed to be productive-

:P


Yeah this is exactly it. But I think it's a good thing. It's a tradeoff between productivity and risk. It is productive but risky for "rockstars" to make big changes with very little consensus-seeking. Big changes are double edged, they have outsize impact, but it may be either positive or negative. So organizations rationally hedge that. The trick is to strike a good balance, giving up as little productivity as possible for the greatest reduction in risk. It's hard! Sometimes the right answer is just "empower one or a few people to do whatever they think best without ever seeking any buy in from anyone else", but not often. And "never let anyone do anything without a big slow sprawling committee giving permission" is essentially never the right answer. But there is lots of space in between these extremes!


Such is the nature of organisations. E.g., X is the boss of a group, the group becomes a department, X hires people to manage the groups in the department, and now all these people have to be informed as well, and of course they have an opinion, too. And they are rarely good managers. Then X leaves and politics kick in because everyone covets the position, and then someone's simple task becomes the focus of someone who wants to place your manager in a bad light. And it never stops, because there's no incentive to make it stop.


Your 10x rockstar is my cowboy coder 0.1xing the team.


The use of the term was supposed to be illustrative and mostly facetious, hence the quotes.


There's huge amounts of anti-process arguments in the threads here that is frankly immature.

Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.

It'd be great if we could all just do what we want, but in my experience, process is crucial when aligning great people's efforts and energy with the business paying them.

I'm not saying scrum is the way, but calls for "just let engineers decide and get on with what they like" sounds like 20-somethings with little experience of when and how often that goes horribly wrong.

Suitable, proper process keeps chaos at bay and ensures everyone's expectations are aligned. That keeps business running and jobs and growth happening.

Funny thing is, 15 years ago, I'd have been the first to agree that processes are "infantile handholding" (to quote one poster here).

15 years later and much wiser, I'd never hire anyone with that attitude.


I'm also 15 years in and have worked at all different kinds of companies with a few different kinds of processes, and I don't really relate to this.

To your point: I don't think any senior engineer argues that it is tenable to have absolutely no process. There needs to be a way for business stakeholders to communicate with engineers and vice versa and for everyone to plan and contribute to prioritization together.

But many senior engineers debate the time scale on which and the audience with which that coordination and planning and prioritization should be happening. The threads here where I've seen the most pushback are where people have been advocating for processes that are designed to give very broad audiences very nearly real time insight into what engineers are doing with their time. And I think that's pretty bad and I think lots of experienced people agree with that, not just whiny youngsters.

What I support is small teams where intra-team visibility is not really an issue, and cross-team coordination on longer time scales (bi-weekly, monthly, quarterly, half-ly, I've seen all of these work fine, with different trade-offs).


I don't disagree with anything you've said. My core point was that process, of whatever form, is right and necessary.

I've seen things go wrong due to no process and I've seen things go wrong due to inappropriate process, and any process applied poorly.

The process isn't the evil, it is poor leadership, regardless of the process that kills projects.

Different projects in different industries, with different software IC contributor types calls for different leadership ... and I've used Scrum and others successfully as tools in that leadership.

Blaming tools (like scrum) or any tool blaming I feel is immature and scapegoating.

Granted my original post was a touch inflammatory, partly out of frustration.

I made my living repairing under performing teams and good processes top to bottom are the number one thing I've found that stabilises things and turns it around.


The opposite of scrum isn't "just let engineers decide and get on with what they like".

In good tech companies, product managers + design + eng collaborate on how to meet a business need with a delightful product. The leaders of those orgs keep everyone aligned around a big goal and high-level milestones, but distribute as much of the lower-level decision-making and planning as possible.

That's mostly orthogonal to the processes for how a project is actually run (scrum vs informal vs plan-build-ship) and how the process style is decided (centrally controlled vs suggested vs team-by-team).


I agree.

My argument was against the immature views expressed elsewhere.

I think the right balance is gained by hiring the right people and empowering them.

If I can't "hands off" projects without interference beyond setting business goals ... I've hired wrong.


We're in violent agreement then :)


> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.

The way FAANG fixes that is to make "impact" the first metric of their engineer evaluation process. Basically, you're quite autonomous to pursue your ideas, but need to be able to justify that you contributed the project in meaningful ways.


>The way FAANG fixes that is to make "impact" the first metric of their engineer evaluation process. Basically, you're quite autonomous to pursue your ideas, but need to be able to justify that you contributed the project in meaningful ways.

Isn't the joke that Google as a result makes and kills tons of half baked products without producing any actual business value overall? The product lives just long enough to get people's promotions approved.


Sounds like their heuristic for impact is wrong then.


There is no viable heuristic because everyone has an incentive to game the system. Even managers. "When a measure becomes a target, it ceases to be a good measure"

The closest one is short term improvements tied to revenue and measured using ab testing. But even those get gamed.


That's interesting when there's wiggle room for that.

I've led projects where there isn't (in defence). Specifications are prescribed exactly and in detail, deadlines are fixed and the ICs job is to implement and that's all.

It was a sad state of affairs, but one caused by software being a small cog in a massive machine that isn't software driven.

Outside FAANG and software centric businesses, the landscape of responsibility and the availability of autonomy can look very different IMHO

Less like craftsmen making fine furniture and more like an assembly line for IKEA furniture. Processes like Scrum form that assembly line which ... like it or not as an IC ... is appropriate in some businesses and industries.


> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.

At my last job, there was a strong bottom-up engineering culture where ICs were empowered to work on what they thought would deliver the most business impact. This was actually quite effective because there were performance incentives to ensure individuals and the organization had aligned goals.


In an ideal world this is the way.

In cases where I've not had hiring decision making and the manager got it wrong (more often than not) this wasn't possible.

Sad reality.


How many employees?


During my tenure, they scaled from around 27,000 to 77,000.


> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.

Is this actually true? I know it’s a common fear of managers who are in over their heads in one way or another, but isn’t it a bit of an infantilising view? I would never want to work at a company where this was how the leadership saw engineering (and I’m a product manager, not an engineer).


I think it is a fair view that a manager is in over their heads if they can't stop this happening. Sadly ultimate autonomy of many ICs without proper leadership results in this and good processes prevent that.

It is the sign of a team being lost, due to bad leadership, and good management prevents a team being lost.

A good way to prevent a team being lost is, in my experience, processes that keep everyone integrated and moving in the right direction ... scrum being one approach (not always the right one).

At the end of the day leadership is needed and a framework that lets everyone know where they stand without it all turning into design-by-committee nightmare.


I tend to think you’d have more success having no manager at all vs the bad manager who thinks his team is going to go off and start making a video game or something more fun instead of working on the business if they don’t crack the whip. It’s a kind of paranoid fantasy coming from insecurity, it’s not actual reality.


I think it's a fantasy to expect that all teams are capable of self direction like that.

I've had teams that were, who embodied the idea of building teams smarter than ones self.

I've had team where given more freedom, they just went in chaotic directions. They just intrinsically were a composition of people who couldn't care less and wanted to work on whatever they found interesting. Usually because they didn't have any stake in the business, interest in what it wanted or respect for its goals. And yeah, I didn't blame them, but I still had to move them in a unified direction writing boring software for a boring industry with no interesting challenges.

It's really more complex than black/white.


There’s so many normal people out there who are basically motivated to do a good job, even in boring industries.. why would anyone go out of their way to build out a team of uninterested people who don’t want to be there? Seems like we’ve just found another path back to bad management.


I agree, thinking back to one project I led though, finding engineers who would have actual interest in the domain we operated in was tough. It was dry, tedious code in a legacy language (for sound business reasons!)

It's sometimes tough. I managed to find compromise by working with ICs to make it interesting by engineering our way out of the boring through seriously cool automation.

I can see how, in hindsight, that was a lucky move and I don't blame other managers in similar situations for hustling teams along with more process. It's the right call.


I would disagree - I think you are reading something into “let engineers decide” that goes with the many micromanaging structures like scrum, which is to start with a close to complete lack of trust in ICs.

In a well managed organisation ICs don’t simply go off and do what they want. In a new organisation with a lack of any experienced engineers a failure in management could easily lead to that as management has failed to do its job and the ICs are not sufficiently experienced or informed to be able to compensate for such a failure.

More experienced engineers can generally identify poor or incompetent management and can compensate (to an extent). But by and large “all the engineers went off and did random things” is absolutely a failure in management, not the ICs.

If there are just a few ICs off in the weeds then that implies the ICs are the “problem”, and that may simply mean having some more hands on management - helping them understand prioritization, etc not micromanaging them - or possibly having them get some help from a more experienced IC, so that they can learn how another IC does their job. At no point have I said anything about peer programming, which should not ever just be a “this is how all dev shall happen” nonsense.

Things like peer programming is a fairly occasional tool in my experience, something that has happened when I’ve been working on a particularly irksome problem with another person on a whiteboard, and it seems like the code might be gnarly, but that latter is very very rare; vs the simply working out the solution.


I agree with what you're saying. What this might discount though is where an organisation can't afford high enough calibre of engineers to make these things possible. That's not always the case.

I've been (and am) a software engineer but also operate in higher level management of dev teams.

In orgs that have an technical product or are software businesses, what you say is true. That culture doesn't exist in all sectors that require software engineers.

I believe that for most organisations strong processes from PRs and code review all the way through time-boxed goals to high level stakeholder meetings its important they're structural and focus on accountability and goal achieving expectation.

I also strongly believe the right ICs should be one of the driving stakeholders of business objectives when hired well.


The above comment comes across as frightened and reductive.

Here’s my own reductive take: If your engineers aren’t being productive if given freedom then fire them


Yes it was reductive, but not frightened. I think building the right team is the antidote to over management.

Sadly I've seen too often teams are not built with this in mind, and stringent process can work to keep efforts in line with requirements. It almost allows broader hiring.

I'm not saying there's a right or wrong, just that arguments dismissing lots of process here don't work for me for their reasons.


My reductive one is “if your engineers aren’t productive when given freedom” consider firing their management.


I'd argue yes, if management allows the situation to persist.

Bottom line is if the eng team don't meet goals the manager is responsible.

For me, good process, not micromanagement allows the eng team to perform to expectation and know the boundaries they operate in. Scrum is a tool in that toolbox.


My interpretation is that if this is an issue for any meaningful proportion of ICs, it implies management is the source of the problem - if an IC does not have a good understanding of targets and priorities, then management has failed to adequately communicate.

If it’s only a small proportion of ICs, then you can look at the ICs being responsible. If it is a new/inexperienced IC I might still lay blame on management for failing to provide adequate support while an IC learns how to do their job. School and/or OSS contributions are good* at teach the mechanics of a job (how do you write code, how do you design a circuit, how do you do design, etc), but atrocious at teaching how to work in an actual work environment where you have to work with other people on your team, other teams, other people on other teams, etc, and how you do things like prioritizing goals.

Individual projects at school obviously do nothing for team interaction, group projects are IME atrocious at reflecting a real environment on far too many levels to express without writing a book (and so learning how to write readable long form prose :) )


100%. A lot of problems at organisations come managers having far too much job security. This gets really acute the larger an organisation gets, where the goals of managers can become completely divorced from that of the company. For example a managers goals might be to increase their budget and team size in order to justify higher pay and promotions due to experience “delivering” “larger” projects. Whereas more team members (especially those easy to hire project managers or engineering padding like architects) end up decreasing productivity. A really good example in the article was Skype, which went Scrum with dozens of staff (if not hundreds) and was beaten by a team of three engineers with no “formal” process at WhatsApp.

Scrum needs to seen for what it is: a way for management consultants to bill lots of hours and a way for organisations to manage projects where engineers have no agency. In a consultancy, for example, where scrum is almost always used, the real money comes from all of the management consulting fees and design iterations and etc etc, and a build is only done when all of that has been signed off and paid for by the client. Then the engineering team is told to go build it. This is VERY similar to the waterfall methodology that Scrum was supposed to replace. And seen in that context, it’s no wonder that successful tech companies have zero time for it.


>Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.

I mean e.g. Facebook does real tech stuff, like TAO.

Sounds the goal of Scrum is herding to write your CRUD app.


Yeah that's fair but there are a hell of a lot of CRUD apps that need writing by middling software engineers who'd rather do some fun refactoring than write yet another boring API endpoint.

I've managed enough of those teams and yes, boredom is a real problem for both motivation and staff retention.

If I let people on those teams do what they wished, the app would've been rewritten 10 times by now in different cool new frameworks and the business wouldn't benefit because the core app was so dull.

But there's a lot of boring software that needs writing that makes money.

It is true that those teams suffered churn of people looking for less boring work ... at the end of the day the right processes (like scrum) and other things that appear like heavy management are just risk mitigation for the fact engineers don't want to work on the core problem

I've spent many years working out how to deal with these situations without degrading engineers with heavy handed management.


This might come across a but harsh but I don't know how else to put it without resorting to my native language but I feel kind of sad for you. You sound like someone's who's lost hope in (at least part of) humanity.

Your anecdote suggests this is a truth about all engineers and while I cannot disprove that I might as well counter with my personal anecdote.

In my experience, engineers tend to be very sensitive to the environment they work in. When unsatisfied with the workplace environment, they might go off on side quests but trying to "fix" that with tightly controlled process is fighting symptoms, not causes and will generally result in an even more unsatisfied engineer who appears more valuable because they've learned how to hide the unproductiveness. I've seen this happen and it definitely didn't add any value for the business.

The real solution is to try and create an environment where people have trust in each other and are committed to a common goal. I've been lucky enough to experience these kind of jobs where it's just professionals committed to a product they believe in. We're all adults so if someone disappears on a side quest that's hurting the team we talk about it. Maybe someone just got distracted and needs someone to point it out, a simple mistake which is ok every once in a while. Maybe someone is having a bad time and needs to take a vacation, understandable and also ok since we're humans first, engineers second.

But most importantly: if someone is regularly hurting the team's goals and isn't willing to improve the situation or acknowledge it then it's simply not a good engineer/colleague/asset to the company. No amount of process is going to change that and even if it does you're probably losing productivity from the rest of the team because they were productive in the old approach. A flat tire doesn't mean cars are a bad mode of transportation and switching to a horse is not going to get you to your destination any faster. Why not replace the defective tire instead?

If the situation I described above sounds unrealistic to you maybe the companies you've worked for need to review their hiring process. If you don't trust the engineers you've hired with the process you'd be doing yourself a disfavor by trusting them to do the job they were hired to do.


"Your anecdote suggests this is a truth about all engineers"

I don't disagree with what you're saying. The scope of what I was saying is more limited to certain projects than it sounds.

I have hope. If I didn't, I'd quit. Unfortunately I make a silly amount of money dealing with these problems in a certain way, and it is a way that (for some projects) involves prescribed process; and it works.


> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity

Sounds like you have infants. Perhaps hand-holding is appropriate here.


Management has changed you.


Very true.


My biggest surprise, when moving from 10+ years at FAANGs to a smaller valley startup, was the cargo cult worship of SCRUM, agile methodologies, story points and all that over just building and shipping value to the customer and moving metrics.

I was also surprised, when starting to get involved in the broader Product Management community, that PMs at most companies (especially outside tech), are mostly project managers pushing JIRA tickets around.


My experience echos your finding, but in reverse. Going from Enterprise companies to a tech company, I was surprised that PMs actually seem to do things. I was so used to Project Managers / Scrum Masters just taking up time with meetings and pushing around Jira tickets with status updates, that when I finally got a Product Manager that really seemed to do stuff, and get a vision for where things are going, but worked well with engineers to get a design it was a breath of fresh air.


That reminds me it's a been an entire decade since I have worked with an actual product manager who knows and cares about the product they manage. Project managers are NOT it


It's also that a huge number of new tech startups are operated by extremely green techs. That means that many have no management experience and very little practical software industry experience. Frequently, they've worked only at other similar companies.

So, a few things happen.

* The boss knows of nothing other than Scrum, so Scrum it is.

* The boss recognizes that he or she doesn't have that experience, figures something needs to happen, and leans on the biggest name in the game.

* The boss is so enamored of the idea that software applies to any of the world's problems that the process is offloaded to Jira or whatever, and now you've got this weird Scrumthing happening.

* The board realizes that there is insufficient management capability and pushes the organization toward it.

If you are a young leader in software, or new to the business, take stock of the reality that software, and business of software, are both far older than Scrum. Somehow, things got done. Ask, "is today's software superior?" That's not really an easy or very fair question, but consider if Scrum contributed to software quality before going all-in.


At my startup, every new management hire thinks they can improve our process by introducing scrum.

Every year we make another attempt at it only to find out that it isn't agile enough to keep up with the pace of the business.


That is why I am building notik.app. Having worked at big tech, I have taken the essence of scrum: single person responsible for a project, weekly updates, clear success metrics and delivering incremental value, and removed the bs which surrounds it.

Oh and tie it with lessons learnt from other projects and suddenly you're in magical territory.


Real product management is actually quite rare. Abs because of the length of the development cycle, mostly not needed


I wrote similar thoughts about all this cargo culting here https://www.lloydatkinson.net/posts/2022/one-teams-eight-poi...


Ok I'll be the contrarian. My team transitioned to agile about a year ago. The difference has been dramatic. We ship faster, are way more focused, able to prioritize effectively, and don't burn out committing to unrealistic delivery dates. We don't have certified SCRUM masters or any of that silliness. We do have a PMO that knows what shes doing and that makes a huge difference IMO. The entire process is very fluid and we'll never go back. It took us a lot of work and iteration to find a working agreement that worked for us. I think maybe thats where a lot of co's go wrong implementing this stuff. You can't just cookie cutter it. It's a collaborative process with your teammates to find a way you collectively like to work together.

And before anyone says that I don't know what it's like on the other side, know my team was following practices very close to that described in this post before switching. An EM decreed that we didn't need all the baggage of sprints etc and ditched it. Total disaster. Everything ended up being high priority (which means nothing was), fire fighting, heroics, burnout, the works. We shipped stuff but we struggled to prioritize larger initiatives such as architectural migrations etc. Really sucked. That EM was an idiot. Also that EM was me.


Please read the article. It has a whole section called "In Defense Of Scrum". The author of the article was personally involved in a situation similar to yours and addressed many of the things you brought up.


From what you describe you don’t seem like an idiot, but rather someone initially doing decisions outside of their expertise and learning from it. That’s the opposite of an idiot in my world.

You have a great PM now which might be a bigger factor than the specific methodology you use now.


Could you share more background? Industry, size of the team, roles within the team, seniority, high level project description, etc


I love this topic.

Scrum is old enough now that many people think it is the way to make software. It's worth recalling that Scrum is a product that proponents use to sell books, win consulting contracts, and get on the talking circuit. It is self-reinforcing. Scrum, itself, is the endeavor. It is not genuinely a tool to assist you in your endeavor.

Also note that practically nobody in industry does actual scrum. And, according to the Scrum authorities:

"Scrum’s roles, events, artifacts, and rules are immutable and although implementing only parts of Scrum is possible, the result is not Scrum. Scrum exists only in its entirety and functions well as a container for other techniques, methodologies, and practices."

I'm not suggesting that Scrum can't succeed, but there's not much to suggest that it will help you succeed. It was never designed for that. Recognizing that Scrum itself is the product is the key to understanding why it often doesn't work and is so disliked.


The phrase “plan -> build (iterate) -> ship” is carrying a lot of water in this article. It’s unfortunately vague. Scrum and Kanban, which are each called out separately, are also plan -> iterate -> ship methods.

Reading between the lines, I think the distinction the author is making is actually between named methods and unnamed methods, some of which are customized variants of Agile methods, and some of which are entirely ad-hoc. For example, Spotify has a very disciplined approach to development that involves coaches and Agile principles, but is listed as plan -> iterate -> build in the article.

The result is that plan -> iterate -> ship basically means “other,” which makes it hard to draw any conclusions.

There’s also an unfortunate lack of distinction between Scrum implementations, which range from lightweight and team-driven to heavyweight and dysfunctional (mostly the latter, to be fair). The author uses the term “heavyweight” to describe Scrum, but it’s not clear that their survey respondents have the same biases.

Ultimately, I think what the author has discovered is less a difference between processes and more a difference between Theory X and Theory Y management. [1]

[1] https://www.jamesshore.com/v2/books/aoad2/management


This is an instance of an abstract pattern I've observed almost every day over my several decades in this industry: people imposing processes, techniques, tools, approaches, architectures, that appear to be inappropriate to the work at hand. A common factor is that the people doing the imposing are not experienced in doing the kind of work they're targeting. Sometimes the thing they're imposing was appropriate in some other context, but not in the current one (e.g. it works at Google but only because they have CI and RCS tools we don't have). Sometimes there's a kernel of truth to the thing, but that got lost in translation (e.g. have daily meetings, but we don't know the purpose of those meetings). The pervasive vibe is of people not thinking too much about what they're asking others to do, and not having any direct experience of the thing they're trying to control.


The idea that Management is a discipline in which one can be effective, independent of who or what one is managing, is silly. But it is a very convenient conceit for Managers of all stripes.

Having a management issue? Bring in a Certified Manager(TM). It could be a PMP, a CSM, an MBA, or a WTF. It doesn't matter. Just airdrop them in and things will turn around.


A long time ago, I hired a scrum master. Two months later, I fired the scrum master, and exorcised scrum from my company. It was bureaucratic, it siloed people (you are front end, and you will only work on front end) turned simple decisions into decision by committee designed to reduce risk to the committee members, favored people who had little knowledge in decision making and put 10 out of 10 client projects behind schedule. I've never really looked at scrum again...


> you are front end, and you will only work on front end

That is something outside the scope of scrum. Whoever you hired may have called that scrum, but it seems they were bringing in unrelated processes.

> turned simple decisions into decision by committee

Scrum isn't a decision-making framework either.


> Whoever you hired may have called that scrum, but it seems they were bringing in unrelated processes.

Every time a Scrum project goes badly, any criticism attributed to Scrum is deflected with this no true scottsman argument.

News flash: nobody is doing the actual, literal version of Scrum as defined in the Scrum Guide. Nobody. So if you use this argument to deflect criticisms of Scrum, be consistent & don't attribute any success stories to Scrum either.


> Whoever you hired may have called that scrum, but it seems they were bringing in unrelated processes.

This is both true and not exculpatory, IME. I've often referred to/heard this referred to as "Scrum-but", i.e. "Scrum, but X", and it's pretty true in my experience that Scrum legitimizes and encourages Scrum-but.

There are places where Scrum is legitimately useful (as a way for managing stakeholders when dealing with war-room problems where the entire implementation team is of roughly equivalent competency and is literally sitting around a table), but beyond that, the first hint of Scrum-but and the boat's taking on water.


Why is it that every time something goes wrong with scrum in an org, it's not a response that all orgs are different, but something that indicates this wasn't true scrum?


Scrum master here (amongst other things). Just want to point out that nothing mentioned in your comment is "Scrum". Though they can certainly be characteristics of terrible attempts at implementations of scrum and other methodologies.

The first thing to do when considering scrum is really understanding the problem to be fixed or process to be potentially improved and if the best approach is an Agile methodology (Not just Scrum, it isn't the only thing out there and isn't always the best solution). Then only adding as much as you need.

Scrum really shouldn't be adding any bureaucracy. Not any more than a sufficiently complex project would already have inherently. In fact just the opposite, it should cut down on bureaucracy and make teams more (drumroll) agile. Which is the whole point of self-managing teams.

Sounds like this guy did a hacky job while calling it Scrum, and was not at all the right hire for your situation.


> Just want to point out that nothing mentioned in your comment is "Scrum".

Oh, it was scrum, complete with an "Certified Advanced Scrum Master(tm)" certification.

>Sounds like this guy did a hacky job while calling it Scrum, and was not at all the right hire for your situation

I agree with this. It was the wrong hire, and the wrong way to manage our software development. It was terrible, and I'll never give scrum a second thought.


No True Scotsman


No True Scotsman.


I left a team mainly because the developers wanted to create the user stories per technology layer, front-end, api, backend, DB basically. I fought, fought and fought, but nobody was listened. Not worth it but unrelated to scrum as many said.


Be careful, a Scum Master(TM) is bound to appear out of the woodwork and will find your lack of faith disturbing. Surely your lack of success can be attributed to a lack of believing hard enough.


> It was bureaucratic, it siloed people (you are front end, and you will only work on front end)

Not sure where Scrum says that. In fact I remember it being the inverse. Maybe you just had a bad hire.


I recently made a transition from a company focused on scrum to a FAANG working the way above. A lot of comments here are saying that engineers in this system can just do whatever they want without constraint.

I think what people are missing is the focus on documentation that FAANG have in contrast to the focus on in person meetings in scrum.

Things like sizing and planning get done in the initial stages of a project. The design docs and plans are peer reviewed and only once it has been signed off on does the actual implementation start. During implementation you have much autonomy but this is because most of the details have already been ironed out. The code is almost an afterthought.

This has been my experience so far but also at a lower than senior level. So take it with a grain of salt.

Also deadlines feel looser. Things regularly get pushed back quarters because of dependencies on other teams. Because there is no real fear of the company collapsing. It's okay we just work on something else.


How much of this is possible due to the fact that "there is no real fear of the company collapsing?" This sounds like process-resembling-waterfall, which seemingly works well because use cases, constraints, and patterns are (relatively) so well understood at an established company. Is there "edge" or value in that process, or is the value in the understanding of use-cases, constraints, and patterns (or a drive to align towards reaching said understanding) that enables the process?


I think the second

"the value in the understanding of use-cases, constraints, and patterns (or a drive to align towards reaching said understanding) that enables the process"

These companies operate at a different scale, speed and constraints to other companies. Just as people say "you are not Google" don't use Google tech solutions for your problems. Don't use FAANG processes for your use case without understanding why those processes are the way they are.

As an addendum to my first comment I want to also say that no scrum does not mean no processes. In fact there is a lot of processes. The number one complaint is that there is too much processes. They are just not scrum.

There is no one size fits all solution which is what the product "Enterprise Scrum" tries to be. Teams need to understand their needs for themselves and build/iterate on a set of processes that works for them. Large orgs need to enable this as well as set the baseline for processes that work across the whole org at every level (indivudual -> team _-> product area -> organization).


Ok, thanks for your insight.


Running scrum is kind of a red flag for me, not a strong one but a concern for sure. Maybe this is unfair but for me it indicates a potential cargo cult mentality.


I was recently looking at the state of the contractor market and was shocked by the prevalence of Scrum, the lack of distinction between Scrum and Agile, and the number of places that apparently think Scrum is a modern development process that will appeal to good engineers. Perhaps they'd like people to go through a four stage process lasting about a month and including a weekend-long take home test and a LeetCode panel interview as well? At least it makes it easy for anyone who is better to stand out!


Exactly - I’ve definitely seen a surprising number of companies here in Europe where they talk about using Scrum as if it’s proof of them being an innovative and modern company, and really make it a big selling point in their hiring materials. It comes across (at least to me) as a bit old fashioned and out of touch.


I suspect one problem is that the people who are high enough to be bringing in senior+ contractors have often been at the same company for a while themselves and may not realise how quickly their impression of the rest of the industry can become outdated. I had a discussion with someone not so long ago who was very senior, basically running all of software development in a tech business well past the early startup rush phase, who told me quite sincerely that everywhere did daily standups and worked off a Scrum-based process these days. This was someone who I respected for being smart and open-minded about most things, yet they had this intense blind spot about how tired Scrum and all the rituals that come with it are looking to a lot of good developers now.


Anecdotally, the worst part about scrum is the skewed incentives it creates. When you start doing scrum, a few things become really hard.

- Long term planning, few impactful projects/features ship in 2-3 weeks. Many can ship in 3-6 months. Talking about these things becomes exceptionally hard as debate on whether there should be a plan takes center stage.

- Focusing on the right things. Do you care that the sprint points got done, or do you care that the customer's problem was solved? what if the two are in conflict?

- Organization politics. When 5 teams publish burn down charts and one goes in the wrong direction, what happens? What happens when you ask another team to do something and they don't deliver? Do we haggle over "acceptance criteria"?


My company is undergoing an "agile transformation" to use Scrum. Whereas before my team was working prioritising people and communication over process we're now "committing" to arbitrary deadlines and bogged down by process and useless meetings. Rather than being able to respond to change we have set scope and bringing work in is frowned upon because it might cross the "deadline". We don't even have a fixed release schedule so the fortnightly sprint boundary is pointless.

I can see why you might need to run a low-trust low-skill offshore project this way but actually existing Scrum is worse than even the waterfall my first company did.


Sorry to hear again that common refrain.

Imagine it's 2002 and you're pitching a great new way to get work done: SCRUM, and its heart, the Sprint.

From the docs: "A new Sprint starts immediately after the conclusion of the previous Sprint."

Now imagine ever believing that sprinting for eternity is good, or even feasible. It really is a testament to their marketing prowess that this ever took off.


No process will solve what is really the underlying challenges companies face when building software.

A process is a set of boundaries and has to be agreed upon within the team. If the agreement is slipping then the boundaries of the process is being broken.

It's never a process fault that software teams can't deliver. Never.

It is always:

- handovers to other teams; testers, release, business. - communication between teams or outside stakeholders which has no actual skin in the game for delivering within the team. - keep communication inwards to the team and have the business really really close. - mandate and empowered teams can plan, run, test, release, monitor and adjust without any outside-of-team interference. -teams should be small because it will decrease the communication links and increase the possibility to create a good culture within the team.

Enough has been said about processes during the last 20 years. Too little has been said about organization layout for creating productive and happy teams.

I have worked in a few places where the process was well implemented but where we didn't have any mandate or autonomy, and because of that our culture was poor and people was frustrated.


All relative I suppose. Scrum on its own might feel refreshingly light if you've been experiencing something like: https://i0.wp.com/jeffgothelf.com/wp-content/uploads/2021/05...


Just wow.

I "love" how in that picture, the individual contributors doing the actual work are the only ones not having a label. And it took me quite some time to even find them, and where the programming happens.


You don't need to label your group of anonymous resources any more than you'd need to label the spokes on a gear.


I had never been able to make sense of that picture. It just looks like someone took all the premade symbols PowerPoint had to offer, combined them with all the buzzwords a quick Google search revealed, and scrambled everything on a canvas.

Still, there’s bound to be thousands and thousands of people taking this seriously (it looks like satire to me) and at face-value.


I suspect it's optimized to help reflect the need to purchase their training materials, pay for their certifications, hire their partners for coaching, etc. If it were simple, why would you need help?


All I see is, "You get a job, and you get a job and you get a job and everyone gets a job!"


My company is in the process of implementing SAF. How doomed am I?


The only benefit of Scrum today is that it has a fixed set of rules so if you come from and old linear sequential methodology it is a great starting point for beginning thinking in agile terms. But as anybody will notice the rules of Scrum are inflexible and completely random in some areas and not very agile. That is why Scrum is pretty much dead.

To use a methodology is extremely effective though. If you work with people with various knowledge and experiences in software development, it is must. I have seen companies go from Kanban back to cowboy coding and completely stopped producing anything at all that ended up in production.


I work in "BigTech" and larger projects often have 'sync' meetings to give updates and note where you're blocked. You also typically have 1:1s with your TL (or maybe stakeholders with other teams depending on scope) where you typically give updates on what you're working on and might chat about challenges/blockers.

These systems are crazy complex, proprietary, and constantly changing plus you need agreement from a number of people to do anything making it very easy to get blocked so you really need to give people a forum to escalate technical or non-technical blockers.


The article mentions Plan->Iterate->Ship a lot.

How is this essentially different from Scrum sprints?


1. Scrum sprints have all sorts of goofy restrictions, that you don't need to waste time on with a true-agile process. You don't need to plan out exactly two weeks. You don't need to worry about having too much or too little work in the sprint.

2. Scrum requires you to break things down pointlessly, when really most teams can execute on higher level goals.

3. Teams do not need to meet for a standup every day.

4. Effort estimates can be extremely approximate and wrong, but scrum tries to force you to spend way too much time on them.

The "plan" in a non-scrum environment is much better at focusing on what matters, rather than ticking all the scrum boxes.


You can do anything badly. Like point one, no you don't need to plan out exactly two weeks. You estimate what you want to get done in two weeks and then at the end you see how close you got.

It's supposed to be a reality check to see how well estimates are matching up to actual effort. Which I don't really see why you shouldn't do something like that.


Our pointing sessions sometimes last 1.5 hours and involve the whole team regardless of expertise. Just imagine how much money we are wasting to determine: is this story a 1 point or 2 point story.


Goes back to the original point. If you do something badly then it's no surprise the results are suboptimal.


Yeah but if we do it well I still have to sit through a 1.5 hour retro. My therapy sessions are shorter.


I'd say 95% of sprint ceremonies I'm in are a complete waste of time. Still, amen for WFH.


Again, goes back to the point of you can do anything poorly. Ours are 30-60 minutes and usually have good conversations about issues the team sees. More often than not meaningful change comes out of it.


You're having 60 minutes of issues every few weeks? And you think something isn't done poorly? That's way too long.


The only meaningful change I ever saw come out of a "retro" was making daily standups async twice a week (done over Slack.) Then the manager didn't like that, so it was rolled back.

Most of the retros I'd just roll my eyes at. "What can we do differently?" cancel the retros...


Haha. Tell them we can't do anything differently because that would mess up the meticulous Jira/Agile flow we've setup in other meetings and have affirmed in retros.


Pretty sure I said 30-60 minutes. We also don't just talk about issues. We talk about wins and things that go well too.


I'm pretty sure 60 is within 30-60.

> We talk about wins and things that go well too.

It's a tactic for managers to feel better about themselves. Look, the team is expressing gratitude! I'm a success as a manager! If you want to thank someone message them on Slack. We don't need a 30 (or 60) minute meeting for an enforced feel good moment.


>I'm pretty sure 60 is within 30-60.

Uh hu, but since I gave a range obviously we don't have 60 minutes of issues every week. If I meant that, I would have said we have 60 minutes of issues every week instead of giving a range for the meeting length.

>It's a tactic for managers to feel better about themselves. Look, the team is expressing gratitude! I'm a success as a manager! If you want to thank someone message them on Slack. We don't need a 30 (or 60) minute meeting for an enforced feel good moment.

You're projecting so hard here.

(1) We didn't have a manager for a couple months now and retros went on unchanged.

(2) Talking about wins isn't an enforced feel good moment. It's about identifying things that worked well so that the team can continue doing them.

Your position is extreme and not everyone feels the same as you.


I am giving my experience and you call it projecting? That’s an interesting debate tactic.

The team doesn’t constantly need to reinforce what is working. That is self-evident and could be an email or slack message.

I’m done with this thread.


>I am giving my experience and you call it projecting? That’s an interesting debate tactic.

Taking your experience and making definitive statements is the definition of projection.

>The team doesn’t constantly need to reinforce what is working. That is self-evident and could be an email or slack message.

So you say. Obviously other people disagree


Plan -> iterate -> Ship is different than scrum. One big difference is that its not usually time boxed into sprints. Another is that there may be multiple independent projects being worked on on one team. So you might have a team with project A being in the final touches, while an engineer leads on the plan phase for project B, as another person might be leading C in Iterate.


Can't even find a blogpost where this is explained. Anyone got any links please?

* What form does planning take?

* Does someone from the business side approve the UX before it's implemented, etc.?

* Is progress not tracked at all or only informally?

* Are people actually treated like adults to ask for help to unblock things instead of needing 8 people on a call so one person can say they're having an issue and need one other person's help?


Planning is what makes sense for the project scope, and can easily be recursive through orgs and teams. Each layer works out the core components needed, then which teams would be needed for those components. Those teams then work out what sub components are most effectively done by which teams, and repeat recursively. Teams may find overlap for components and assign subsets from multiple teams to work together to plan for those overlapping areas. This would seem like the obvious way to plan for anything.

Why would business side have significant input into UX - that’s the whole reason you have hired UI and HI teams?

An effective bug tracker has the concept of related and blocking bugs, along with priorities, etc. Anyone can see how progress is happening in trackers. Occasional meetings between ICs or PMs can identify where things are falling behind or getting blocked, and either work to reprioritise or re-resource as appropriate.

I my experience that has never been significant impedance between ICs and managerial types communicating directly. This feeds into ICs being able to decide what to work on - if another IC comes to you needing help or needing some particular bug fixed earlier than expected you can always in crease the priority or provide short term work around, etc - or talk to other ICs in your team to see if someone else has capacity to get it fixed earlier.

To me this all seems like the sensible way to get software done at large scale. As I’ve said elsewhere maybe it’s different at small or young companies, but the share interminable “process” of scrum, “agile”, etc seems asinine.


Thanks


There is no Scrum Master but Tech Leads. The former in their degenerated but sadly all too common enterprise form know primarily process, shy away from deeper knowledge and focus on enforcing the former. Shipping becomes the side effect of process. It is safer that way as process can always be controlled the challenges of shipping a deliverable not - at least for the scrum master.


A lot of comments in here misunderstand what it means when they think that teams at Big Tech aren't required to follow a specific process as meaning that those teams don't follow a process at all. The difference is that most senior, capable, and autonomous teams define and figure out their own process that works for them, not a pure absence of any process. Empowered and capable teams are simply better suited to figuring out and defining "just enough" process that works for them. It could just be an ultra-basic Kanban board managed through Linear.app or JIRA. Or it could be a more mature team that is still doing a more streamlined version of Scrum that probably is one or steps away from really looking more like Kanban. But autonomous teams does not mean that those teams "do whatever they want" with zero process at all. That's chaotic and doesn't yield very good results.


I have a couple of problems with this.

1. Scrum is not a heavy process. It's an agile process, the opposite of heavy. It rarely involves Gantt charts, UML, or book-length specs. Kids these days . . . sheesh!

2. "Plan -> Build(Iterate) -> Ship" seems like a made up methodology. Or a badly-copied O'Reilly title.[1] It does not describe the way teams behave. Consider teams as software-producing systems. Systems need feedback to correct; to adapt; to learn. Where is the feedback in this methodology? Yes, there's probably some feedback smuggled in with the iteration, but there needs to be feedback in the system about the system itself.

And because of #2, I'm calling BS. Which was not intended as a poop joke but I will pretend I meant it that way now if anyone asks.

[1] https://www.abebooks.com/9781491984871/Design-Build-Ship-Fas...


> Unlike Skype, Whatsapp never bothered with a framework like Scrum. Early employees shared how they never even muttered the word and deliberately ignored all heavyweight processes. Whatsapp out-executed Skype, built a more reliable messaging experience than Skype, and ultimately won the battle of messaging and communication apps.

Article overall is ok but this part killed me. It's like it's implying that the team management methodology is what made Skype lose, and not that Whatsapp was the right project at the right time that filled a mobile niche, while Skype was an older project that was unlikely to go through a full rebrand. To put it simply, Skype was really bad. Team managament can in some cases stale the developers, but in this case it wouldn't do anything if Skype was already headed nowhere. Unless not using scrum magically made the people working on Skype have a vision and good UI taste.


When it comes to engineering success you have to be very clever to be successful. I’m not sure how hard a truth this is for some people. Every time I hear of a learn-to-code boot camp I worry that we’re training bus drivers when we really need master stone masons.

The worst people to work with are the ones that don’t realize how wrong they are. If you end up with a group of them all reinforcing their mediocrity on a day to day basis then you are in a lot of trouble.

Some shops can tolerate this, especially if they are just screwing together business logic without having to really innovate in terms of pure software. Scrum probably works quite nicely for them. They ship their ISO compliant b2b software once every quarter with the dollars rolling in.

If you know Douglas Adams you’ll know the term B-ark. These are the B-ark software teams.


It's funny that Scrum is called "heavyweight" process in this article. I remember agile methodology being called lightweight in contrast to traditional waterfall process. And the whole "Plan/Iterate/Ship" sequence really reminds me of waterfall.


That's my problem with process management in software development. Things change, processes evolve, thihgs get lighter and easier. We went from bi-weekly releases to deploying a few times a day thanks to automated tests, canary releases, 1-click rollbacks etc. We went from waiting 2 weeks for the ordered servers to arrive to automated capacity management in the cloud. And yet in the process management 10 years ago we had Scrum, and today we have Scrum. No wonder it feels heavy when things around got easier and lighter.


I think over-reliance on scrum or other heavyweight methodologies create a very low ceiling of performance for your team. You can't hire good people into that environment, and you're teaching a form of learned helplessness.

I wonder if instead you could teach better practices to a broader swath of engineers. I don't think you have to be a 90th percentile engineer to respond well to empowerment, accountability, and an expectation to collaborate with others. They may not have seen that behavior modeled before, but I bet a lot of them would rise up and meet the challenge. Even if 50% of the team didn't meet the challenge, you're likely still better off with that smaller but more productive team.


This is useful information for projects/teams/companies doing Scrum. Many of them are likely doing it because they think this is how big tech companies operate and wish to emulate them.


I've been at one small tech company, one large fintech company with insane redtape and one mechanical engineering company. Of all the three, only the mechanical company was slightly religious about using our ticket tracker to figure out what I was doing. Maybe because I was the only developer there for a while.

I wish I could have worked at a place where scrum means something, or where PR reviews actually did what they're supposed to. I'm tired of always being the one who has to teach others.


My reading of comments if that people seem to see two options:

* Scrum/Agile/Current-hot-buzzword

* PMs micromanaging via jira/whatever bug management

My experience in corp engineering has PM as facilitators between teams that are for various reasons not directly interacting at a planning level. I’m not actually if they create bug reports unless they’re filing actual bugs they’ve found.

Generally bug tracking for feature development is essentially umbrellas so you can see where things might be getting held up, and so you can (as an IC) prioritise.

Many of my assigned bugs are my own personal task bugs, again the bug tracker is helping me manage my own tasks. I’m fairly sure this is how the majority operate.

ICs have plenty of freedom for their own prioritization, and generally how they get to the end goal, and they’re able to experiment if they believe their current load has the spare time. There’s never really an ability to just randomly go off into some side project without at least arranging task redistribution with your manager.

Another thing to realize is at bigcorp/FAANG many engineers are working to support multiple unrelated projects, which is again a thing the PMs help with - occasional meetings between people from the different teams working on a given project helps find if there are any issues being missed.

The important summary is, I guess:

* no annoying formal processes like daily stand ups, etc that waste time if different people in a team are working on unrelated things

* PMs aren’t driving development, they exist to support communication

* ICs have freedom to choose what they work on, and how they prioritise, but have all the info required to actually do that correctly.

* with said freedom ICs are expected to do prioritise appropriately and effectiveness impacts responsibility (and presumably comp) in the future

* I don’t know how things differ if you’re an IC wanting to go into management/PM land because that all sounds awful so I’ve never investigated :)

The way things like scrum appear structured seems to be based on a lack of trust in ICs, or a sufficient overall lack of experience among people in the org, so maybe this simply represents a valuable tool in early stage companies.


My hot take - project management is about co-ordination with other teams

If you are co-ordinating through the code, you hardly need a common methodology let alone a common tool. If you are co-ordinations outside of the code (ie by having a project manager send their updates up a chain) then yes common methodologies matter

So this is a problem of software literacy - a company that cannot introspect itself cannot see where it is.


The best of modern software project management is not just Agile. It is the confluence of the spirit of Agile and the workflow patterns that go best with a git-based coding workflow. That has had a few years to evolve now and has incorporated CI/CD. That may seem obvious to some. But I think the influence of git having been designed for a certain type of workflow is underrated.


Can you describe some of these patterns for someone at a SCRUM feature factory with no CI/CD


Working on it and more re how Agile patterns really emerged.


Agile industrial bullshit scale:

Level 0, aka "kanban with continuous deployment":

- Tickets in backlog arranged in approximate priority order. Meetings with business stakeholders (e.g. product managers) take place where the most pressing objectives are discussed, and as a result new tickets are refined and added to the backlog. As a result of changing priorities, existing tickets may be reprioritised or deleted in these meetings.

- Pull requests are reviewed and then merged to the main branch. After passing integration tests they may be deployed to production either automatically, or at the click of a button.

- status update meetings take place a couple of times a week. Or perhaps they are daily but asynchronoulsy via slack or a web app.

- there is regular, adhoc communication and planning both within teams and between teams

Level 1, aka "scrumban":

- In addition to the meeting(s) to refine tickets, there is also a "sprint planning" session where approximately 2 weeks of tickets are put into the following sprint, and a "sprint goal" is assigned that approximates what the team hopes to achieve

- If another team or business representative brings up new requirements half way through a sprint, they are told it must wait until the next sprint.

- If the team runs out of tickets towards the end of the sprint, more are drawn in. Other times work will take longer than expected, or be blocked, and tickets will roll over to the next sprint.

- There are short daily standup meetings, where each team member gives a brief, free-form statement of what they've been up to, shares any learnings or problems they wish to share with the team, and maybe gives a funny anecdote from their personal life

- Every sprint has a retrospective meeting where people talk about what went well, and complain about things that they didn't like.

Level 2, aka "Scrum as a religion":

- Managers use burn-down charts as a way of measuring the effectiveness of the team's organisation and planning. If tickets need to be drawn into a sprint or roll over to another, this is taken as a sign of bad planning. The team is asked how they will go about being more consistent in future.

- Developers may be required to vote on a number of story points for each ticket. Managers may become very exicited that story points don't estimate the amount of time taken to complete a task, they measure complexity, whatever that is. Even though the creator of story points has stated he meant for them to measure the time taken.

- The daily standup meeting requires each team member to follow a precise process, where they give an update on each ticket they have worked on in each column. Any time a ticket has taken more than a day, the manager asks whether there are any blockers. Any time a ticket has been moved to done, they are congratulated by the manager. After these updates, the team must then discuss the sprint goals

- the bi-weekly retrospective meetings are required to have outcomes. These often involve either more meetings or more processes

Level 3, aka "Am I in a Dilbert cartoon?":

- The senior managers involved in coordinating planning across multiple teams decide to introduce some ideas from the scaled agile framework. Multiple sprints are grouped into blocks of 6-8 weeks. Large numbers of JIRA tickets are refined and divided into epics before the start of each mega-block-sprint. Each week senior management measure the progress towards business objectives with regard to how many tickets remain incomplete in each epic. If a team gets to the end of the 6-8 week megablock having somehow done exactly the same work as they originally planned they are congratulated for being highly Agile. Those that don't achieve this are told they've missed their targets.

- Releasing any service to production requires filling out a 20-point spreadsheet checklist along the lines of "have any breaking changes been notified to other teams? has this been tested in a development environment? have new tests been added?". Pull requests must then be made to multiple repositories to actually tag and release a new version, and a series of slack messages must be manually posted on the status of the release.

- Making any decision beyond completing work on a JIRA ticket requires scheduling a meeting

Level 4, aka "Scaled Agile Framework ®"

- The megablock multi-sprints are 10-12 weeks, and are alternated with "hardening" sprints where people fix all the bugs they introduced in earlier sprints, and possibly "innovation and planning" iterations, which is the only place anyone is allowed to do any work that requires any thinking.

- Half the employees have titles like "Solution Train Engineer", "Release Train Engineer" and "Scrum Master"

I've just reached level 3, previously couldn't get beyond level 2. Anyone managed level 4? Are there bonus levels beyond that?


I've been at a level 3 org, had to fill out questionnaires in Confluence and link them to Jira tickets to get stuff deployed. Fun times. Not really. I left pretty quick.


great summary. not sure whether to laugh or cry. my org was level 4 and was one of the major factors in causing me to move out.


When this article was first published I contacted the author and told him it was misleading; Scrum (or large portions of it, like time boxed iterations, daily stand ups, and sprint boards) is popular in a number of big tech firms (I’ve worked at 3 on teams that have used it).


honestly, i've always been of the impression that SAFe and, to a degree, Scrum have just been adapters installed to "increase engineering autonomy" in a way that maximizes compatibility with traditional project management. at hugeco's, someone, somewhere, is translating story points into man-hours and velocity into project burn in dollars, usually in a guarded Excel spreadsheet.

engineers running the show at these places would accelerate companies to the stratosphere (because engineers can do pretty much anything with enough money and autonomy). a man can dream.


Great intro story of Skype vs. WhatsApp!


This is basically why I opted out of any leadership role at my company. I'm not interested in being more involved in scrum/jira/whatever. Absolute hell and bringing in a dedicated scrum master makes it 1000x worse.

I hate that I have to hear about deadlines as they relate to scrum and not business objectives. Or rather, the only business objective is to get something done in a quarter so we can show it off.

We also can't start work on something mid-quarter if it won't finish in a quarter. Outcome is the same EOY but getting there sucks.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: