Do not confuse difficulty with prudence. Your organization will become slow and ossified by default unless you take specific steps to maintain flexibility. One of these steps is to install a culture of change. Things that are easily undone should not require approval to be done in the first place.
Do not allow long-timers in your organization use "caution" or "good engineering practice" as an excuse to slow everyone else down. Emphasize that the most important part of software development is moving fast. Let new people try new things. Minimize the number of people who can say "no".
Most of all, do not just _believe_ people who talk about best practices and software "quality" and stuff like that. Most of the time, they're just finding fancy-sounding ways of saying "nothing should change unless I say so".
You really can't make categorical statements like "formal process is the enemy". You put formal process into place, and that creates a dialectic where you're waiting for someone to come in and champion the antithesis: "formal process is the enemy". Then that works for a bit until it doesn't and someone synthesizes: "sometimes formal is better sometimes informal is better". Then that becomes the thesis.
Then there's a new dialectic in place. Someone will come in and say "the character of our company is such that we're generally better with formal processes". Then someone synthesizes that, and suggests that all processes are on a trajectory from formal to informal and the company is good at expediting that.
Then someone realizes we need a process for regular deformalization.
This continues forever. None of these ideas are really that much more generically true than any other.
So to embrace Hegel, the goal is just to question yourself as far as you reasonably can, so as to get the best view on the current situation and only the current situation. Make a choice and move on, knowing the next situation is likely to be different.
Edit: I should add this is the process by which we hurry into The Future, which some people might point out means turning our backs on Everything That Is Holy and is therefore a tool of Satan, but I'll leave you to explore that with the rapidly growing universe of anti-Hegelian YouTubers.
Being unwilling to adopt a necessary amount of structure and process is just as much of a hindrance once you hit a certain size.
That may be a newer or less common problem, but it's a very real one.
Every change I proposed was questioned to death and generally dismissed, I was micro-managed within an inch of my life, and I found the two key Directors just could not let-go of any control of their 'baby' (the company), to a point where any tasks I tried to assign to team members were met with pushback because the Directors had already assigned them other things, told them to work on other projects or asked them to report back before accepting any work from me.
One time when I was asked to work on a proposal to take over the migration, management and support of the Intranet for a large local University, I completed the initial RFP document and reported back to the Directors with a plan for a meetup with the potential customer because some key points needed elaboration, and some of the SLA terms could not be met without additional team resource, but they told me they had already phoned the Uni and agreed to go ahead 'as is'.
I left after 9 months.
No, that is not the case most times. The most important part of software development is, as with any other part of the company, to maximize the value of the company. That may be speed, but if you have 1m users the most important thing may be to preserve trust, to acquire new customers, to automate existing infrastructure, etc. etc. Speed of software development is not a goal in many environments. Thoughtful execution may be hindered by a cowboy attitude in development. I'm not saying that speed in development is not sometimes the goal (maybe in startups within an enterprise trying to test an idea), but often it's not the most important thing.
To this you've appended the hackneyed conflation between "speed" and "cowboy attitude". It's a common category error, but in truth, rapid iteration is one of the enablers of high quality and is a huge effectiveness multiplier for top shelf dev teams. Slow development merely enables lazy programmers (since it conceals their apathy) and gives us waterfall disasters. "Thoughtful execution" is possible in any context, and most thought processes are greatly enhanced by evidential techniques such as rapid prototyping / feature spiking.
Fast+incompetent = cowboy attitude, sure, but the problem there isn't the fast bit.
In a well-functioning company, I don't need to make that decision myself, of course: I have management who has evaluated the cost of having the bug and the benefit of having the feature, or who's experienced and smart enough to determine whether doing the short-term thing now or investing in the longer-term thing will be likely to have better payoff. Because this process is there, if I trust my management, I don't have to spend-time second-guessing them (or worse, doing the research myself), and then I get to move quickly. But again, moving quickly isn't the point. It's a tool on the way to delivering value.
Exactly right. Frequently, in industry, I see people confuse the "slow" and "competent" bits. They think that just because a given change needs to get a bunch of sign-offs that the overall effect of the system requiring the sign-offs is a net benefit to the organization.
It's not. It's paralysis, and you won't notice how ungainly and slow you've become until a different org a tenth of your size manages to clone your entire feature set in a hundredth of the time it took you to develop it, all because your developers have to deal with bullshit all day long and your competitor's can just act.
A lot of people will claim that you need gatekeeping and approvals and such to maintain a rigorous standard of quality. You have to be brave enough to disregard their advice. You have to be able to take the gut-wrenching step of saying, "No. We're not doing that. I don't care if this change caused a SEV, we're not adding process. Moving fast is important!".
It's very easy to believe the nostrum of "no pain, no gain". But sometimes you have to realize that pain is just pain.
The moment that some minor disaster hits, which it inevitably will, whoever is in charge will find themselves no longer in charge.
I think this is one of the fatal flaws of a big company. Everyone is terrified of risk. I'm not sure it's possible to counteract this tendency unless the company was built from the ground up not to punish risky behavior (e.g. Facebook, supposedly).
Enabling people willing to act is important though and I fully support the do nothing aspect of failure, or better yet support and stabilize but not reward.
I think that's the key thing here. Rather than saying "whether you succeed or fail, you're still gonna get your $250 bonus", you're saying "if you try and succeed, you get $500- if you try and fail, you get $100, if you don't try at all, you get nothing."
I admit to inheriting a fairly smart-risk-accepting technical culture, but also worked to extend and cement that by holding regular blame-free post-mortems on production problems and reporting them weekly to our business operations meeting. Making failure and the analysis/correction thereof a regular part of company operations makes it normal, accepted, and less scary. We would also pretty regularly respond to asteroid-type production problems with "no preventative action intended; cost of prevention exceeds expected losses". (We held a view that there is [conceptual process] green tape and red tape; make sure if you're fixing problems with process that it's actually green tape and if fixing a problem required adding red tape to the system, we were very skeptical and tended to avoid adding that process/step/gate/check.)
Couple that mindset and transparency with a metrics-supported track record of making things better overall and management will support. I'm not sure our CEO ever saw any of that sausage-making, except for the very largest or most damaging problems and even then, it was mostly a courtesy message to him. He cared about the overall pace and metrics, not how many outages or bugs we had along the way.
The ratio of students that decided to major in engineering because they were good at math in high school vs. the students who decided to major in engineering because they they wanted to make something inspiring, I feel ( and let me stress my inexperience as a current undergrad, without experience in the actual field ) is vastly favored in career growth towards those that decide to create instead of simply learn.
While there is always room and lucrativity to tie engineers on a leash and juice mathematics and logic out of them until they reach an existential crisis, there is always (ALWAYS) room within the budget to let them set out upon a path which is in tune with their not necessarily young, but foolish and hopeful beliefs.
There's nothing wrong with process and best practices. Process is often scar tissue left behind after critical mistakes in the past. A company without some formal process is a company young enough to not have made disastrous mistakes yet.
I've worked in a company of more than 50 that was basically without a trace of software development process. It was a clown circus. Nothing could be released on time reliably, quality was a joke, there was no documentation, no code review, and no plan or roadmap. Sure, they moved fast, but it didn't matter.
This doesn't seem to me to be true. Can you expand on your thoughts here?
I'd agree that the ability to move fast is an essential part of effective software development, but I don't think it's the most important (the most important is producing business value accurately and reliably), and I think that moving fast for the sake of moving fast is likely to result in moving quickly in the wrong direction. In particular, it's easy to be so worried about moving fast that you have no time to take new business needs/opportunities into account (see also Tom DeMarco's Slack), and it's easy to get excited about shiny new technology and spend a lot of time implementing it and roleplaying Google for no reason, which is a great way to move very quickly while not delivering a cent of business value.
Finding somewhere with formal process, with full-time managers who aren't reluctantly moved to management for career advancement, with clear hierarchy and goals, was an important part of my most recent job search, simply because, as a good engineer, I hate to see my work go to waste. I have heard the same sentiment from lots of my talented friends.
By default, all software development organizations slow down. I say that developer velocity is the most important goal because, without focusing on this velocity, you end up solidifying, like a cooling lava, and it becomes difficult to accomplish any of your other goals.
I can make the exact same argument about automated regression tests. As the size of your codebase grows and the number of customers (i.e., people triggering edge cases) grows, unless you take conscious steps to make sure that every build doesn't regress previous bugs, all your effort will get sucked into diagnosing and fixing things that you've fixed before. And all your developers will be fixing bugs but you won't be fixing new bugs and certainly not shipping features.
In particular, if you want to refactor / rewrite parts of your code to improve development velocity, you'll suffer from the well-documented problem (see e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-... ) of losing all the knowledge of the weird edge cases you've fixed along the way. So, testing is strictly more important than development velocity!
I can make the same argument about a half-dozen other development best practices, and I'll have actual citations to back them up. Why is development speed the most important goal, why is formal process the enemy of speed (instead of its friend, because it makes sure work doesn't get duplicated or wasted!), and what is your evidence for these claims?
In particular, if your claim is true, we would not expect to see as much feature development / new products from old, large companies (I'm thinking of Oracle, Microsoft, Apple, Amazon, etc.) as we do. We would expect to see these companies grind to a halt and perform poorly in the markets (investors really want to see growth, not just continuing to be as good as you were last year), and that's not what's happening.
It's not a question of preventing formal processes, doing that indiscriminately is a recipe for disaster; every engineer will do things their own way, and once the head count exceeds the ability for everyone to talk to each other daily, there will be an explosion in complexity and duplicated effort. Rather, you need to put in the right processes, maintain them just as you would code, and kill them when they no longer serve the company. Where "process" goes wrong is when it's used as a hammer to address issues that would be better served by team structure, common tooling or architecture. For instance, if you have a quality issue, adding more code reviews will generally cost a lot more, and do a worse job than making the responsible team wear the pager for their own code.
The real secret sauce to staying agile as a company grows is small, cross-functional teams. This is where SOA (microservices for the kiddies) really pays for its overhead—by decoupling human teams and giving them agency. This won't make a big company as agile as a startup, because that's literally impossible, but Amazon and Facebook have shown how much more agile it can make you than any prior megacorp.
Funny you mention microservices, because I've been thinking about Susan Fowler's book about them and how the advice is basically "Here's how we made sure that we kept up our agility by introducing tons of process before anyone could think of deploying a microservice, because otherwise every team would do literally everything their own way and it would be great until it turned into an unquenchable tirefire."
I'm not aware of Uber's technology organization having stopped producing business value. That company has a lot of problems, but the inability of their software engineers to engineer software doesn't seem to be one of them.
You also have the situation that people invent rules and requirements in the absence of a documented process, which can change arbitrarily and are impossible to work with because it depends upon the changing whims of whoever feels they are in charge. Clairvoyancy and divination should not be a job requirement!
Process becomes problematic when the process grows and ossifies and becomes the end itself, rather than the means to an end. You end up with it being impossible to get work done because the process is so burdensome. When your bosses care more about the process being followed to the letter rather than the actual completion of the work which the process should be enabling, that's IMO the point where it needs a rethink. Unfortunately, organisations have a tendency to accrete the stuff; I've in the past argued for less to the amazement and incredulity of my bosses. It's also a factor that process is used as a blunt instrument to wield power over others, and I think that's certainly a driver as well. But often organisations favour accountability and rules over creativity and efficiency, even if you spend 95% of your time with process, and 5% actually doing productive stuff...