> If it's entangled, the branching analogy is a merge conflict.
I like this observation. As long as two developers touch the same code, you will have a conflict. Now your choice will be where to best place that conflict. In an SCM, in conditional statements, or somewhere else.
> For testing, the prod tests run the prod config, which is likely with all optional feature toggles turned off.
I don't understand this. If you are going to activate a new feature in prod, surely you test it first? Or do you start testing all over again before you switch on a new codepath? Feature toggles must also include externalities such as backend systems changing their APIs. Even if you are forwards compatible, you must take care so your application doesn't break when that happens, and that means testing in advance.
If you really follow the master-is-production branch free model all merging must cease before you activate a feature in prod? That doesn't sound very practical.
> Do you run all possible branches at the same time? Is it an exponential combinatorics problem to have multiple branches? Not really, each team runs the tests for the features they work on.
A team can not be responsible for testing only their own development work, they must also know that it works with other people's changes that might activate before theirs does. That cross team communication is hard enough using branches, but at least teams do not disturb each other's testing. That enables asynchronous development. In a master must always be production model, which seems to be what most branchless development people seems to advocate, every combination is blocking. That's the difference.
> The great thing about feature flags for testing is that you have greater control over who is exposed to your new feature.
Right. I'm not saying that feature flags are useless. I'm just questioning that it can be used instead of branching in non-trivial projects. There are a lot of use cases where it makes sense, for example user interface stuff where branching unnecessary.
As you point out, feature flags are just a new term for application configuration. We've had that forever. It is as an alternative to developing in branches I question it. We generally don't celebrate making everything an option.
> I mean no insult, but you are jumping to conclusions from lack of experience. Feature toggles as a pseudo-branching workflow predate git by decades, they are used on the largest codebases in the world -- did you miss the comments in this thread from MS and Google?
Oh, absolutely. I type this because I want to hear from people with experience. What I meant by "my suspicion so far" is that the experience with branchless development I've heard so far is either trivial or theoretical. There's a lot of pointing at Facebook as a good example but not much in way of first hand knowledge.
And yes, I did notice the comments. The Microsoftie called it "if-statement hell", the Google commenter said it "impacted developer velocity" primarily because of the testing requirements. The former can perhaps be explained with culture, but the latter matches my experience exactly. These comments concerned feature flags in general however, not branchless development which is a special case of it.
Your examples from the game and film industry sounds like it could be made workable, but I imagine that development looks different from the average ERP system or backend service. Certainly with smaller startups you shouldn't overcomplicate matters.
> If you are going to activate a new feature in prod, surely you test it first?
Yes, my teams test features before pushing to master regardless of branch strategy. What I meant is that the prod/master/nightly tests stil run the prod config as-is. Teams run tests, but features under toggles aren't tested using the master branch's automation tests until the feature is turned on. Exactly like how a team branch won't get tested in the nightly run until merged.
The QA dept. is typically running the staging branch, which is whatever got pushed to master, but hasn't gone live yet. When I turn on a toggle, it's just like I merged a branch, so QA starts testing the new features at that point. When we want extra testing, we set the feature toggle so that QA is exposed and the public is not, then if it passes, the code goes live with the feature toggle turned off.
> A team can not be responsible for testing only their own development work
When we test feature toggled code, my team runs all the automation tests with my team's features turned on. Similar to how we'd run tests as if our branch was merged into master, but without pushing the merge.
What we don't do is speculatively run tests with other teams' feature toggles turned on. Exactly like how you wouldn't run tests against someone else's unmerged branch. Their features are tested against ours after one of the teams turns their own toggle on, once one of the features is made public -- to at least our teams if not everyone.
> The Microsoftie called it "if-statement" hell
I've heard first hand accounts about their branch hell too. And it's a worse hell than I've ever seen personally. From about 10 years ago, not today, so I don't know how different it is, but someone in Research said it took a full month of merging and conflict resolution for a code merge in any given dept. to propagate to the rest of the company.
If that's happening, feature toggles definitely won't fix it. There is a chance it could make things worse.
I'd tend to agree if you strictly feature-flagged all commits as a complete replacement to branching, it would probably be ugly. Git is useful, and branches are a nice workflow for most dev.
My criteria is to choose the one that makes sense. Feature toggles are currently better IMO for long-running feature dev or very large features, branches are better for medium sized features. Adding, merging, then removing a feature toggle during a single sprint is more overhead than it's worth.
What we did in games before git was everyone pushed to master at all times (or whatever they called it in Perforce, or the lovely SourceSafe before that.) Most people did not wrap their features in toggles. We lost a ton of developer velocity due to someone checking in a bug that broke the build for everyone who pulled after that. More often than not people would hurry to try and fix their bug rather than revert, occasionally checking in further bugs. Company-wide breakage was a weekly occurrence. Those were dark days, it was ridiculous. Feature toggles were the only safety mechanism we had, which is maybe why I see them in such a positive light.
I would not recommend going whole hog with feature toggles and eschewing branches. But I do recommend trying one or two and feeling out exactly how the whole process would work for you from start to finish. They are still useful in a branch environment, especially for features that would cause lots and lots of branch noise.
I like this observation. As long as two developers touch the same code, you will have a conflict. Now your choice will be where to best place that conflict. In an SCM, in conditional statements, or somewhere else.
> For testing, the prod tests run the prod config, which is likely with all optional feature toggles turned off.
I don't understand this. If you are going to activate a new feature in prod, surely you test it first? Or do you start testing all over again before you switch on a new codepath? Feature toggles must also include externalities such as backend systems changing their APIs. Even if you are forwards compatible, you must take care so your application doesn't break when that happens, and that means testing in advance.
If you really follow the master-is-production branch free model all merging must cease before you activate a feature in prod? That doesn't sound very practical.
> Do you run all possible branches at the same time? Is it an exponential combinatorics problem to have multiple branches? Not really, each team runs the tests for the features they work on.
A team can not be responsible for testing only their own development work, they must also know that it works with other people's changes that might activate before theirs does. That cross team communication is hard enough using branches, but at least teams do not disturb each other's testing. That enables asynchronous development. In a master must always be production model, which seems to be what most branchless development people seems to advocate, every combination is blocking. That's the difference.
> The great thing about feature flags for testing is that you have greater control over who is exposed to your new feature.
Right. I'm not saying that feature flags are useless. I'm just questioning that it can be used instead of branching in non-trivial projects. There are a lot of use cases where it makes sense, for example user interface stuff where branching unnecessary.
As you point out, feature flags are just a new term for application configuration. We've had that forever. It is as an alternative to developing in branches I question it. We generally don't celebrate making everything an option.
> I mean no insult, but you are jumping to conclusions from lack of experience. Feature toggles as a pseudo-branching workflow predate git by decades, they are used on the largest codebases in the world -- did you miss the comments in this thread from MS and Google?
Oh, absolutely. I type this because I want to hear from people with experience. What I meant by "my suspicion so far" is that the experience with branchless development I've heard so far is either trivial or theoretical. There's a lot of pointing at Facebook as a good example but not much in way of first hand knowledge.
And yes, I did notice the comments. The Microsoftie called it "if-statement hell", the Google commenter said it "impacted developer velocity" primarily because of the testing requirements. The former can perhaps be explained with culture, but the latter matches my experience exactly. These comments concerned feature flags in general however, not branchless development which is a special case of it.
Your examples from the game and film industry sounds like it could be made workable, but I imagine that development looks different from the average ERP system or backend service. Certainly with smaller startups you shouldn't overcomplicate matters.