We have an automated system that emails you after 30 days of a toggle being on 100%. After that, it emails you once a week. Soon you get bothered by it enough you take it out. System works.
The team I joined uses feature flags like this a lot.
> When are those conditionals deleted?
For us, almost never. A few weeks after the feature flag becomes True everywhere, we just never even look at what would happen if the flag is False. This helps the confounding of multiple flags, but also thankfully multiple flags usually don't collide-- they're all fairly specific changes.
> What strategy do you use to decide when to permanently enable a feature and simplify the codebase?
When I'm refactoring/cleaning-up other stuff.
The really big benefits I see for feature-flags is that:
- You can merge into master faster. Often my feature has added some utility function or refactored something or added some css that the other engineers could (and should) use. So it's great to have the code rejoined sooner.
- Our tester can test on prod, for the few situations where our QA server is not an exact enough replicate of prod.
- It makes it really easy to show marketing/business/TheBoard new features that are _almost_ ready but all our bugs weren't quite finished before boardmeeting/sprintend/etc.
That's dangerous. I remember an incident in which a team accidentally emptied their production experiment config (which controls which features are on: during ramp-up as a whitelist of enabled users and/or a percentage in (0%, 100%); most experiments in the file had long since reached 100%). Suddenly all their experiments reverted to 0% and they were serving live traffic in a totally untested state. Some of these experiments had been on for literally years and now suddenly were off. The combination of experiments had never been tested. There had also been many changes to the code since most of these had last been disabled. As you might expect, this didn't go well.
(That incident could also be part of a long and sad cautionary tale about the dangers of global configuration files, btw.)
The best practice I've seen is for the configuration to specify an end date for each experiment. If it's still listed in that date, a ticket is automatically filed against the defined "owner" of the experiment. The ticket reminds the owner to finish the experiment, remove the conditionals from the code, do a release, wait until they're confident there will be no roll-back to prior to that release, and then remove the experiment from the config.
For temporary feature flags, we try to clean up as soon as possible. As part of the rollout plan for a new feature, we'll have defined what metrics we're trying to achieve, and once we're confident that we've met those marks, we clean up the flag.
We eagerly create "cleanup branches" and corresponding pull requests to remove a flag-- that front-loads the cleanup work and means we aren't context switching back to try and remember how to clean up a flag.
Also, we've found that in practice flags are usually independent-- testing permutations of flags shouldn't typically be necessary. People usually take an analogous shortcut with feature branches-- we test branches in isolation but don't usually re-test after merging.
Funny thing is, that I've worked with multiple (unrelated) teams who simply refuse to use branches. This keeps boggling my mind (I usually don't work with code directly, since I am a tester, and focus on the end to end verification and validation, not with the details and unit tests), especially the argument that is used: "we want to move fast". On one occasion I could understand it, since they used SVN and externals, and each branching process somehow required 3 developers and 4-8 hours (I never bothered to drill down to the "why" behind those numbers, but I believe they mismanaged the sources, and used externals, where they should have been using just binary dependencies). Yet my current team uses git, and straight on decided that branching is too much hassle.
What strategy do you use to decide when to permanently enable a feature and simplify the codebase?