I co-founded Terrateam[0] which started out as GitOps for Terraform but has expanded to OpenTofu, Pulumi, and now "GitOpsify anything (that has a plan/apply-like workflow)". An example is feature flagging, you can manage your feature flags via a GitOps workflow.
The number one nuance we run into is that GitOps is not application code. We are trying to cram this different workflow into a similar looking one, and if you squint hard enough, it fits, but it is actually different. I think that, on net, GitOps is the best default choice for managing infrastructure, the important difference is that infrastructure is "physical". Only one instance of your feature flag configuration will exist for your production environment at any point in time. Your dev environment will never actually be the same thing because, like apartments in an apartment complex, even if every one is stamped out of the same mold, the fact that they physically exist will always mean they are slightly different. So, unlike application code, where you can build an artifact, push it to a dev environment, run some tests on it, then take that same artifact and push it to prod and know they will do the same thing given the same context, infrastructure is never like that. "testing" infrastructure in a dev environment will only give you a signal that the same change will do what you want in prod, but never proof-positive.
How does all that come into play in practice?
1. A branching strategy like GitFlow dooes not make much sense for things being managed under GitOps. There is nothing to "tag". There will never be multiple versions out in the wild. There is only ever one version and when you apply the next version, the old one is gone. Maybe you want to remember what a state looked like in the past, but you can never go back to it, you can only go forward.
2. Some users want to do a branch-per-environment workflow. So when there feature branch is being merged to "dev", the changes go to the "dev" environment, and then when that branch is merged to "prod", those same changes go to "prod" environment. But this doesn't really work either. We can do this in application code because we are just getting artifacts and the artifact is the same whether it's prod or dev. What you end up doing in branch-per-environment workflows is your change looks good going into "dev" then it doesn't look good going "dev" to "prod", so now you have to modify "dev" in place because the "feature branch" to "dev" workflow isn't giving you the feedback you need to understand if your change reflects what you want. Again, the "dev" environment will never be the "prod" environment because these things are physically different.
3. There can be some aggressive discussion around GitOps workflows because, on one hand, it feels like coding like application code. We even call things like Terraform, Tofu, and Pulumi "Infrastructure as Code". But going back to my original point, this is not like application code so we have to be thoughtful about if the workflow we want is actually implementing meaningful feedback. A simple example is that applying a change in GitOps might fail for no reason related to the code. It could be there was a network blip while performing the apply, it could be that there was an API limit hit, it could be that some data just takes an undefined amount of time to replicate through the system so a dependency fails. All this means that you might need to plan and apply again without making any actual code change. This, alone, makes the GitOps workflow different than an application code workflow. In Terrateam we support this through a number of ways, but the main path is users commenting on the pull request to initiate an operation again.
I said that I think GitOps is a net-positive in the end, despite these nuances, and the reason is that I think the differences are not so subtle that we can't address them and guide users via automation. Terrateam tries to say "hey, <this> happened, if X, do Y, if A do B" so when users have to break out of the illusion they don't have to think too hard. Being able to push operations into a workflow that is like development reduces the amount of new knowledge a users needs, which I think is the real value of GitOps. You are already using VCS, using pull requests, you have an approval process, GitOps is just injecting a bit more information there, and I think that is very valuable to a huge number of teams.
Is GitOps perfect for every scenario? No. Certainly there is some infrastructure that is not API friendly, and unfortunately that just is what it is. Also, if your main job, day-to-day, is managing a piece of infrastructure via a bespoke interface that adds value to the experience, then certainly that is the correct choice for you. And, finally, a GitOps workflow just might not work for your team. Maybe you aren't trying to emulate a developer experience. That's fine. Do what works for you, just make sure it is working for you.
The number one nuance we run into is that GitOps is not application code. We are trying to cram this different workflow into a similar looking one, and if you squint hard enough, it fits, but it is actually different. I think that, on net, GitOps is the best default choice for managing infrastructure, the important difference is that infrastructure is "physical". Only one instance of your feature flag configuration will exist for your production environment at any point in time. Your dev environment will never actually be the same thing because, like apartments in an apartment complex, even if every one is stamped out of the same mold, the fact that they physically exist will always mean they are slightly different. So, unlike application code, where you can build an artifact, push it to a dev environment, run some tests on it, then take that same artifact and push it to prod and know they will do the same thing given the same context, infrastructure is never like that. "testing" infrastructure in a dev environment will only give you a signal that the same change will do what you want in prod, but never proof-positive.
How does all that come into play in practice?
1. A branching strategy like GitFlow dooes not make much sense for things being managed under GitOps. There is nothing to "tag". There will never be multiple versions out in the wild. There is only ever one version and when you apply the next version, the old one is gone. Maybe you want to remember what a state looked like in the past, but you can never go back to it, you can only go forward.
2. Some users want to do a branch-per-environment workflow. So when there feature branch is being merged to "dev", the changes go to the "dev" environment, and then when that branch is merged to "prod", those same changes go to "prod" environment. But this doesn't really work either. We can do this in application code because we are just getting artifacts and the artifact is the same whether it's prod or dev. What you end up doing in branch-per-environment workflows is your change looks good going into "dev" then it doesn't look good going "dev" to "prod", so now you have to modify "dev" in place because the "feature branch" to "dev" workflow isn't giving you the feedback you need to understand if your change reflects what you want. Again, the "dev" environment will never be the "prod" environment because these things are physically different.
3. There can be some aggressive discussion around GitOps workflows because, on one hand, it feels like coding like application code. We even call things like Terraform, Tofu, and Pulumi "Infrastructure as Code". But going back to my original point, this is not like application code so we have to be thoughtful about if the workflow we want is actually implementing meaningful feedback. A simple example is that applying a change in GitOps might fail for no reason related to the code. It could be there was a network blip while performing the apply, it could be that there was an API limit hit, it could be that some data just takes an undefined amount of time to replicate through the system so a dependency fails. All this means that you might need to plan and apply again without making any actual code change. This, alone, makes the GitOps workflow different than an application code workflow. In Terrateam we support this through a number of ways, but the main path is users commenting on the pull request to initiate an operation again.
I said that I think GitOps is a net-positive in the end, despite these nuances, and the reason is that I think the differences are not so subtle that we can't address them and guide users via automation. Terrateam tries to say "hey, <this> happened, if X, do Y, if A do B" so when users have to break out of the illusion they don't have to think too hard. Being able to push operations into a workflow that is like development reduces the amount of new knowledge a users needs, which I think is the real value of GitOps. You are already using VCS, using pull requests, you have an approval process, GitOps is just injecting a bit more information there, and I think that is very valuable to a huge number of teams.
Is GitOps perfect for every scenario? No. Certainly there is some infrastructure that is not API friendly, and unfortunately that just is what it is. Also, if your main job, day-to-day, is managing a piece of infrastructure via a bespoke interface that adds value to the experience, then certainly that is the correct choice for you. And, finally, a GitOps workflow just might not work for your team. Maybe you aren't trying to emulate a developer experience. That's fine. Do what works for you, just make sure it is working for you.
[0] https://github.com/terrateamio/terrateam