I never understood the desire to make things declarative. It seemed to me to always hide what is actually happening and it made it more difficult to understand. Is there a simple way to understand why declarative stuff is desirable to some people?
It lets you separate "here is the final state of the system that I want" from "how to get there".
If a SQL compiler or `terraform plan` command can convert "the current state of the system" + "desired end-state" to a series of steps that constitute "how to get there from here", then I can usually just move forward to declaring more desired states after that, or debugging something else, etc. Let the computer do the routine calculations.
When using a path-finding / route-finding tool, having the map and some basic pathfinding algorithms already programmed in means we no longer need to "pop a candidate route-segment off the list of candidates and evaluate the new route cost"... I simply observe that I am "probably here" and I wish to get to "there"; propose a route and if it's good enough I'll instruct the machine to do that.
If I can declare that I want the final system to contain only the folder "/stuff/config.yaml" with permissions 700 -- I don't care what the contents of stuff were previously, and if it had a million temp files in it from an install going sideways or the wrong permissions or a thousand nested folders in it, well, it would be great if the silly computer had a branching workflow that detected and fixed that for me, rather than me having to write yet another one-off script to clean up yet another silly mis-configured system that Bob left as a dumping ground that I have to write yet more brittle bizarre-situation-handling code for.
Same for SQL and data. "Look, Mr. Database, I don't actually know what's in the table today, and I don't know why the previous user dumped a million unrelated rows in the table.... Can you answer my query about if my package has shipped, or not?"
It sounds like you need a shim between every single dependency to make its setup work declaratively. That sounds like a versioning nightmare to rely on and was borne out in my experience with it.
I think the main reason people like the declarative approach is that done right, it's idempotent. You also don't have to think about the current state of the system at all. You just need to describe what you want it to look like. Of course in practice it can be more nuanced than that, but thinking declaratively can make things much simpler in some scenarios.
Ansible playbooks are usually a list of steps to execute in order (imperative). Those steps may try to present a declarative interface to what they are supposed to do, but many fail to fulfill the definition of "declarative" that you have linked. E.g. with the built-in modules it is impossible to declare a desired set of installed packages, only a set of packages to be installed _in addition to all already installed packages_. This means it is impossible to remove an installed package again by removing it from the declaration, you have to specify a second step (imperative) that explicitly removes the package. This makes it impossible to declare a final state for "installed packages" with ansible.
This is debating state-management though, which Ansible makes the correct choice about: Ansible largely works the way a user expects when they transition to it from doing things on the command line, and guides them towards idempotency (which is a pre-requisite for declarative configuration).
The problem is to track deletions you either have to constantly have a view of global state (i.e. do you want to put `linux-kernel` in your package list?) or you need to store specific state about that machine (i.e. `redis` was installed by playbook redis-server.yml, task "install redis") - because the packages absence in that list doesn't necessarily mean "uninstall it" if something else in another playbook or task will later declare it should be present.
As soon as you're trying to do deletions, you're making assumptions that the view of the state you have is complete and total and that is usually not the case - and even if it is within the scope of your system, is it the case on the system you're interacting with? Do you know every package that should be installed because it comes out of the box in the distro? Do you want to (aka: do you have the time, resourcing and effort to do this for the almost zero gain it will get you in the short term unless you can point to business outcomes which are fulfilled by the activity?)
> As soon as you're trying to do deletions, you're making assumptions that the view of the state you have is complete and total and that is usually not the case
terraform does this, which is why it tracks the its own representation of the prior global state. So when you remove a declared resource the diff against the prior state is interpreted as a delete. Note this does introduce the problem of "drift" when you have resources that are not captured in the scope of the state.
> i.e. do you want to put `linux-kernel` in your package list?
Yes. At least I want to put something like "core-packages" or "default" or similar as part of setting my explicit intent.
Yes, this is debating state management. For full declarativity some form of state management for the parts of the system that should be under declarative control (like terraform) or a stateless but very holistic view of the system (like NixOS, I guess also Guix System) are needed.
Given that ansible has neither it can't be much better then what it is. I disagree that that is the right choice though. As it is I see not much more value in ansible than in some sort of SSH over xargs contraption combined with a list of servers. The guarantees they give are the same.
> Do you know every package that should be installed because it comes out of the box in the distro? Do you want to [...]?
No, I don't want to. Thankfully, with NixOS I don't need to, since the pre-installed packages are automatically part of the declared state of my NixOS systems (i.e. I declare the wanted state in the same way in which the defaults are also declared, which makes it easy to merge both).
> Ansible playbooks are usually a list of steps to execute in order (imperative)
You can't be declarative all the way down because reality is not declarative.
You can have all modules being declarative but if you need orchestration, it's not declarative anymore unless you create a new abstraction on top of it.
So people keep arguing about declarative vs imperative and fail to specify at which abstraction level they want things to be either.
I agree with you, your declarative abstraction has to have an imperative implementation underneath that will do all the dirty work. Ansible presents this declarative interface at the module level (if the module is implemented properly, most aren't), and a playbook is an imperative list of declarations to be applied. Roles also combine a list of imperative steps into a declarative interface.
Since apparently (I try to avoid ansible, so I might be missing something) playbooks are the go-to approach of using ansible this means that most uses of ansible are imperative (in the context of configuring a system), unless you only ever give a system a singular role and then you are probably defining your role in imperative steps.
A system like NixOS on the other hand presents the entirety of a system configuration in a single declarative interface that is applied in one go, while applying such a configuration to a system can be a thought of as an imperative step (although it is usually a singular, unconditional step). So it is declarative at a higher abstraction level.
It’s the cattle not pets mindset. In most organizations the sysadmin team is really undersized. Not uncommon to have one admin per several hundred systems. In such places, there is no time to care for individual servers. If a server is misbehaving we blow it away and spin up a clean replacement.
Declarative scripts make it easy to manage a fleet.
I think that's.. perhaps not orthogonal, but has some orthogonal component - you could certainly have something like:
for i in range(100):
ip = cidrhost(subnet, i)
if exists := get_server(ip):
continue
create_server(ip=ip)
and so on. I don't like it, but because it's procedural/imperative, not because it's particularly more 'petty' than the Terraform (or equivalent) would be.
For me it's more about what I'm doing, conceptually. I want a server to exist, it to have access to this S3 bucket, etc. - the logic of how to interface with the APIs to make that happen, to manage their lifecycle and check current state etc. isn't what I'm thinking about. (In Terraform terms, that belongs in the provider.) When I write the above I'm just thinking I want 100 servers, so:
resource "cloud_server" "my_servers" {
count = 100
ip = cidrhost(subnet, count.index)
# and so on
}
This fails completely even at small scale when the script is interrupted before finishing.
The difference between just using some Python vs Terraform is idempotency. TF isn’t going to touch the nodes the script succeeded on; if you have to start your for-loop script it will, which may not be desirable.
Frankly these days configuration management is a bit dated…
You’re much better off in most cases using a tool like Packer with whatever system you want to bake an image, then use a simple user-data script for customization.
It’s very hard to scale continuous config management to thousands of servers.
Eh, any way you do it could leave it in an unfinished state if interrupted, I'm not too bothered about that. (But it does sound like you think I was speaking in favour of doing it in a procedural python script sort of way? I was not.)
Packer and Terraform do different jobs (they're both by Hashicorp!) - you can bake an immutable image all you like, you still need to get a server, put the image on it, give it that S3 bucket it needs, IAM, etc.
They work together to produce immutable cattle. The alternative is managing a pool of servers where you are doing things like in-place patch upgrades, vs a teardown of the old infra and replacing it with the newly baked servers.
Declarative structure is at the heart of functional programming. Declarative is not the right choice everywhere, but when it makes sense, it can significantly raise the quality of the code.
Because what most people want is actually something closer to "Goal Seeking." If the system works as intended (and as you point out with the need to debug is often does not!) then defining the desired end-state and letting the system figure out how to get there, is a simpler, higher order abstraction. And it can also often be clearer to just say "ensure these prerequisites are met" such that alternative implementations can achieve the same outcome. In practice, abstractions are leaky.
It makes people feel that they're smarter than you. See also functional programming. That said sometimes it's useful as s way to auto generate imperative actions.
I really can't see how you could feel that way about it after spending even just a few minutes (which it sounds like you have) to understand what it means beyond just reacting to terminology, something having a name.
I definitely think it'd be easier to explain a python-like declarative language to someone who asks what programming is than actual python. 'It's just describing the way things should be' vs. 'it's like a series of instructions for how to compute ...'
Certainly not more clever IMO, if anything the opposite. Like I said above or elsewhere in this thread, when I'm managing infrastructure with Terraform I don't want to (and don't have to) be thinking about how to interface with the API, check whether things exist already, their current state, how to move from that to what I want, etc. I just know the way I want it being, I declare that, and the procedure for figuring it out and making it so is the provider's job. That's not smarter! The smart's in the provider! (But ok if you're going to make me flex, I've written and contributed to providers too... But that's Go; not declarative.)
