Puppet and Chef are yet another thing to learn and maintain, if the guy is a part-time admin with a lot of other responsibilities and a small number of servers it may not be worth it.
I was in the same position - too many distinct environments for bash/Fabric, too little time to learn Chef/Puppet/CFEngine. Ansible [1] seems like a good compromise: you get the simplicity (runs over SSH) and host targeting of Fabric with the declarative nature and idempotency of the more complex tools. You can start with all-in-one "playbooks" [2], then split out tasks, handlers, Jinja2 templates, files, and variables [3].
As an aside, I think the default fail2ban config is too loose and quiet. Here's [4] an Ansible task file that configures it to be more aggressive and send notification emails.
I'm using fabric quite frequently, and am trying to understand what makes a configuration management tools a much better choice. I'm currently using fabric for anything from bootstrap a new environment from scratch, via restoring a snapshot from backups, to pushing code updates stored on git. Perhaps I'm being really daft, but it always evades me why something as simple as
I do use a pretty homogeneous environment, which makes things simpler, but this is a deliberate choice to avoid complexity. If I know all my hosts are e.g. debian 6, then what makes ansible/chef/puppet so much better than fabric?
I'm not trying to be provocative or negative. I'm really trying to understand the supposedly big difference between what's labelled a deployment tool, and configuration management tools.
Deployment tools like fabric are imperative, and configuration management tools are declarative. With configuration management, you define the final state you want the server to be in, and it will do whatever is needed to get it into that state. Some or all of the parts might already be done, and it won't change the parts that are already correct (the declarative configuration is idempotent).
Deployment tools just execute whatever script you hand them, and so the scripts are either more brittle (server must be in a precise state beforehand or it doesn't work right), or require more effort to duplicate the work that the configuration management software does to only make the needed changes.
If your deployment scripts get complex, it's more difficult to see at a glance what the end configuration is supposed to be.
Thanks for the clarification. I think I get the theory a bit better now.
I'm still struggling with seeing some practical examples of those differences.
for example, `apt-get` is pretty much idempotent already, no? if I run `apt-get install -y <package>` x 1000 times in a loop it won't install it 1000 times...
Fabric does give you the building blocks for those kind of checks elsewhere, such as `exists`, `contains`, `append` (which is supposed to also be idempotent) etc... I've designed my deployment / bootstrap scripts with fabric to take this into account. It does add a little overhead, but nothing that makes me feel I need a better tool... Maybe my deployment base is still relatively small and homogeneous.
So it's true that I have to put in those checks myself, and that I don't have a very easy way to discover what state a server is at.
I'll try to take another stab at one of those tools. Maybe things will sink in when I actually use them. Thanks again for explaining.
When you install the package you do it as a on-off. When a configuration system does it then it will keep re-happening if anything changes.
Say you install "less", and I remove it. With Chef this will be fixed the next run - with fabric? You'll need to re-run your magic script.
That's the main difference. Automation like Chef/CFEngine/Slaughter will keep fixing breakages - as defined by differences in state from their rules - whereas you'll be playing a game of catchup.
If you're only doing one or two things, the value is a bit more vague. But consider even the simplest interaction: the config file for a service should look like X, and if it has to be changed, the service needs to be restarted afterward. Oh and there are 10 config files, but you only want to restart the service once if any of them changes, after they've all been changed. That's not hard, but it's already starting to look non-trivial.
And what if you want to have the same logic for several services? I guess you abstract it out to a function. But then it turns out one of those services doesn't have a restart command, and you have to do stop+start. And another service won't start if you use restart while it's not running, so you have to check if it's stopped and use start, otherwise use restart.
It's much more than just whether the code is declarative or imperative, or whether it's idempotent or not. An imperative tool can change your system from known initial state A to desired state B. A declarative system can change it from whatever initial state it's in to desired state B, even if you never considered it might be in that state.
When you want to make a configuration change to your servers, and they aren't already in a "known state", why would you think any tool could put them into a known state? When a computer has a virus, say, you don't think it's been put into a "known state" after the anti-virus program gets done with it; the virus may have done any number of things you might be unaware of, altered any number of data or configuration files in subtle ways that the tool doesn't look for, but implicitly relies upon.
Example: what if, say, one of the provisioning requirements is "make sure this gem is installed", but on one of the servers /etc/gemrc has "install: --no-rdoc --no-ri" in it? Now on one server, the docs are missing, while everywhere else they're available. That sort of thing.
If you're already running on an IaaS like EC2, I think there's a simpler, better way: rather than trying to get "unknown state" to "known state", why not use the simplest known state of all--unprovisioned? Write an imperative script that reinitializes a freshly-provisioned IaaS node to your known state, and then do a rolling reprovision, terminating old nodes and provisioning new ones.
An equivalent comparison[1]: would you feel safe running an automated script that would SSH into a production machine and "git pull" a checked-out repo sitting on it, from whatever state it happens to be sitting at, up to refs/heads/master, so as to deploy code from it? Could you guarantee that that repo hadn't been moved to some state where refs/heads/master isn't a fast-forward commit? Or would you rather do a fresh "git clone"?
---
[1] A contrived comparison, though; although the former option is unreliable, the latter is terribly inefficient, and they're both horrible for keeping a .git directory inside of a directory that might very well be web-accessible.
apt-get is most definitively not idempotent without a lot of care. It will upgrade packages if a new version is available, and such upgrades can have widespread implications.
Otherwise I agree with you - Puppet/ Chef etc. are overly big on ceremony for me. We "only" use about 100 VM's on about 20 physical servers at the moment, so it's a "simple" setup.
I think where people get the most value out of systems like Puppet and Chef is if a lot of their servers or VM's have simple enough requirements that they can largely use "off the shelf" community provided configurations. The more customisation you need, the more the leve of ceremony starts biting you. The more you can rely on ready-made scripts, the more those apps help you because of the size of their communities.
I'm not that familiar with Fabric, so maybe it can do some of the things I mention here out of the box without a lot of custom code. I'm only familiar with Capistrano and I associate them in my head, perhaps naively.
Your example is great to illustrate why configuration management is so powerful.
Most importantly, your example will upgrade the package to the latest version every time you run it (which is probably not what you intended, but maybe it is). I'm sure there is an idempotent command for each of apt, yum, ports, homebrew, etc, but I have better things to do with my time than figure that all out.
In puppet your command would be:
package { 'foo': ensure => installed } # or 'latest'
What if you also have RedHat servers? Now you need to figure out the right yum command to use to perform the equivalent. Or your developers Macs? Homebrew. Chef/Puppet are aware of the environment they run in and install appropriate packages using the appropriate package management script with minimal alteration to the underlying recipe.
For RedHat systems, it would be:
package {'foo' : ensure => installed }
On a Mac, it would be:
package {'foo' : ensure => installed }
On that Arch Linux machine that your devs are running, it would be:
package {'foo' : ensure => installed }
Now, let's throw some dynamism into the mix...
What if you want to build an haproxy configuration file that pulls the list of hosts from an authoritative source? Do you "just" build it up from scratch every time and replace the file on every run of your script? With chef/puppet, you can pull that information in from your node classifier using the same query for dev, staging, and production and have the same recipe work for multiple environments. In some environments, the haproxy config has now changed, so I want to tell haproxy to reload. But in other environments, the config stayed the same, so I don't want it to reload. The file and service don't get touched if they didn't change.
What if a service has multiple files associated with it that when one or multiple of them change, you have to restart or reload the service? You don't want to restart each service every time one of the multiple files changes, so you end up having to build a queue to handle the service notifications.
Say I've got a few dozen users on my hosts and someone leaves. Which is easier - writing a purge script which knows how to purge the user from all environments that you support, or this...
user { 'joe' :
...
ensure => absent
...
}
I'm certain that that you can do these things with fabric scripts, but by the time you have, you've either written your own configuration management tool (which is unlikely to be as robust as chef or puppet) or you have a hodgepodge of unmaintainable scripts.
Eh, I didn't like Ansible. Every command is done over SSH, so you don't need an agent/ssh to box and run locally, but... scripts would take much longer to execute than in puppet.
We're currently using local runs of puppet, and it's easy to do dry-runs and see what fails before applying when testing. I'm not sure how Ansible gives feedback on that (I assume it must) but I find puppet pretty good about it.
Puppet and Chef are yet another thing to learn and maintain, if the guy is a part-time admin with a lot of other responsibilities and a small number of servers it may not be worth it.
Speaking from experience as a part time admin, it is definitely worth it to automate the process, even with one or two servers. Any build system will repay in spades the first time you have to set up a new server or rebuild an old one, esp. if you're in a hurry after a server failure.
That said Linode offers other easier options (Linode specific of course). You can take the commands usually executed, and put them in a shell script in the language of your choice which is uploaded and executed on first run (they call this StackScripts - there are lots of examples and helper scripts on their site, but the scripts are really very basic). It's perhaps easier than learning a DSL as it's just codifying the same commands you would run via ssh by hand. You can also clone nodes so you could set one up in a known good state and clone that each time.
This does tie you to linode, so long term one of the other options available would be worth looking in to.
You don't have to re-learn puppet/chef/cfengine/bash script every time. You learn it once. It requires much less effort to "maintain" than manually updating more than 1 box.
I think being a part-time admin with other responsibilities is a perfect reason to learn something like Chef or Puppet. It'll end up saving you a lot of time that you can devote to other things.
I think the author meant for this to be a simple set of security guidelines for beginners. But yeah, any reasonably large prod stack should probably be deployed and setup with puppet and what not. It's DRY and less error prone than a human low on sleep.
Without Chef there would be no way for me to rollout a new server in our cluster. Investing time into Chef was one of the greatest things I ever did. Chef is the best documentation of our infrastructure. The second best tool I'm using is fpm[1] to make custom debian packages.
While not directly related to this article, last I checked Puppet/Chef don't have particularly useful windows support. Is there an chef like solution for windows?
You seriously do this by hand for every server? That seems error prone and a huge waste of time when tools like puppet and chef exist.