
Why Puppet Should Manage Your Infrastructure - twampss
http://www.engineyard.com/blog/2011/why-puppet-should-manage-your-infrastructure
======
donw
I have one big gripe about Puppet:

Puppet requires both a puppet server (Rails) and client, SSL key exchange,
firewall rules for the puppet server, proper DNS records for everything, and a
host of dependencies, all of which you need to set up before you can actually
do anything.

Any system management solution that requires anything more than a bare machine
with SSH and sudo is, in my book, not terribly practical, because these are
the lowest common denominator on what you'll get from any hosting provider or
OS install.

In a nutshell: I shouldn't need to configure my servers before I configure my
servers.

~~~
dan_manges
You can use Puppet without doing all of that. At Braintree, we use Capistrano
to upload a tarball of our Puppet scripts and run the puppet command.

sudo puppet --templatedir $HOME/puppet/templates --factpath $HOME/puppet/facts
puppet/puppet.pp

~~~
mechanical_fish
Let me endorse both of the above posts. Having now spent many months tinkering
with Puppet, I wish to god I had fully understood the value of _not_ using any
of Puppet's server features but had instead just shipped the manifests via
some other method and applied them locally.

If you need a super-performant version of that strategy (which you probably
don't) try googling up Twitter's "murder" project. (Not as violent as it
sounds! It's the crow kind of murder.)

~~~
donw
Until I read this thread, I didn't even think of that as an option --
everything in the Puppet documentation, and the community, and in what they
offer as far as training courses, indicates that Puppet's client-and-server
are the light and the way to rightness.

I'll have to take a second look at the Puppet manifest system coupled with
Capistrano (which I really like), but it still irks me that there isn't one
tool to handle configuration management across a large number of servers.

------
DougBarth
We've been using Puppet to manage our servers for some time now. As a group of
developers doing our own operations work, we've found puppet both good and
bad.

Setting up puppet was relatively straight forward. We had the puppetd auto-
updating our servers for a while, but ultimately decided to manually run it
when deploying changes. Managing zero-downtime changes was more error prone
with it running.

Some aspects of Puppet have over time proved frustrating to us. The top
annoyance is we never quite figured out a good way to test our puppet changes
before checking them into git to deploy them to our puppetmaster. That has
lead to a number of "fixing errors" type commits. The second annoyance we've
found is actually highlighted as a feature: no implicit ordering of
operations. While it might sound great to be able to reorganize your configs
without fear of breaking the deployment, we've found that the tradeoff is that
you don't find out that your configuration doesn't define its dependencies
correctly until you try to kick a new server after spending months
incrementally adding to your existing servers. For us, at least, having an
implicit top to bottom ordering would lessen that headache.

Despite some of these headaches, simply having our configuration in version
control is a huge win for us. We can setup a box much more easily, and we have
a comment trail of why changes were made.

~~~
Goladus
About testing puppet changes...

If I had to do it again I would probably ditch the puppetmaster altogether and
use an rsync server to distribute the entire configure repository to every
server, then run puppet locally to apply changes. This way you can simply
modify any local repository and run puppet to apply the desired configuration
to any machine you want. When you're happy with the changes you can check them
in.

Using the puppetmaster and the puppet fileserver was trickier, essentially I
would use FACTER_var="value" to pass in a value to puppet that would use local
files rather than central files (which came pretty close to the purely
decentralized model anyway).

------
nwmcsween
You know what I can't even express the amount of dislike I have for puppet
from variables that have 4 purposes (who ever thought up :ensure that can be
"latest", a version, a requirement of being available or not or if a service
is running needs to be shot buried and encased in cement) to the DSL that
tries to be declarative yet puppet isn't and allows for half installs due to
failures. Chef isn't any better as it's extremely opinionated - AMQP have to
use it, deprecated merb have to use it. Cfengine is in it's own world of suck
(ever write unportable scripts with no abstractions? well you do now). I'm not
being snarky, I gave each a fair shot while evaluating them by implementing a
provider for a distribution.

------
legooolas
There have been a lot of suggestions (on here and other sites) to run puppet
with locally-rsync-ed (rsunk?) copies of manifests, but there are a few things
which won't work if you do this, unfortunately. Most importantly is the
`storedconfigs' which (afaict) require the puppet server to work.

This means that you lose a large amount of the power of using puppet, by which
you can use configurations across machines to do things like collect up all
the services you run on a set of machines and generate a nagios config, or
firewall config, or whatever. Without using stored configs for this I assume
it's possible but will require more explicit configuration rather than the
rather more elegant solution provided when using a puppet server.

Side note : I've used puppet on a fairly small scale of up to ~50 machines,
and just started using it for VMs, and it's pretty straightforward to
integrate into a bootstrap install to get ruby and puppet installed so that
you can use it to install all the rest of the dependencies. But of course, the
most use is for changes later on rather than at install-time when there are
already a huge number of tools to set up or image machines or whatever.

Side side note : I've not used Chef to compare this with.

------
mtrn
I'm setting up a single server, and even there, puppet and chef come in very
handy. I can reuse the recipes on a local vagrant-managed virtual box OS and
test both the server configuration and the deployment.

At the moment I like chef-solo a bit better (because it uses an internal dsl).

Just because I beginning puppet standalone and chef-solo - are there some
longer term experiences, pitfalls, etc, you can share?

~~~
asenchi2
I'll write up more about Chef later, but I really look at the two differently.
Puppet is really great at managing infrastructure and server state. Chef is
really good at integrating with your application (especially if you are using
Ruby). I typically think of Chef as a framework to program your infrastructure
against. Puppet is more of the middle manager. :)

Both are easy to test, with Puppet winning slightly with 'puppet apply
<manifest>'. Chef-solo is nice but takes a little bit more to setup (solo.rb
and node.json for example). Either way, test and see what you think will work
best for you.

------
iuguy
Puppet is one of those things I like in principle but is too much of a PITA to
set up. Take for example the class definitions, the class definitions don't
appear to offer a great deal more than a shell script, but in the example
shown in TFA for the price of about 6 lines of puppet code we could've just
run rsync -e ssh -avz ntpd.conf puppet@server:/etc/ntpd.conf && chown
root:root /etc/ntpd.conf && chmod 644 /etc/ntpd.conf.

Of course in the real world you'd have a tarball you'd rsync over, then use
SSH to extract and run the base script and robert's your father's brother. A
lot simpler and the way I'd automated Solaris admin years ago. Puppet's
drawback is that it doesn't offer anything sufficiently compelling for people
to change from what they use, and presents an awful lot of work in it's syntax
for people getting started. Once it's up and running it's brilliant, I've seen
it. But it just seems like so much hard work to get there it's like a barrier
to entry.

~~~
Goladus
The original example is actually kind of bad and doesn't demonstrate puppet's
abstraction facilities.

You can define a custom resource, for example "system_file" that provides a
default "root:root, mask 444" permissions such that you just have to define a
source and destination for every file, overriding the default permissions when
you want.

define system_file($mode = 444, $owner = root, $group = root, $content = '',
$source = '', $ensure = 'present') {

    
    
            file { $name:
                    owner => $owner,
                    group => $group,
                    mode => $mode,
                    ensure => $ensure
            }
    
            if $source != '' { File[$name] { source => $source } }
            if $content != '' { File[$name] { content => $content } }
    
     }
    

So in the article's example, it would look like this:

    
    
            system_file { "/etc/ntp.conf":
                source => "puppet:///ntpd/ntp.conf",
                require => Package["ntp"]
            }
    

Or even

    
    
            system_file { "/etc/ntp.conf":
                source => "$configfiles/ntpd/ntp.conf",
                require => Package["ntp"]
            }
    

Where $configfiles might be the puppet server or some other location. One of
the things you get with puppet is access to any of the local host properties
that can be discovered with facter, so you can dynamically configure something
like a source file.

~~~
uggedal
Or you could override the default file resource attributes in sites.pp:

    
    
      File {
        owner => root,
        group => root,
        mode => 644,
      }
    

Then you could override that again in individual file resources. If you need
several file resources with the same attributes you can override them within
the scope of a class:

    
    
      class postgresql {
        File {
          owner => "postgres",
          group => "postgres",
          mode => 640,
        }
      
        $pgsql_root = "/etc/postgresql/8.4/main"
        file { "$pgsql_root/postgresql.conf":
          content => template("postgresql/postgresql.conf.erb"),
          require => Package["postgresql"],
        }
      
        file { "$pgsql_root/pg_hba.conf":
          content => template("postgresql/pg_hba.conf.erb"),
          require => Package["postgresql"],
        }
      }

------
forsaken
The real point that should be made about Chef and Puppet is that they are so
similar, it really doesn't matter which you use. Using one or the other is a
much more important choice than which one you use.

I don't know the best way to express this sentiment (feels like there should
be a word for it). But really, just use something to automate your
infrastructure and your life will be measurably better.

------
DanielBMarkham
It's been extremely interesting to watch these meta server tools evolve. We're
reaching the point where there's not too much of a difference between a
scripted network graph and a suite of VMs and cloning abilities. Each
technique would have it's advantages, though. Perhaps somebody with large
scale infrastructure experience could do a side-by-side comparison?

------
jollojou
We configure our production servers and push new releases there with Puppet. I
like Puppet: its fail-safe and reliable.

There is, however, one thing I don't fancy in it. Puppet does not support
insecure client–master communication. Requiring SSL communication is OK, but
one should be able to switch it off if it brings no value.

We are running our our servers on AWS, and we rely solely on AWS security
groups to grant and deny accesses. Puppet's SSL traffic brings no additional
security to us; it only complicates matters. For example: we would like to
shut down the Puppet master EC2 instances when they are not needed. However,
this is not possible, since after start-up the EC2 instances have new IPs, and
this breaks the Puppet-signed SSL certificates.

~~~
legooolas
> after startup the EC2 instances have new IPs

They have static ("Elastic") IP addresses:

[http://aws.amazon.com/articles/1346?_encoding=UTF8&jiveR...](http://aws.amazon.com/articles/1346?_encoding=UTF8&jiveRedirect=1)

..don't these let you keep the same IP for your Puppet master instance?

~~~
jollojou
That's correct. The problem is that the EC2 internal IPs change even though
the instance would have an Elastic IP. EC2 instances use internal IPs when
communicating with other EC2 instances (this is a feature of AWS DNS). As a
consequence, Puppet clients cannot access the master by using the master's
Elastic IP.

~~~
legooolas
That sounds rather inconvenient. Are the external elastic IPs non-routable
internally? I mean, if you add the IPs explicitly to hosts files then will
traffic to those IPs not work?

------
jarin
If you want a dead simple way to use Puppet with Rails, check out Moonshine. I
did an overview of it at SD Ruby:

[http://jarinheit.posterous.com/a-talk-i-gave-at-sd-ruby-
depl...](http://jarinheit.posterous.com/a-talk-i-gave-at-sd-ruby-deploying-
rails-apps)

------
nimrody
A small question:

Assuming you use a single type and version of the OS (say ubuntu vXXX) -- Does
it make sense to use the OS native packaging system instead of something like
Puppet?

I.e., maintain a private packages repository where you add your custom
packages, and have the various servers pull from that repository?

Obviously, this doesn't work if you have different types of servers - but for
many servers configured identically, it should work.

~~~
Goladus
Of course, for many different servers configured identically, you could use
systemimager instead, which is OS-agnostic.

------
ashayh
The Foreman project takes care of managing a whole bunch of puppet related
things for you:

<http://theforeman.org/projects/foreman/wiki/Features>

