

Using Puppet to Automatically Configure New EC2 Instances - abyx
http://www.codelord.net/2010/12/19/using-puppet-to-automatically-configure-new-ec2-instances/

======
Andys
I've recently done a project with puppet that turned out horribly. I'll share
what I discovered:

* Puppet's error messages are next to useless. Sometimes it can only tell you which file had an error, and not which line number. Sometimes you'll be left wondering where in the many files that make up a complex project there was an error.

* The only way to actually test the correctness of your puppet scripts is to deploy on real servers. Your staging environment has to be identical to production if you don't want any nasty surprises.

* Despite being written in ruby, the design doesn't follow ruby's philosophies or best practices. The config language is not dynamic or ruby-ish. Compared to ruby, you get code blowout when you can't use DRY and have to unroll loops to fit inside the bounds of what the config language can do.

* Simple things are hard. You have to write your own modules or use 3rd party modules for what I consider simple things, such as editing an existing system config file to add two lines in the middle if they don't already exist.

* There's a steep learning curve because a large project is like spaghetti code where you aren't quite sure which file to find what you're looking for. It is very easy to create dependency loops, and hard to find them, and even harder to resolve them. I ended up generating the dependency graph to a PNG file and eyeballing it, several times per day.

I got the impression that puppet is ideal for getting a "PHP+MYSQL" server up
and running that requires almost no deviation from the operating system
defaults. If your setup is a more complex one I'd stay well away.

------
fredoliveira
I for one, appreciate a pretty succinct article on setting up Puppet. Having
used Chef briefly in the past (and now getting ready to use Chef - or puppet -
across a brand new infrastructure setup), I came across the lack of (or just
bad) documentation for Chef many, many times.

~~~
sam
I've not used chef, but I'm in the process of setting up a new infrastructure
using puppet and I've found the documentation to be pretty good.

------
irons
I've never used puppet before, but in an EC2 context, I'm puzzled about why
one would want to use on-the-fly configuration instead of fully setting up
AMIs to serve specific roles.

At first blush, future-proofing against lower-level AMI changes and
maintaining instance role flexibility don't seem like a good tradeoff against
increased startup time in most cases.

~~~
abyx
We're just starting off with Puppet and EC2, but in general my current feeling
is that I'd rather postpone maintaining images as long as I can. This is
because a) I don't trust myself to keep these up-to-date, meaning I'll need
configuration once the instance is up b) The cost of making a change is tiny
with puppet. If you've got images you have to save them again after making the
changes. That extra step currently has no value for me, when setting up a
machine takes 2 minutes (which is fast enough for us)

------
sammcd
If anyone here could discuss the pros and cons of this compared to chef for
server configuration i would be very interested.

I often spend way more time than i expect to when configuring a new server,
and have recently been very interested in ways to speed that up.

~~~
spidaman
Initially, it won't speed you up. Developing the right provisioning formula
requires some investment of time but I think it's time you get back many times
over because you don't need to follow a bunch of manual steps and making
changes to the infrastructure is easier when it's all code. I prefer chef
simply because it's easy to develop customizations; the DSL is just ruby. I
have a pretty non-standard application stack but automating the provisioning
is pretty straight forward. I can bring up the whole thing from bare machine
images in minutes. Check out
[http://wiki.opscode.com/display/chef/Launch+Cloud+Instances+...](http://wiki.opscode.com/display/chef/Launch+Cloud+Instances+with+Knife)

I don't know if there's a puppet equivalent to chef-solo but using it with
vagrant and VirtualBox is my favorite way to develop chef recipes. I did short
presentation about this a few months ago, see
[https://docs.google.com/present/edit?id=0AVIsoyl05MRmZGNwY2N...](https://docs.google.com/present/edit?id=0AVIsoyl05MRmZGNwY2N6MzRfOGY2bmZnZnpy&hl=en)

BTW, the OP talks about needing a separate security group for the puppets but
last I checked, the default security group on EC2 makes the machines you own
visible to each other and nobody else's that you haven't explicitly permitted.
So I think all of that was unnecessary.

~~~
sam
The equivalent to chef-solo in puppet is "puppet apply". It's very handy for
developing puppet recipes on an offline virtual machine.

------
cowmixtoo
I'm going to sound like a broken record but I don't understand why bcfg2 isn't
more popular.

If anything, I'm going to 'port' this post to an bcfg2 example.

------
kljensen
This is super useful thanks. I'm mystified why puppet lacks substantive ec2
documentation on their website. I would have thought this was a major use
case.

~~~
WALoeIII
I wrote a pretty detailed piece on doing exactly this last summer, hopefully
soon iClassify will be replaced by Puppet Dashboard, but for now I run both.

<http://onehub.com/blog/posts/coordinating-the-onehub-cloud/>

------
swalberg
I like his way of doing the auto signing. I wrote about the topic at
[http://www.ibm.com/developerworks/linux/library/l-migrate2cl...](http://www.ibm.com/developerworks/linux/library/l-migrate2cloud-4/)
and just had the new instance run the manifest locally.

The other option is to have the instance sign its own key, much the same way I
have the instance set itself up in Cacti.

