
Shell Scripts vs. Ansible - mattjaynes
http://devopsu.com/blog/ansible-vs-shell-scripts/
======
agentultra
I still use shell scripts because I don't have the time or energy to invest
becoming a domain expert in installing packages.

When my shell scripts become untenable I happily hire someone who is an expert
at installing packages on a cluster of machines.

It's a complex problem and CMs so far are complex enough that I'm put off by
how dense the CM language is. Every time I come across chef my ears start
leaking precious brain fluid. Recipes, cookbooks, knives, bags, dependencies,
acronyms I haven't bothered googling for... there's enough there that someone
can dedicate their time to mastering it and be paid a decent salary.

No thanks, I'm already spending enough time as it is being an expert in
something else. I'll just stick with shell scripts because at least I can
understand those without much effort. If they become a problem I'll hire
someone to fix it. :)

 _update_ : That was a little rant-y... after reading this article I might try
Ansible if I get the chance. The problem I have with CMs is that they tend to
be very complex because they solve a complex problem. Shell scripts tend to be
simple (but crude by virtue). I am curious whether there is a middle-ground.

~~~
mpdehaan2
disclaimer: author of project here.

Achieving a middle ground was one of the points of creating the project --
making configuration tools more accessible to everybody, that you can forget
and come back to in six months and still know how to work with.

I definitely would be interested in your feedback after your trying things,
but I think that's exactly the point Matt was trying to make in the article.

~~~
agentultra
Well the proof of the pudding is in the tasting... I will have to give it a
whirl I suppose. Chef/puppet/etc seem easy on the surface when you're getting
your feet wet. I find that the complexity monster rears its head when you have
to start supporting multiple OSs and combinations of configuration options.
Shell scripts fail in this area too but it's generally much easier to find the
problem than a morass of metadata files, recipes, and specifications. I would
be more than happy to provide feedback (being the curmudgeon that I tend to
be).

I appreciate attempts at reducing complexity. I've been meaning to look into
Marelle a bit
([http://larsyencken.github.io/marelle/](http://larsyencken.github.io/marelle/))
as I find logic programming and CBS to be where CMs should be heading. Is
Ansible somewhere along that spectrum?

~~~
mpdehaan2
Ansible doesn't seek to be programming, so my guess is no. I generally am not
a fan of that syntax and would disagree that that is the way most
administrators and developers brains work -- but I guess I can see it for some
who do think that way.

Multiple OSes are pretty straightforward though. Look at the 'group_by' module
for dynamically carving up machines based on operating system, and things like
the 'when' statement for introducing conditional behavior.

Variables can also be overriden based on groups, so you can define defaults in
one group and then have different settings in other groups.

------
mattjaynes
This article will be preaching to the converted for many of you, but I still
see so many folks still doing manual installs and shell scripts even for large
companies that I felt I needed to try and convert a few more folks over to
saner systems.

Fortunately Ansible has dramatically lowered the learning curve and now is a
great time to make the switch. So, if you have friends that need an
"intervention" from their manual installs, hopefully this will help you make
your case to them :)

Of course there are many great config mgmt tools out there. I singled out
Ansible because it's so simple to get started with and I've found that the
learning curve is usually the biggest hurdle to adoption.

------
e1ven
I've used a lot of various configuration management systems over the years -
CFengine, BCFG2, Puppet and now Ansible.

Two notes-

1) They all work well for the base cases, and make it much easier to handle
hundreds of machines - Being able to say "Ensure libcurl is installed on
appserver, and make sure that nagios has these plugins we wrote" is amazing.

But where every system starts to break down, is when we have setups that I
have a good reason for, but don't fit into the preconceived notion of how to
handle things.

For example, if I want to install a package from source, or if I want to
manually wget a .deb from an internal server, then dpkg -i it.

What's nice is that most systems allow some flexibility - You can do 90% of
things in the pre-written modules, and break out to do manual commands where
necessary.

2) One thing I'd really like to see, though, is a system which operated with
even LESS intelligence.

Rather than using YAML files, or any sort of "user-friendly" config, let me
define variables and use pure Python to iterate and execute using them.

I'd love to have people give me pre-written modules as classes I can call, but
having that exposed in a way I can get at it with native loops, if/then, etc,
would make things a lot more straight forward.

    
    
        for server in webservers:
    
           server.ensure_perms('/var/lib/somefile',755)
    
           if server.name in qa_servers:
    
               server.ensure_line('/etc/profile','EXPORT RUBY_ENV=QA')
    

I know I can write modules to do this, but it'd be awesome to do away with the
concept of cookbooks, and let me just write everything in a more
straightforward language.

~~~
txutxu
Yesterday I was playing with ansible, testing to migrate my already made
across years scripts.

My vote for the one who puts here an elegant solution for something as simple
as:

    
    
        ssh user@host sed -i '/FOO/ s/FOO*/BAR/' /etc/bla/bla-*
    

And a raw: command: or shell: does not count as elegant solution for the vote.

The first parts is easy, you just have to learn a new DSL because ansible
ignores your .ssh/config.

The sed is easy to replace to with a lineinfile:

But... how can I iterate over the remote files?

I don't like my solution, is not elegant:

    
    
        - name: Get the list of remote files
          raw: printf '['; printf "'%s'," /etc/sysconfig/network-scripts/ifcfg-*; printf ']'
          register: p
        
        - name: Apply the change to the list of files
          lineinfile: dest={{ item }} regexp=^NETWORKING_IPV6 line='NETWORKING_IPV6=no'
          with_items: p.stdout
    

My impression in general, after my first weekend with ansible: great

But I see my old script to setup a daemon in a chroot, and I see "my" ansible
version... and I don't know what to tell (the ansible version just multiplies
the lines, in a much more verbose way, that finally looking at it, you have to
think and concentrate much more, than looking at my old and direct .sh)

At the end... this is not new:

    
    
        ssh WORD < SCRIPT
    

Where WORD comes from your .ssh/config definitions (the inventory) and SCRIPT
is up to you to define and organize properly.

But I wish users of my software where so happy first day as I'm with my
ansible experience.

~~~
txutxu
And other thing I've learn this weekend.

The documentation is great, but there is no "a reference", you have to read 5
or 6 different (long) pages to have an overview of what the DSL is, but the
only reference is for the modules, not for the core.

You can search for ansible playbooks in sites like github, and... even if you
will find a few which do what they say, you will find many "test" repos, and
you will find that the 95% of the available code (no from ansibleworks, but by
third parties) implement very poor deployments, without even following basic
best practices or any security in mind.

I (as final user) would like a "cpan" or "rubygems" or whatever of ansible
playbooks, more extended than the ansibleworks examples. Where people could
share, search, vote, download (ansible get?), etc ansible playbooks.

Just suggestions.

~~~
mpdehaan2
Thanks for feedback.

I would suggest looking at github.com/ansible/ansible-examples for a fair
amount of examples.

I think it's generally true of most systems that a lot of the community
implemented content is suited to the needs of someone who wrote it.

As for content sharing, we might be working on something :) Stay tuned!

------
kbenson
As someone who has written utilities to check for configuration changes
presented by packages systems when they update, glossing over this issue is a
bad idea.

Having a unified configuration that you can track and push is good, but if you
are installing from RPM or DEB, you want to know if the package is pushing a
new config itself, or providing a new config for review.

The utility I wrote looked for .rpmnew and .rpmsave save files and checked
with a central repo for whether they were whitelisted (by MD5) or not.
Anything not whitelisted resulted in mail being sent alerting admins. Along
with an fairly comprehensive per-host auto-update exclude list (which I got
yum-autoupdate patched upstream to allow), this allowed us to be fairly
confident in allowing our 100+ boxes to auto update every day.

Without paying close attention to packages release notes, interesting things
can come down from the maintainers, usually from them back-porting a fix from
a later version (or a feature, if it's a RHEL point release).

I think configuration management is a good thing, but I'm not sure it's the
best thing to be taught _first_ , unless there's mentors to work with. Some
things you just won't think of until it's bit you in the ass, and have that
bite be at small scale until you've figured it all out is probably a good
thing.

~~~
falcolas
To address just the package concern, it's worth noting that Ansible can pin
packages to a particular version, by naming the version you want installed.

Then, even if a new package is pushed to the repo, you will not receive it
(and will be notified by an error by subsequent runs if it's been manually
updated) until you update the inventory file.

Many things in Ansible can be pinned in similar ways.

[http://www.ansibleworks.com/docs/modules.html#yum](http://www.ansibleworks.com/docs/modules.html#yum)

~~~
mpdehaan2
I'd also highly recommend maintaining a local mirror, regardless of your CM
system.

Makes things faster and ensures even if you don't pin versions you are not
going to get upstream surprises.

------
dogas
A coworker and I needed to get an ElasticSearch server up using chef, and the
whole time we were complaining that we wished we could just use shell scripts.

So, we spent a friday and came up with
[http://fuckingshellscripts.org](http://fuckingshellscripts.org). At first it
was just a parody, but then it started to get real after we found a lot of
value in just using shell scripts for simple servers.

Granted, we now use ansible and it has been far easier to implement than chef!

~~~
chubot
Oh man I thought fuckingshellscripts.org was going to be like
giveupandusetables.com (which is now down).

I want to just use fucking shell scripts to configure servers. But it's a gem
too which I don't want to install.

------
bryanlarsen
I think your ansible example could be improved by removing all the names.

    
    
        - name: Ensure https support for apt is installed
          apt: pkg=apt-transport-https state=present
    

smells a little bit like:

    
    
        // add one to x
        x += 1;
    

Additionally, I would far rather see "apt: pkg=apt-transport-https
state=present" in my logs than "Ensure https support for apt is installed"

~~~
mattjaynes
That's a great point. I initially didn't have comments for any of those code
sections. However, most of my reviewers encouraged me to add the comments so
it would be clearer what each step was doing for beginners.

~~~
bryanlarsen
In that case, you can use YAML comments rather than ansible names. It makes it
much more clear that it's optional, and you get the actual code in the log
rather than the name/comment.

~~~
mpdehaan2
Ansible actually encourages the use of the names because they show up as task
headers.

These are especially valuable when managing a large number of hosts.

~~~
bryanlarsen
That's exactly why I actively discourage the use of names. I'd much rather see
"apt: pkg=foo" than "Ensure foo is installed" in my logs. Especially when I
have a large number of hosts.

------
sixbrx
This is a bit OT because the article is about configuring servers, but I
wonder if anyone has found CM systems like this helpful for maintaining their
desktop or laptop system configurations?

Every few upgrades of Ubuntu I seem to get into a situation that I want or
have to do a clean install, then waste time getting the new system back into a
workable state. It would have to be pretty lightweight and intuitive for my
purposes.

~~~
mattjaynes
Great question.

Github has a project called Boxen that does this for Mac OSX:
[http://boxen.github.com/](http://boxen.github.com/)

Google uses their own Puppet solution for their developer workstations:
[http://www.youtube.com/watch?v=YlKXRdSAZhY](http://www.youtube.com/watch?v=YlKXRdSAZhY)

Ansible will work great for this too - particularly if you're on Linux.

------
spindritf
> Which method supports easily templating your configuration files?

But you don't actually use templating. You just copy the file over.

What do you do when you have to edit the configuration file? When it needs to
be different on every machine for whatever reason (IPs to listen on,
hostnames)? What is the smart way to do that automatically?

~~~
falcolas
Templates and host variables.

Templates use the jinja2 templating engine, so variables are represented by
`{{ variable_name }}`, and you can put variables (as well as defaults) into
Ansible configs to set them to different values for different servers.

------
aktau
In my opinion, ansible only beats puppet here in ease of initial setup. I even
find the equivalent puppet script for this task to be a tad bit clearer. But
we're talking just syntax now. I only see now just how similar puppet and
ansible (and perhaps chef too) are. What with the ensure'ing and all.

Puppet is a bit difficult to setup (not too much, mind you, it does its own
key generating and being a PKI on first setup just fine, and you can run it
from its own internal webserver until you have to go all "webscale"), but
after that it hums along nicely. Haven't had too much complaining to do.

Except that they changed the string escaping rules somewhere in the 2.7.x
branch, which debian of course pushed because it's in the same point release.
That was fun.

~~~
eli
Maybe it's me, but I found Puppet very difficult to get started with. To the
point where I would happily use a less capable system that was easier to
learn.

~~~
__david__
That's interesting. I was setting up a server the other day and wanted to
document it in a repeatable way but, having used Chef and Anisble in the past
made me not want to go through the hassle of setting up a whole "thing"...

So I looked through the Puppet docs for about 5 minutes and found I could just
run "Puppet apply whatever.pp" on the server and got everything working in
about 15 minutes. And I'd never used it before in my life. So for now, I'm
quite impressed with the ease of Puppet.

Chef and Ansible both are really great for 80 to 90% of what I've needed to do
in the past. But that last 10-20% is _really_ painful. It requires huge
contortions, or poking through the source code so that you understand the
details of the system so thoroughly that you can write an new module, or abuse
the existing modules and bend them to your will. I suspect that, in the end,
Puppet will have the same sorts of limitations as the others. But it did have
a very nice way of setting things up that didn't require a whole
infrastructure.

~~~
mpdehaan2
Hmm, I haven't heard that feedback before. I'd be very interested in hearing
what the last 20% you found complicated was. Drop me an email at michael AT
ansibleworks DOT com if you like.

------
jrochkind1
This article doesn't do a very good job of stating it's thesis: The ansible
version is no fewer lines than the shell script version; but requires you to
understand ansible and how it works.

It doesn't seem preferable to me, from this example.

Note, I don't need to be convinced of the benefits of configuration management
in terms of reproducibility and reliability. I'm just saying the OP's examples
don't really make the case it suggests they make, of showing how ansible is so
much easier than shell scripts. It does not appear to be.

~~~
mpdehaan2
I think Matt did a pretty good job of showing how they were at about the same
level of complexity for a new audience, but one of the things you don't see is
that

(A) there's error handling,

(B) the system will only run commands as needed

and of course using things like 'with_items' and 'notify' ends up making it a
lot shorter.

But yeah, most of Hacker News already has a read on that :)

------
bcoates
Does ansible have some sort of REPL so I can use it for both manual
administration and automated? I use shellscripts for a lot of stuff because
they're just bash one-offs with seniority.

~~~
bryogenic
There is an unofficial one: [https://github.com/dominis/ansible-
shell](https://github.com/dominis/ansible-shell)

------
zimbatm
The important concept here is Reentrancy [1] and versioned code.

If the configuration is small enough it's easier to write a script in bash
that approximates the re-entrancy well enough instead of having to bootstrap
the configuration manager, distribute the description files and execute it. In
fact a bash library could go a long way covering the scope of these tools.

[1]:
[https://en.wikipedia.org/wiki/Reentrancy_%28computing%29](https://en.wikipedia.org/wiki/Reentrancy_%28computing%29)

------
mratzloff
Can anyone discuss their experience with Ansible versus Chef + knife-solo?

I found Chef clumsy and hard to set up on its own (its central server
requirement seems like it was designed just to support a business model), but
with knife-solo it's extremely usable.

~~~
dalore
I've used both, it's similar but a main difference is that you kick ansible
off from another (say your dev machine) which then ssh's in and does it's
stuff, whilst knife-solo starts from the host.

~~~
pjungwir
> knife-solo starts from the host.

Huh? You run `knife solo cook example.com` on your local machine. It then sshs
in and runs what it needs to. (Granted, it runs everything via chef-solo, but
the user experience is running a command locally.)

~~~
dalore
Oh I used it where the host boots up and then it it runs solo on the local
machine. So in your example it's similar.

------
hbbio
What seems to be missing is _updating_ the nginx.conf with sed or something.

In the article, you're just copying a new magic configuration from somewhere
vs. "proper" sed use (in reality, it can be a bit more than that) in the shell
script.

~~~
mattjaynes
Initially I was using Ansible's `lineinfile` module to do exactly that. It's
pretty trivial to do if you want to compare true Apples to Apples. Here's an
article with some examples: [http://fabian-affolter.ch/blog/the-lineinfile-
module-of-ansi...](http://fabian-affolter.ch/blog/the-lineinfile-module-of-
ansible/)

However, I wanted to show the actual process I went through. Editing the file
in place seemed silly to do with Ansible, so I went with putting the config
file in version control and copying it over. That also had the benefit of
showing how Ansible encourages better practices than shell scripts.

Of course, that's totally subjective, so for some folks that use shell
scripts, they may already know to put their config files in version control.
However, in practice, I've not seen that be the case.

~~~
tfinch
Also worth noting, ansible comes with a 'template' module which allows you to
generate custom files with variables.

------
mylons
The title of this article is really tilting. Does anyone defend shell scripts
anymore?

