
ShutIt – Automation framework for programmers - indatawetrust
http://ianmiell.github.io/shutit/
======
zwischenzug
Author here. Quite surprised to see this.

Another related tool I'm working on is:

[https://github.com/ianmiell/shutitfile](https://github.com/ianmiell/shutitfile)

Why this over Ansible? It's a long story :)

Mainly because not everyone is a great or even decent programmer, so I wanted
an easy way in for them for the situation I was in. Hence automation for
everyone. But I don't think this is a serious competitor for Ansible.

Happy to answer questions as they arise.

~~~
themckman
> Why this over Ansible? It's a long story :)

As someone who just spent the last year of his life using Ansible, something
tells me I already know this story. I feel for you.

~~~
Rafert
As someone who hasn't done much with it yet, care to elaborate? It seemed
easier for me to grok than Chef.

~~~
Azeralthefallen
My life resolves around Ansible and has for the last year and a half. Ansible
is a great tool, unfortunately it has a bunch of short comings. Especially now
some of these are becoming a major issue.

\- Its dependence on Python 2.X, unfortunately all the newer OS's have Python
3.X installed by default. While i have a role for installing this manually it
isn't super nice.

\- My main devops project consists of 11 different environments, and it
manages nearly 2k servers. Trying to write environment agnostic roles is an
absolute nightmare. Furthermore my group_vars, host_vars, and dynamic
inventories have turned into spaghetti. Why? Because of how confusing variable
precedence becomes, when you get into extremely complex hierarchy of
group_vars which can randomly break.

\- It is really hard to handle cases where playbooks/roles fail and deal with
it cleanly. Blocks have helped a bit, but there is no way to grab the
exception that caused the playbook to fail.

\- The yaml syntax becomes messy fast, Jinja2 is powerful but there are times
i wish you could just shove a single line of python into places to prevent the
huge mess of with_items, register, set_fact, etc.

\- AWS modules are a very mixed bag. A lot of them are missing features, and
there is some very weird issues that apparently boto is the cause of (e.g.
they should move to boto3). Even then i need to do a huge amount of
boilerplate to launch ec2 instances.

\- There is no way to share a common group_vars between inventories (all is
specific to inventory). Which leads to a lot of duplication. (Yes the symlink
trick does work, but causes a bunch of other issues).

I still love Ansible and i would pick it over Chef/Puppet. Salt is also
another decent one that has made a lot of progress lately.

~~~
chucky_z
Regarding variable precedence, it's pretty simple to grok in 2.x+, and well-
documented now (it _was_ a serious nightmare, and a very valid complaint...
but isn't as much anymore).

For sharing common group_vars, what I do is my inventory passes an
environment-aware variable which I include in the playbook by doing like:

`- include: "env_vars/{{ environment }}/main.yml"`

I feel I sidestepped some of the AWS modules issues simply by using Terraform.
I still build all my images in Packer using the Ansible builder though.

~~~
Azeralthefallen
Unfortunately the above way will load the vars at a higher precedence then
group_vars. Also if you have multiple groups with the same var but differing
values you will need to start loading multiple files (or you can do |default,
and use an alias to the value in group_vars (which leads to even more
spaghetti)).

------
emilong
Just curious how you decide the line between what is handled by ShutIt and
what's handled in the shell?

For example, from the example on the home page:

    
    
      # Ensure git is installed. This handles different distros gracefully.
      shutit.install('git')
    

Awesome and very powerful abstraction! Vs the next lines...

    
    
      # If the directory does not exist, we create it
      if not shutit.file_exists('/opt/shutit',directory=True):
      	shutit.send('mkdir /opt/shutit')
    

Why not just:

    
    
      shutit.send('mkdir -p /opt/shutit')
    

There are a few other examples where I was wondering how you decided to do
some operations in ShutIt vs delegating to shell. In fact, some entire
examples look simpler to me in shell that you could just send. Does using the
ShutIt native commands make things more testable?

~~~
zwischenzug
That's just an illustration, not 'ideal' code. Of course mkdir -p serves the
same purpose; the point is to show the 'file_exists' function.

Mostly it's where using the shell would be painful. Do you have other specific
examples?

~~~
mrb
I am a firm believer that example code that serves as a reference should
illustrate the absolute best practices. "mkdir -p" is the best practice to
create a dir that may or may not already exist, therefore this is a bad
example (you are teaching some devs who don't know "mkdir -p" a bad practice).
If you want to demonstrate the file_exists function, pick a better example.

This is the same reason why I dislike, for example, seeing the documentation
of a library showing how to perform actions X, Y, and Z, but a footnote says
"by the way, there is no error checking in this example". Well then, why use
this code as an example?

Anyway, sorry for the rant. ShutIt does seem interesting :)

~~~
zwischenzug
I know what you mean, but I couldn't think of a better example that would not
have involved explaining something else that would have distracted from the
point. I wasn't comfortable doing it, but at the same time, couldn't think of
anything better.

Actually, I have similar problems when teaching - I'm writing a git course at
the moment, and it's hard to illustrate concepts in a realistic way without
hopelessly artificial repos.

~~~
rickycook
there's another file_exists for the git directory that seems to be an even
more complete example :)

------
throwanem
For the working sysadmin or devops specialist, what does this offer over e.g.
Ansible?

~~~
collyw
I wish this sort of question was asked a lot more often when we see something
"new" on HN.

~~~
dreamsofdragons
And I wish people like you kept their mouths shut. Seriously. Never ever give
someone shit for creating.

~~~
throwanem
Who, aside from you, is giving anyone shit here? I'm not saying ShutIt
shouldn't exist; I'm asking what, if anything, makes learning it a worthwhile
use of my time. Should the answer to that question be "nothing", it's still no
judgment on ShutIt's existence; it's just a piece of information useful to
people who calibrate their expenditure of attention perhaps differently from
the way in which you do.

~~~
dreamsofdragons
I apologize if I misread your comment.

~~~
throwanem
No apology necessary; it's entirely reasonable to have read my original
comment in the fashion you did. I've edited it to clarify my intent.

------
leetrout
There's also a somewhat similar tool (much more bare bones) called fabric.

[http://www.fabfile.org/](http://www.fabfile.org/)

I've used Fabric a lot and it's "cousin" invoke. I wish I'd seen fabric listed
in the requirements.txt because I feel like it's got a nice API and has been
around a while and is probably a bit more battle tested...

~~~
zwischenzug
That is quite similar. Looks like it's a bit less general-purpose than ShutIt
though.

~~~
gingerlime
As a fabric user, I'm curious what makes it less general-purpose than ShutIt ?
(or trying to understand the differences between them)

~~~
zwischenzug
Looks ssh-based - ShutIt can be applied to various delivery methods (there are
4 at the moment - bash, ssh, docker and dockerfiles).

------
Phithagoras
code at
[https://github.com/ianmiell/shutit](https://github.com/ianmiell/shutit)

