
ShutIt – A Python-based shell automation framework - zwischenzug
https://learnxinyminutes.com/docs/shutit/
======
zwischenzug
Source is here:

[https://github.com/ianmiell/shutit](https://github.com/ianmiell/shutit)

More background here:

[https://ianmiell.github.io/shutit/](https://ianmiell.github.io/shutit/)

This is used, for example, to automate kubernetes cluster tests for Chef
scripts for OpenShift:

Chef scripts:

[https://github.com/IshentRas/cookbook-
openshift3](https://github.com/IshentRas/cookbook-openshift3)

[https://github.com/ianmiell/shutit-openshift-
cluster](https://github.com/ianmiell/shutit-openshift-cluster)

[https://zwischenzugs.wordpress.com/2017/03/04/a-complete-
ope...](https://zwischenzugs.wordpress.com/2017/03/04/a-complete-openshift-
cluster-on-vagrant-step-by-step/)

~~~
zwischenzug
Oh, and if you have anything you want automated, let me know - I love an
automation challenge :)

~~~
samstave
Contact info please

~~~
zwischenzug
@ianmiell

------
ivan_ah
Seems like another version of `Fabric`, though the expect functionality is
nice. After investing so much in learning the quirks and specifics of the fab
command, I'd need a very compelling reason to switch.

~~~
tyingq
There are at least some cases where the expect functionality is either
required, or makes things much easier.

I know expect, or expect-like functionality is used often for automating
things across groups of Cisco devices, for example. Where you need to process
the output of one command to drive another, or where commands have a separate
"commit" Y/N type prompt.

Outside of that, most of what this describes feels like something I would do
with Ansible.

~~~
philipov
Sounds great for pipelining data load jobs.

~~~
zwischenzug
Message me if you have a use case @ianmiell or on GH.

------
dvfjsdhgfv
Suggestion to the author: try to use portalocker [1] so that the module can
also work in Windows. Currently it doesn't because the code uses the fcntl
module that is *nix-only. [1]
[https://pypi.python.org/pypi/portalocker](https://pypi.python.org/pypi/portalocker)

~~~
zwischenzug
Thanks - did you try it in windows? I use CygWin at work, but it would be
great to use it natively also.

~~~
dvfjsdhgfv
Yes, it failed to import because of fcntl. Since you can't just replace fcntl
with an equivalent win32api call, I suggest using portalocker. I usually log
on *nix servers from a Windows desktop, so ShutIt could be an useful addition
to my toolbox - once it works on Windows, that is.

------
falsedan
This looks like a provisioning framework rather than 'shell automation'
(scripting). I couldn't work out out if it supports pipelines.

In all honesty, who is this aimed at? Some who know python and doesn't want to
switch to bash, but is happy to learn the command-line tools you use in bash?

> _ShutIt was built originally to facilitate the deployment of complex Docker
> containers, so that developers can quickly prototype builds in a structured
> and flexible way with as shallow a learning curve as possible._

It's for setting up developer environments? Like a sandbox? Like docker-
compose?

~~~
gvalkov
It's a nice API around pexpect [1] and therefor fills the same niche -
spawning and interacting with console applications in a programmable way.

[1]:
[https://pexpect.readthedocs.io/en/stable/](https://pexpect.readthedocs.io/en/stable/)

~~~
falsedan
I'm familiar with expect, but I don't have any modems anymore which require AT
commands to be sent to it with precise timing.

It seems like you are describing what it is, not what it should be used for
(which is what I can't figure out, based on my knowledge of other tools).

~~~
SwellJoe
expect was used for a lot more than modem interactions. Certainly, there are
many ways to skin this cat today, but it's not crazy to want something that
removes the boilerplate of capturing output from commands and figuring out how
to proceed to the next step based on what it gets.

To answer your question with specifics, you could use something like this to:

\- Start services, and then do something (maybe alert, maybe make a change and
try again) if it failed.

\- Make configuration changes across several files and check the results of
the changes (i.e. did things continue to work after the change).

\- Check for updates in version control and then optionally deploy them based
on the results (e.g. if there's a new tagged version).

\- Interact with some API periodically, and do something if results change.

\- Tons of things I don't do, but somebody else might. It's kind of an IFTTT
(If This Then That) for the command line (which is maybe obvious, since I
guess IFTTT was an implementation of expect for the web, in the long tradition
of "take a UNIX utility and make a version for the web"). It isn't for any one
thing; it's for whatever thing you want to programmatically do with stuff that
maybe wasn't meant to be programmatically worked with (or maybe just doesn't
have a Python API).

All of this can be done with any reasonably competent programming language,
including Python (I use Perl with some tiny custom libs or bash with
boilerplate, usually). This removes a bit of code. Maybe only a little code is
removed, maybe a lot. It depends on what you're doing, and how well the
commands or APIs you're calling lend themselves to being called from a shell
or Python or whatever.

Expect was a tool to make working with tools that maybe didn't intend to be
called from shell (or Tcl, or C, for that matter) programs work better when
called from programs. There have always been many ways to do it. If this
doesn't fit the way you think or the tools you use don't need it, there may be
better ways for you to accomplish similar tasks. But, it's not a crazy idea.

~~~
falsedan
I know what expect is (see my grandparent comment). I still don't know what
this framework is to be used for:

> _Start services, and then do something (maybe alert, maybe make a change and
> try again) if it failed._

It's a replacement for upstart/systemd/Launch Control?

> _Make configuration changes across several files and check the results of
> the changes (i.e. did things continue to work after the change)._

Puppet/Chef/Ansible + Sensu/Nagios?

> _Check for updates in version control and then optionally deploy them based
> on the results (e.g. if there 's a new tagged version)._

cron?

> _Interact with some API periodically, and do something if results change._

Also cron?

~~~
wybiral
I have several IoT devices around my home and want to push different code or
configs out to them. Sure, something like Chef could work... But I'd have to
install a client on each device when all I really want to do is script SSH
sessions.

~~~
falsedan
Great, that really helps me visualise a use case where an expect framework
would shine.

------
wybiral
Looks interesting. I'm going to give it a spin on some RPi3s I have hanging
around and see how it compares to having used fabric.

One thing: the link "learnshutit.html" at the top of this page is a 404.

------
halfnibble
This is incredible. The best part is that it's incredibly simple. Excellent
job documenting via example.

