
Ansible Alternatives in Python - alexeiz
http://blog.rfox.eu/en/Explorations/Trying_Ansible_alternatives_in_python.html
======
bamboozled
How hard is YAML to use really? The world has much more interesting problems
to solve than a decent markup language.

I use Ansible with zero issues daily as do many in our team. It’s perplexing
coders fined it difficult or hard to use to the point where something as
useful as Ansible becomes undesirable because it uses YAML?

I hate to say it, but the “alternatives” are just the least good somewhat
similar products.

~~~
thr0w3345
The problem we have with ansible is not yaml related but the push based
approach, even with awx/tower as soon as you’re running hundreds of machines
of a given group it gets painful.

We are actually seriously considering a shift to salt atm...

~~~
jdboyd
The push versus pull difference between ansible versus salt is a worthwhile
design choice. In my opinion, salt does start to make sense until about when
you would outgrow ansible. That said having to replace all the playbooks and.
Roles written in ansible.when you outgrow it is a bummer. I don't want ansible
to move away from push, but it would be nice if there was a layer to let it
work both ways.

~~~
oddly
Or you can use Ansible-pull.

[https://docs.ansible.com/ansible/latest/cli/ansible-
pull.htm...](https://docs.ansible.com/ansible/latest/cli/ansible-pull.html)

~~~
em-bee
likewise salt also has a way to push via salt-ssh which works just like
ansible without a salt-minion.

so at this point both ansible and salt can go both directions, and i do infact
myself make use of that. i use pull/minions on all servers in our network, and
push to those servers outside our network, because they can't reach the master
behind the firewall.

although zerotier has helped solve that problem, so i'll probably be able to
stop using salt-ssh soon

------
fermigier
I have tried or used most of the projects cited in the blog post. I pretty
much agree with the conclusion that Pyinfra looks the most promising at this
point. I've also tried Bundlewrap
([https://bundlewrap.org/](https://bundlewrap.org/)) a couple of years ago
though I lack the perspective to compare them. Another interesting project not
cited in the post is Nuka
([https://doc.bearstech.com/nuka/](https://doc.bearstech.com/nuka/)).

Some personal perspectives:

I have used Ansible in the past and didn't like it.

I have used Fabric and Fabtools many years ago (Fabtools happens to be written
by a friend) and I agree that something like Fabtools is needed to make Fabric
relevant. Fabric has a completely revamped version, which was quite long in
the making, but not fairly stable, but AFAIK Fabric 2 doesn't (yet) have
something similar to what Fabtools was to Fabric 1.

I had great hope for opsmop
([https://github.com/opsmop/opsmop](https://github.com/opsmop/opsmop)) by
Ansible's creator, which was supposed to address Ansible's shortcomings, but
he dropped the project (citing lack of traction) a few months after starting
it in 2018.

At this point I'm sticking to Fabric 2, but I'm still looking for
alternatives.

I'm also wondering if Augeas ([https://augeas.net/](https://augeas.net/) \- a
configuration editing tool which parses configuration files for a large set of
software in their native formats and transforms them into a tree) could be
used in combination with one the of tools mentioned in the post to ease the
parts related to managing configuration.

~~~
Bystroushaak
Thanks for the links. I am kinda surprised by the activity on this blogpost,
it's just something that I've put together in an hour or so, mostly as notes
for myself.

I do "exporations" where I put bunch of related stuff into the wiki node and
then take an evening to go over them and do some evaluation. It wasn't really
meant as a comprehensive study.

------
INTPenis
The author of Ansible actually tried to make a Python3 DSL based Ansible
alternative called opsmop.[1]

I'm not sure why but it was discontinued. I would think that if anyone could
make that work it would be that particular person with their previous
experience from Cobbler and Ansible.

I started using Ansible when it was brand spanking new and what I've noticed
over the years is that people often over complicate their Ansible use.
Sometimes it's better to out source something to an RPM or a container than to
use Ansible to do every single thing.

1\. [https://github.com/opsmop/opsmop](https://github.com/opsmop/opsmop)

~~~
jabl
Here's an explanation by the sensible/opsmop creator:
[https://medium.com/@michaeldehaan/revisiting-systems-
managem...](https://medium.com/@michaeldehaan/revisiting-systems-management-
assumptions-5eee2d5fa82e)

~~~
INTPenis
Thanks, I just skimmed it and found this.

>Nobody wants to keep up with a configuration management tool, especially
learn another new one (which is why creating opsmop failed), learn another
YAML dialect, or maintain a full active cloud abstraction layer running on top
of their cloud itself.

------
chopraaa
If you feel YAML is too complicated for you, a host of problems await for
anything you want to do in infrastructure...

Ansible has it's own set of features that are largely abstracted out by it's
YAML playbook syntax. You can run it across any (or almost) system, from Linux
to Windows and even networking devices. You can create modules in any language
you want.

For speeding up Ansible runs, a number of optimisations can be made including
using Mitogen
[https://mitogen.networkgenomics.com/ansible_detailed.html](https://mitogen.networkgenomics.com/ansible_detailed.html)

Alternatives listed in the blog post don't even come close to the speed and
power of Ansible.

~~~
sa46
> Ansible has its own set of features that are largely abstracted out by its
> YAML playbook syntax.

I don't understand what YAML has to do with abstraction. The example from
pyinfra is pretty much the same as the ansible equivalent. Whether the feature
set is comparable is a different question (I agree with you, it's probably
nowhere close to ansible).

    
    
        apt.packages(
            {'Install iftop'},
            'iftop',
            sudo=True,
            update=True,
        )
    

> If you feel YAML is too complicated for you

Ansible YAML is a poor substitute for a real language once you add conditional
logic to the mix. Another huge downside is that composition in YAML is much
harder than in any programming language. A Python (or dhall, or cue) library
that compiled to ansible would be pretty swell.

~~~
kitd
_Ansible YAML is a poor substitute for a real language once you add
conditional logic to the mix. Another huge downside is that composition in
YAML is much harder than in any programming language_

Given that the average devops person is already stuck with yaml, many of those
problems are addressed by jsonnet. But I agree, having it in the language
would be preferable. I think it stems from the desire to make infrastructure
declarative, which isn't really the case when your declarations are dynamic.

~~~
dynamite-ready
I've written a lot of Ansible in the past year. While I'm happy with the work,
I'd admit to looking at some of the logic I've had to put together with
clunky, esoteric constructs like `with_nested_items` and the super colloquial
variable system, while thinking "it's great to be able to do this stuff, but
htf am I going to explain this code to a newbie?".

I'd like some kind of 'sensible', backwards compatible layer in Ansible that
would allow me to rewrite stuff in raw Python, with 'normal' code, where it
makes sense, but keep YAML for constructs that do actually look and work like
lists of basic tasks.

~~~
delsarto
Stuff that gets too complex you can pretty easily abstract into a role and
jump into Python.

[https://opendev.org/zuul/zuul-
jobs/src/branch/master/roles/t...](https://opendev.org/zuul/zuul-
jobs/src/branch/master/roles/tox/library/tox_install_sibling_packages.py) is a
nice example of some pretty complex co-install of libraries for testing under
tox (and you can unit test bits of it too...).

[https://zuul-ci.org/docs/zuul-jobs/](https://zuul-ci.org/docs/zuul-jobs/) is
a collection of roles and playbooks that do a lot of complicated things with
nice abstractions such as this. Although they're designed to be called from
the Zuul CI system, there's many good examples for anyone wanting to build out
their Ansible in a scalable and maintainable way.

------
stevenjohns
Probably worth checking out Mitogen[0] if you want to take away some of the
pain points with Ansible. It’s an extremely well built product that really
complements Ansible.

[0]
[https://mitogen.networkgenomics.com/](https://mitogen.networkgenomics.com/)

~~~
Bystroushaak
I've seen Mitogen, but that to me looks like lowlevel ssh paralelization
library. Great project, but it doesn't seem to provide any abstractions, for
example declarative package manager wrapper, that would work on multiple linux
versions.

------
BossingAround
Kubernetes already requires YAML. If you use Kubernetes, you might as well use
Ansible.

How YAML got so popular is a mystery to me. It's a huge pain. Then again, I've
given up on fighting it, and simply learned how to use it, since I can't
escape Kubernetes anyways.

~~~
lazyant
> How YAML got so popular is a mystery to me.

It's good at being human-readable. It's basically json with indents and
comments. What other configuration format is out there that allows for tree
structure and it's easier on humans? (not XML or json).

~~~
randallsquared
I'd say YAML's good at _seeming_ human-readable...

JSON has some surprises and a lot of limitations, but is so much simpler than
YAML that it seems worth putting up with them.

------
ShepherdKing
There is no mention of Python Bundlewrap
([https://bundlewrap.org/](https://bundlewrap.org/)), which I have really
liked, though I don't know how speed and scalability compares. I believe
Bundlewrap will welcome contributors for these enhancements.

------
syllogism
Fabric was a casualty of the Python 2/3 split. Unicode isn't a great default
for the shell interaction so the project took a long time to be ported, and by
then it lost all momentum.

The successor project, Invoke, was finally released a year or two ago, but now
it's unmaintained too.

~~~
acdha
I think it’s more that Jeff Forcier is one – very productive – person and
while tons of places used Fabric there were far fewer who contributed
significantly to supporting it.

I much prefer Python to programming in a YAML derivative with multiple bolt-on
extension mechanisms but at the end of the day Ansible brings a lot to the
table because there are a bunch of people employed by Red Hat to work on it
full-time.

~~~
syllogism
Yes that's true, but turning it around: the task is too big (and also sucks
too much) for one person to do mostly alone. And also like, why?

There would've been a more active community of companies depending on it and
it wouldn't have blocked out on him if it had maintained the momentum it had.

~~~
acdha
I was talking about it at the peak of that momentum. This is a common problem
in open source: companies often don’t contribute or only support specific
things they need. It’s much harder to get general ongoing contributions even
if that would make the entire community better off long-term.

------
theonemind
"In Python" seems like a weird constraint on the problem. Python doesn't
really read like a great deployment configuration language.

[https://en.wikipedia.org/wiki/Infrastructure_as_code#Communi...](https://en.wikipedia.org/wiki/Infrastructure_as_code#Community_content)
the space has a lot of players. For upgrading like one VPS and some home
servers, I'd probably go with Ansible. It probably has the lowest barrier to
entry, and getting an IDE just solves the problem that you created by wanting
something in Python. Basic config management doesn't need an IDE.

~~~
divbzero
“In Python” does seem like an odd constraint. I take the OP’s intent to be
“let’s find something imperative not declarative”. If that’s the case, I’d
love to see the idempotent functionality of Ansible’s modules ( _e.g._
lineinfile) made available as simple CLI commands.

~~~
inshadows
I assume you consider Ansible to be "declarative". By that definition of
"declarative", the following Bash script is also "declarative":

    
    
        #!/bin/bash
        rm -f /etc/bar
        mkdir -p /etc/conf.d
        cp -u /srv/src.txt /etc/conf.d/
    

because Anisble playbook is really just bunch of following statements

    
    
        if (!stuff_exists(stuff))
            create_stuff()
    

which the above utilities in Bash script do with the provided flags.

Ansible "declarativeness" isn't anything worthy of worship. It's plain dumb
definition of sequence of steps to take with added guard for each step.

So let's be honest and stop pretending that this "declarativeness" is anything
novel or of any value worthy of writing sequence of (guarded) commands in ad-
hoc YAML language. Just add some sugar for the pattern above to a decent
language and that's all that's needed.

------
skywhopper
This is fascinating to me, as someone who manages and Ansible-based config
management infra that keeps thousands of servers running. The problem I have
with Ansible isn’t the YAML, it’s the Python. I spend at least half my time
struggling with reconciling which Python runtime and modules are installed on
which hosts. And between the Python 2 EOL, the growth of ARM hosts in our
inventory, a fuzzy and changing line between what’s “Ansible” or “Python” or
“Jinja” in the syntax, and regular OS upgrades, it’s a lot more work than it
ought to be. And at scale there are some really frustrating performance issues
that aren’t entirely Python’s fault but which would be much better if we could
use native code. I’ve been hoping for years that someone would build a Golang
tool that could take Ansible playbooks and do the right thing with minimal
tweaking, but I think we’re beyond the point where anyone is going to invest
that much time and effort into new CM tooling.

~~~
stevekemp
I suspect if anybody is going to write something new it won't benefit so much
by being constrained by Ansible's YAML parser, etc.

I toyed with writing a simple thing, inspired by Fabric, and that was
moderately useful:

[https://github.com/skx/deployr](https://github.com/skx/deployr)

Later I tried again, with something more puppet-like, to experiment with how
you might handle dependencies in a simple and consistent fashion, ideally
without using a complete language (i.e. puppet/ruby):

[https://github.com/skx/marionette/](https://github.com/skx/marionette/)

The problem with the simpler approach is that they do only basic things, to
support "everything" means writing a hell of a lot of glue, and making a lot
of busywork until it is remotely useful to others.

Puppet/CFengine/Chef/Salt/Ansible sometimes seem like they're a dead-end with
how many things are moving to master /golden images (packer, etc), and
containers.

------
dmsimard
It's healthy to look at alternatives, good on OP.

I wanted to comment on a specific part that mentions that Ansible's YAML is a
nightmare to debug.

I wouldn't describe it like that but I'm biased because I've written ARA
Records Ansible exactly to make playbooks easier to understand and
troubleshoot: [https://github.com/ansible-
community/ara](https://github.com/ansible-community/ara)

I've been using Ansible for many years in different contexts and I think it is
easy to get into the trap of "when you have a hammer, everything looks like a
nail".

For me, Ansible shines as an excellent abstraction layer for supporting
different operating systems or as a glue between different tools.

I am very much a believer of using the right tool for the job. Ansible isn't
always the answer and I think that's OK.

~~~
justaj
So when is Ansible not the answer?

------
mongol
I think there would be value in providing idempotent operations to all common
Linux tools. This often exist, but sometimes with weird side effects.
Sometimes an add operation returns a failure code because item already exist
and so on. It would be quite worthwhile to go through the Ansible
documentation and document a command line alternative for each. Then it would
be easier to use a regular programming language to achieve the same effects.

------
torvald
If one is not tied to python, NixOps is a breath of fresh air.

------
RNCTX
I don't see anyone else mentioning that the selling point of using YAML in
Ansible is "human readable."

I see no practical benefit to requiring devops people to use one scripting
language over another, when there are so many equivalent choices to accomplish
the same tasks.

~~~
acdha
This isn’t a simple binary flag - there’s an inflection point for most
projects where YAML starts easier to read for simple tasks but you end up
having to do things with loops, value manipulation, etc. which would be much
easier to read in normal Python.

~~~
RNCTX
Well, that presumes the person is a regular user of Python. Maybe they write
bash scripts instead. Maybe they're using a BSD and write bourne shell scripts
and hate bash. Maybe they're Windows servers using powershell.

As someone else stated the beauty of Ansible is that it abstracts away the
most common functions so that the scripts are portable, and the person
maintaining the scripts doesn't have to keep up with every syntax change in
version if they keep Ansible updated.

Unless you're going to convince individual open source project maintainers to
agree to and convert to a somewhat rigid style guide for command line flags,
API syntax, and configuration syntax (which no one has managed to do, hence
the existence of distros, containers, and Ansible), you're gonna have a hard
time convincing anyone to give up on markup languages in deployment tools, I
think.

~~~
acdha
My point was that language complexity needs to be there once you get over a
certain level of project complexity, which many projects hit. Once you need to
do more than the most basic operations you’re learning something which is
going to be better off learning a regular programming language which can take
you much further rather than learning at least two separate syntaxes (Python
and Jinja2) plus all of YAML’s quirks and duplicate syntax structure, and
almost all programming tools have better IDE support, linting, and validation
tools.

If you want a document language, TOML is a better choice (or JSON or even non-
enterprise XML). If you want to write programs, anything else is a better
choice. Hashicorp’s HCL is interesting in that it’s much better designed and
supported but even there the main thing which makes it work is the presence of
an escape hatch when you hit a limitation.

------
dmarinus
What I like most from Ansible is that a playbook is idempotent and (mostly)
declarational. Biggest downside is that it's hard to debug (no realtime
output) and a bit slow (but acceptable)

------
linsomniac
I wasn't aware of the VSCode Ansible plugin, I've got to check that out.

Pyinfra examples he shows don't look that different from Ansible YAML.
("apt('nginx', update=True, present=True)")

I'm pretty happy with Ansible, with one exception: speed. I may play with
Pyinfra, but I'm heavily invested in Ansible. But if Pyinfra is fast, I might
try replacing our Fabric with it. I get pushback about how long Ansible takes
for our deploys vs. Fabric.

~~~
larsnystrom
So I looked at the VSCode Ansible plugin and it says it is authored by
"Microsoft"[1], but the repository is owned by a Github organization called
"Visual Studio China"[2] and the plugin identifier is "vscoss.vscode-ansible",
when other Microsoft plugins usually has identifiers in the "ms-vscode"
namespace (for example the TSLint and C/C++ extensions) or "ms-vscode-remote"
or some similar namespace starting with "ms".

So why is this ansible plugin such an outlier? How can I be sure that this
plugin is actually authored by Microsoft and not by some other entity?

Maybe I'm just being paranoid.

Edit: Ok so if you click the name "Microsoft" in the vscode marketplace it
will take you to a list of Microsoft authored plugins, which I guess means the
organization name in the marketplace is verified somehow. So yes, I was just
being paranoid.

[1]
[https://marketplace.visualstudio.com/items?itemName=vscoss.v...](https://marketplace.visualstudio.com/items?itemName=vscoss.vscode-
ansible)

[2] [https://github.com/VSChina/vscode-
ansible](https://github.com/VSChina/vscode-ansible)

------
ctippett
I’ve gotten a lot of mileage out of using cloud-init to bootstrap VMs. The
configuration is in YAML, but doesn’t require any other client side tooling.
Most major distributions (incl. the public clouds) all support it and make it
available in their machine images. It integrates nicely into Terraform too.

[https://cloudinit.readthedocs.io](https://cloudinit.readthedocs.io)

------
simonw
pyinfra looks interesting.

The thing I want it a way to keep configuration for a single server in a git
repo, and have that server periodically run "git pull" and apply any changes.

I ended up bodging together some custom scripts to do this but I really didn't
want to. Maybe I can get pyinfra to do this? I don't want to have to run it on
another machine and run commands over ssh though.

~~~
moreati
For reference, in Ansible land the two approaches to this would be
[https://docs.ansible.com/ansible/latest/cli/ansible-
pull.htm...](https://docs.ansible.com/ansible/latest/cli/ansible-pull.html) or
[https://github.com/ansible/awx](https://github.com/ansible/awx) (upstream
project of Ansible Tower)

~~~
simonw
I had missed ansible-pull, looks like the exact pattern I want. Thanks!

------
rcarmo
To be honest, I do everything inside my nodes via [https://cloud-
init.io](https://cloud-init.io) these days. A single YAML file (yes, yes, I
know) that typically bootstraps the machine, grabs a compose file, end done.

------
meddlepal
What I really want is a strongly typed version of Ansible possibly
substituting the YAML bit for just calling library code in say Go or something
else.

~~~
woile
I'm on the same boat. I'd like to try cuelang but I haven't had the
opportunity

------
throwaway42092
None of this stuff works with pipelined / staged deployments, where tools
cannot make changes across stages using SSH.

------
idoby
For all its flaws, Docker does solve a lot of the pain the author mentions.
Deploying postgres, for example, with a config file and keeping it running is
really very easy with Docker Compose.

Not affiliated with Docker except for using it, and while I do have some
thoughts about design choices made by Docker, it's still a very good tool.

~~~
dynamite-ready
I don't see Docker as an alternative to a configuration manager. For one,
turning to containers represents a distinct architectural choice, with a clear
set of tradeoffs, that a developer may or may not want to introduce to a
project.

~~~
idoby
It sure does, but even if you choose not to take advantage of container tech,
it's still a very easy way to deploy stuff on a single machine. Not sure why
the downvotes.

~~~
dynamite-ready
I didn't downvote. I see your point, but it's a bit of a digression, tbf.

------
dazoot
What about SaltStack ?

