
Show HN: Aviary.sh – A tiny Bash alternative to Ansible - dxchester
https://github.com/team-video/aviary.sh
======
xbryanx
I'm not sure if I am understanding this (aviary.sh) correctly, but it looks
like this requires an agent to run on the configured hosts.

One of the things that I love about Ansible's model is that we never need an
agent on the host. Once you configure something with Ansible, there's no
artifact of the configuration left on the machine.

~~~
thr0w3345
The agent approach is mostly ok, it tends however to suffer when you have more
than a few hundred machines (I’ve found, anyway) that your ansible code rots a
bit. Say for that one server, you know the one, the weird etl thing bi uses,
was run once and during a prod problem you suddenly have to start fixing the
ansible code to rebuild it.

With an agent, it’s applying all the time and this doesn’t happen.

We actually moved to salt from ansible and we’re happy..

~~~
geofft
I'm a bit confused - agent vs. agentless isn't obviously correlated to
continuous vs. on human action to me. Write a cronjob / systemd timer /
scheduled task / Jenkins job / Travis cronjob / GitHub Actions scheduled event
/ CloudWatch + Lambda / whatever you like to run your Ansible playbook, from
one machine/container/whatever, on your entire fleet. (It's certainly no
harder than writing a cronjob or whatever to run your config management on
_every_ machine - if you can schedule tasks on all your machines, you can
certainly schedule them on one.)

That gets you the standard advantages of agentless setups, including not
requiring the runtime of your config management tool to be everywhere, being
able to reprovision ephemeral + immutable cloud resources, and being able to
centrally report errors, without any more risk of configuration drift or
bitrot.

------
ahnick
If you need a simple way to inject secrets into your aviary variables you may
find encpass.sh
([https://github.com/plyint/encpass.sh](https://github.com/plyint/encpass.sh))
a convenient choice.

~~~
logie17
That's a pretty cool project. It also is written in bash as well!

------
ggregoire
Ansible is agentless tho. That's a pretty big difference.

~~~
peterwwillis
Yes, but it's also horrible to use. I would rather set up a Puppet agent once
than struggle with Ansible for an eternity.

~~~
pricechild
Would I be right if I guessed you have a development background?

For reference, I have an ops background.

~~~
jeltz
I have a dev background and think that Ansible is much nicer to work with than
Chef and Puppet.

------
simonw
This looks really promising. I've been wanting a very lightweight solution for
this which runs on a single host (no need for another host just to run the
scripts) and pulls its configuration from a GitHub repository periodically.

I ended up writing my own - and hating that I had to do that - because I
couldn't figure out how to use any of the existing options. Their
documentation all seems to assume multiple hosts and a separate machine that
you run the scripts on.

"Each host periodically fetches the latest version of the inventory to see
what roles should it be performing" \- that's exactly what I want.

~~~
miked85
> which runs on a single host (no need for another host just to run the
> scripts)

This isn't a limitation of Ansible, you can run playbooks against localhost.

~~~
zhengyi13
True, but I think OP's point at least in part was that an awful lot of the
tutorials out there start with the assumption that the audience are sysads and
SREs who want to manage a fleet.

~~~
simonw
Yes, exactly. I could tell that Ansible could do it but so much of the
documentation assumed you would be using it with a fleet that I eventually
gave up figuring it out.

------
dheera
> Install from the command line, on a box to be managed by aviary.sh:

It would be even cooler if I didn't need to do anything to do the box besides
install openssh-server. I should be able to:

    
    
        $ sudo apt-get install aviary
        $ aviary install my_server 192.168.1.25 # a fresh ubuntu 18.04 install
    

and everything else should just be magic from there.

------
tony
If you like this - Fabric is awesome.

[https://www.fabfile.org/](https://www.fabfile.org/)

Use python over SSH. Super fast to begin, if you can ssh into the server, you
can run the equivalent of shell commands (subprocess calls in python)
remotely. With python, you can abstract and reuse, with the scriptability of
bash, but higher level niceties of the language, libraries and toolchain
(linting, formatting, autocompletion, imports).

Can it pull the latest version of your app, build it on your server? Yep.
Restart daemons? Update libraries (npm, python, etc)? Update system packages?
All work fine.

Can it configure a vanilla server from scratch? Yes, it's helpful to bring in
a higher level reusable library like
[https://github.com/fabtools/fabtools](https://github.com/fabtools/fabtools)
helps with that.

When does it hit limitations? More complex orchestration where you have
multiple servers (and variations of them) with configuration to talk to each
other over networks. Eventually you get to the kind of setup where having a
tool like Ansible / Salt / Puppet / etc. makes sense.

For a basic PHP/python/ruby/node site that's just running on a single cloud
server? A declarative config manager would be overkill for me.

~~~
linuxftw
> Use python over SSH. Super fast to begin, if you can ssh into the server,
> you can run the equivalent of shell commands (subprocess calls in python)
> remotely.

This literally describes ansible. In fact, ansible one-ups this because you
can specify raw commands if python is not installed remotely [1].

Ansible provides lots of modules, you don't have to use any of them. I have
plenty of 1-off ansible playbooks that I don't care about idempotency that are
just a bunch of 'cmd' statements. It's a very flexible tool.

1:
[https://docs.ansible.com/ansible/latest/modules/raw_module.h...](https://docs.ansible.com/ansible/latest/modules/raw_module.html)

~~~
maxmalysh
Not really. Ansible is a complex DSL based on YAML. Fabric is really simple
and uses real Python.

~~~
llama052
Can't say I've ever really thought of YAML as complex. Especially when the
Ansible modules are translating things so you can just define the action you
want to take, and each module is very well documented.

~~~
dnautics
Embedding python in yaml has a _ton_ of gotchas for cases where string
processing doesn't quite line up with the way yaml wants things. Variable
interpolation is also a real mess.

~~~
geerlingguy
Typically the recommendation is to write your own python plugin/filter/test if
you find yourself writing a lot of python in yaml, that way the yaml stays
sane and readable, and just has some light Jinja.

------
burritosnob
This was written by team.video who is in a hyper competitive space. Every hour
spent on Aviary is an hour not spent on differentiating customer features.
Why? There are a ton of open source CM tools that give you same functionality
and do it better. This is a problem you should have solved after you were a)
profitable b) at scale and c) probably not even then.

~~~
Uehreka
So like, unless you're one of their investors upset at how your money's being
spent, why is this any of your business? Startups often take unusual paths to
profitability. They all seem crazy at first, many are in fact crazy, a few
turn out to be quite prescient.

~~~
burritosnob
Not being an investor doesn't preclude me from having an opinion. There is
nothing prescient here... just an engineering team burning cycles better spent
elsewhere at current stage of the company.

------
MINIMAN10000
Unfortunately all I could think of was "that sounds cool but I'd want to stick
to dash instead of bash"

Also didn't realize it wasn't agentless which is pretty critical imo

~~~
yjftsjthsd-h
> Unfortunately all I could think of was "that sounds cool but I'd want to
> stick to dash instead of bash"

Well, POSIX sh, ideally. That way you get free portability to ~everything;
it's not a great burden to install bash or dash on ex. NetBSD or AIX, but sh
is as close to universally preinstalled as you're going to get.

> Also didn't realize it wasn't agentless which is pretty critical imo

Ouch, yeah that outright disqualifies it for me.

------
arminiusreturns
Now this is something I can get behind! Very cool work. I think bash gets a
bit of a bad rap and it's very popular to bandwagon hate on it, but it's one
of my favorite tools ever.

------
peterwwillis
I don't care what language or paradigm the tool is written with, Configuration
Management tools are evil and should be avoided at all costs. I would
literally rather maintain a crappy server and fragile software install
manually and treat it as a pet, than use any CM tool at all and pretend my
servers are cattle. In reality, the CM becomes the pet.

The solution to CM is to use immutable infrastructure, and versioned immutable
artifacts. With these paradigms, state never drifts, so there is never a need
for configuration management at all. Everything becomes stable and predictable
and you no longer have to maintain a finicky pet.

But how do you bootstrap your systems, you say? The simplest way possible:
make a crappy procedural program that bootstraps the very beginning of your
system just enough to push versioned immutable artifacts and run arbitrary
commands (essentially just "scp", "reboot", "docker pull", "exec"). With
cloud-based systems, you shouldn't need to bootstrap anything at all. Build
your versioned system images and containers, deploy them, destroy them, re-
deploy them.

No offense meant to Aviary, I'm sure there are still legacy systems that
require some CM before they can be abandoned, but I really hope people
building new systems will abandon them ASAP.

------
t0astbread
This looks really effective! I do something similar to keep track of what's on
my workstations: I just have one large idempotent shell script to set up all
the software I use. To help myself not loose track of updating it I have a
little helper script to detect packages that aren't tracked in the config.
I've recently thought about breaking it up into smaller "module-style"
scripts. Maybe this could be a good fit (with minor adoptions)!

------
freedomben
Really neat. I still need to evaluate it more, but this is the kind of thing
I've looking for for small infrastucture that isn't likely to grow (like home
services or small office, etc).

------
markstos
I'm all for trying simpler approaches and have written some tooling in bash
myself, but at this point Python is pre-installed on many servers like bash
is, a large benefit of Ansible is the VAST number of modules already written
for Ansible.

Imagine if this grew to support all of Ansibles features and modules, but did
so in bash.

 _shudders_

------
trynewideas
it requires an agent, so it's a tiny Bash alternative to Puppet, but that
doesn't sound remotely as cool

~~~
stevekemp
I mentioned it above already, but I've been experimenting with an agentless
puppet-lite:

[https://github.com/skx/marionette/](https://github.com/skx/marionette/)

Early days, but the primitives are enough to do useful things on my own
systems..

------
mothsonasloth
Currently I do a fresh install of Ubuntu every time I do new work (every
6months). Normally I spend about 6 hours getting my machine configured up
(installing softwarem configuring my terminal, etc)

Will Aviary be able to solve this?

~~~
whycombagator
Maybe you have an involved set up and already do these things, but surely you
could just write a bash script that installs all the software you need and
store it with your dotfiles.

After a new Ubuntu install you can just pull down your dotfiles and run your
install script. Aside from some things like ssh keys, gnome tweaks, etc. -
this approach would likely get you 90% of the way to set up and would take ~15
mins.

Arriving at this set up would take a little bit of time, and require periodic
maintenance, but given a weekend you could have it set up and tested.

Here[0] is an interesting way to store dot files.

[0] [https://www.anand-iyer.com/blog/2018/a-simpler-way-to-
manage...](https://www.anand-iyer.com/blog/2018/a-simpler-way-to-manage-your-
dotfiles.html)

------
stevekemp
I've been working on a localhost-only system for automation myself, inspired
by puppet but with far fewer modules:

[https://github.com/skx/marionette/](https://github.com/skx/marionette/)

As of this week it can now pull down docker-images from dockhub, which was
something I'd been needing to rebuild my own systems. Early days, and it is as
much proof-of-concept as anything else, but it seems robust and reliable in my
own use.

Of course I need to add "written in go" to advertise properly!

~~~
ahnick
Does this save time in writing your own personal automation or make that
personal automation less error prone? How would you compare it to just writing
your own bash scripts or using something like Bashible
([https://github.com/mig1984/bashible](https://github.com/mig1984/bashible))?

~~~
stevekemp
Making things repeatable and scripting will usually be a time-saver (unless
you spend 100+ hours writing the automation and never reinstall your hosts!)

For me it was a learning experience as much as anything else. I'd probably
have stuck with ansible, puppet, or similar established project if I didn't
want to learn/experiment.

------
drinchev
> Installation

> curl [https://aviary.sh/install](https://aviary.sh/install) | sudo bash

I would really try to avoid this, especially when it's targeted for sysadmins
( I guess DevOps nowadays ).

What's the big deal of having this instead :

> $ wget
> [https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary...](https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-
> install.sh)

> ./aviary-install.sh

~~~
nerdponx
Better still:

    
    
        wget https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-install.sh
        less aviary-install.sh
        ./aviary-install.sh

------
thayne
Wow, perfect timing. I was just looking for something like this. I haven't
looked at it too in-depth yet, but the design goals align with what I want.
Minimal configuration management that pulls config from git and runs locally.
Ansible pull, salt masterless, chef solo, etc. work, but they weren't designed
to be run that way, so have unneeded complexity (and are kind of heavy for
what I want).

------
efrecon
I have been working lately on something a bit similar, but in pure POSIX sh.
It's called primer,
[https://github.com/efrecon/primer](https://github.com/efrecon/primer), has no
agent, cannot work on remote host but supports amalgamation for easy remote
installation and operation. Primer is very much docker oriented in the sense
that it seeks to automate the installation of all tools before a docker
project, then supposes you run docker-compose for the rest of your projects,
or orchestrate in any other way.

------
Hackbraten
I’ve been waiting for a tool like this.

~~~
dxchester
Here you go! Yeah, I've used a bunch of CM tools over the years, and it always
seems like you're learning a DSL of one sort or another, just to have it
translate into the bash that you already know.

~~~
cloin
And don't get me wrong, I love bash. I automated my first job using bash. But
relying on bash just relegates aviary to a certain job description and thus
just low value, domain specific tasks. Seems like a neat way for a linux admin
to run a homelab, but not something that's easily adopted for larger teams
working across OS, network platforms or integrating with external services.

~~~
chillfox
Not everything needs to be for large teams. There’s value in simple solutions
for small teams.

------
hummo56
Looked good but I needed to leave the site after seeing they suggested doing

curl [https://aviary.sh/install](https://aviary.sh/install) | sudo bash

~~~
lukevp
What’s the problem with running a script this way? Several major tools install
like this (docker, netdata...) here’s a take on this that was posted to HN a
while back: [https://www.arp242.net/curl-to-
sh.html](https://www.arp242.net/curl-to-sh.html)

~~~
effie
The problem is that this way, you're giving total control of your machine to a
script that curl will download from a website, before you check the script.

A better way to try a new software is to download the sources, check them,
build/install/run them under a low user with lowest required access to your
system. Even better, do this in a virtual machine.

------
lifeisstillgood
I think there is an Genuine Enterprise Need explosion at certain scales. It is
not box ticking exercises or empire building but sometime after (my guess)
1,000 units of _anything_ (users, servers, customers, whatever) you start
finding that "simple as possible" does not help that much anymore. LDAP
support becomes useful and so on.

Maybe, I hope maybe, it's not a real portal that we pass through, but it does
seem so.

~~~
chillfox
The metric is number of different things you do as an organisation, or number
of bored staff.

You could probably keep things simple for longer if you issue everyone with a
new fidget spinner monthly. A lot of what I see is just people who can’t
resist fiddling with stuff that works until it doesn’t.

~~~
lifeisstillgood
The usual approach (cheaper than fidget spinners) is to have so few people and
so much work they only do the barest essentials

~~~
chillfox
Yeah, but that doesn't work when you have middle managers who's pay is tied to
the amount of people that they manage.

------
Propolice
If you like rerun, drist or bashing. I made a short Go program to replicate
the experience.

[https://github.com/tongson/rr](https://github.com/tongson/rr)

Yeah, naming conflict with the debugger. Was a simpler replacement for rerun
so the shorter command name made sense to me.

------
ansible
I do something sort of similar, also using git to maintain the inventory of
what to run on what machine.

In my case, I'm using docker-compose.yml files for each machine, and then
running:

    
    
      docker-compose pull
      docker-compose up -d
    

We initially wanted to use Watchtower, but it didn't work well for us.

~~~
yonixw
Nitpicking: Why pull separately? it will pull if no image stored locally or
even if the `docker-compose.yml` changed.

~~~
yjftsjthsd-h
Stale images can be a reason. If I specify `image: foo:latest`, then push new
versions, it won't automatically pull new versions unless you do an explicit
pull.

------
comba
More modular, smaller, simpler than aviary is rrconf
[https://github.com/rrconf/rrconf](https://github.com/rrconf/rrconf)

------
senorsmile
How is this idempotent?

    
    
      cat <<EOF > /etc/motd
      "Ever make mistakes in life? Let’s make them birds. Yeah, 
      they’re birds now."
      --Bob Ross
      EOF

~~~
mike_d
Paraphrased from Wikipedia: "A function looking up a customer's name and
address in a database is typically idempotent, since this will not cause the
database to change. Similarly, changing a customer's address to XYZ is
typically idempotent, because the final address will be the same no matter how
many times XYZ is submitted. However, placing an order for a cart for the
customer is typically not idempotent, since running the call several times
will lead to several orders being placed. Canceling an order is idempotent,
because the order remains canceled no matter how many requests are made."

The greater-than redirector fits that description, as it is normally
implemented as an atomic write. Note that it does not say anything about
optimization or efficiency. Only that the result remains unchanged.

Web developers have bastardized the term and take it to mean many things it
does not because of the definition of HTTP GET, even though no GET operation
is ever idempotent in the real world.

~~~
senorsmile
In automation (like Ansible), the term is overloaded. Something is only
idempotent if it only tries to take action if the actual state does not match
the desired state.

This will overwrite every time, which is inefficient.

As a plus, Ansible also notifies you which things have changed. This also
enables easy dry runs (check_mode in Ansible).

------
easterncalculus
Very nice! Ansible can be a little heavyweight for some of the things that me
and a lot of people do. I'll definitely be taking a look at this.

------
rawoke083600
Bash just use bash ! Nothing new to learn for developers or to debug. Ok i'm
ready for my down-votes...

------
nodesocket
Typically I use Packer + Terraform and it works generally well (if you going
with the immutable servers philosophy). However, often times immutable servers
is hard to actually do in practice. If Hashicorp added a new product similar
to Ansible / Aviary.sh / Fabric I think it would complete the entire DevOps
loop under a single ecosystem.

~~~
raffraffraff
If you're sticking with hashicorp tools and use consul, then something like
consul-template should work. Your packer builds would just have to copy
consul-template configs and template files for the things you want to manage.
Each template watches the consul kv store for matching keys, and rewrites the
target config file using their values. You can instantly rewrite configs and
execute follow up actions across your fleet by updating a consul key value.

~~~
nodesocket
Agree Consul is an elegant solution for managing files, what about running
commands?

~~~
johntash
I don't see it used very often, but Consul actually can execute remote
commands. You do have to enable it in the config though.

[https://www.consul.io/docs/commands/exec](https://www.consul.io/docs/commands/exec)

edit: Also, Nomad would be another choice if you're looking at hashi tools.

------
nyxtom
Nice, I plan on using this

------
q3k
> bash is just fine (yes, it is)

No, it's not. It's 2020, we need to stop it with the stringly-typed, footgun-
filled madness that is bash scripting. We deserve better than this.

Unless I'm missing something, this README example already contains a bug:

    
    
        template $(dirname $0)/motd.template > /etc/motd
    

This will fail if the script path contains a space.

~~~
montroser
I think you are forgetting that serious operators do not have spaces in their
paths.

~~~
hk__2
*People who are used to work with bad tools don’t have spaces in their paths.

~~~
tenebrisalietum
"C:\Documents and Settings" still hurts after all these years.

~~~
linuxftw
"C:\Program Files(x86)". Who ok'd that?

~~~
GrinningFool
I haven't looked in a while, but last I checked you could also still get to it
via C:\PROGRA~1? (This was years ago, that might have been 'fixed'?)

~~~
joombaga
These are called 8.3 filenames. They're a filesystem feature. They're still
created by default in Windows 10 on NTFS, but can be disabled or stripped
using `fsutil`.

------
salasrod
> But it's too funky you say. Well, yes, bash can have its quirks. But we need
> some funk every now and then. Let's just embrace it!

My god, no, stop. Funkyness in production equals outages.

~~~
ironmagma
As bad as Ruby is, it’s still better than Bash and you can do all the same
effortless execution of random commands in your script like you would in bash.
If one can’t be bothered to write it in a real language like Rust or even
Python, you could at least do Ruby.

~~~
cortesoft
What makes python more of a real language than ruby?

~~~
ironmagma
It’s more mainstream for tools like this.

~~~
rad_gruchalski
Please explain. Have you heard of chef? That is ruby and it can be beautiful.
Chef > Ansible. Chef server is meh.

~~~
ironmagma
The point wasn’t that Python is better than Ruby, they are about the same IMO.
The point was they are both better than Bash, and Ruby actually makes it super
convenient to write bash-like scripts, so there’s no excuse to not be using
Ruby.

