
Literate DevOps - geocar
http://www.howardism.org/Technical/Emacs/literate-devops.html
======
Arnavion
>vagrant ssh-config --host clientvm >> $HOME/.ssh/config

You may want to instead do:

    
    
        vagrant ssh-config --host clientvm > $HOME/.ssh/config.vagrant
    

and add:

    
    
        Include config.vagrant
    

to ~/.ssh/config. Otherwise if you delete your VM and make a new one, the
entry from the old one is going to stay in your main ssh config, so it's going
to fill up with cruft over time.

~~~
cat199
to add, same mechanism can be used for multiple 'config.vagrants', 1x per
host, to allow the output redirect to only config for 1 host at a time

------
vr46
Not sure how old this post is, 2014 by the looks of it, but modern declarative
automation makes this redundant. Ansible, as an example, would abstract away
the CLI commands and be very readable in very few lines.

The setup described in this post looks fragile, regardless of whether it's
literate. I mean, we used to do these things, but we've moved on.

~~~
trabant00
That's what ansible PR material would have you believe. In reality only very
simple things can be translated into ansible/salt and even then you may hit
one of their many bugs. If you try to do complex things even bash would be a
better choice (although obviously the best).

There's one thing ansible/salt are good at: make it easy for cheap replaceable
devops to write tons of repetitive boilerplate yaml. Harder for them to shoot
themselves in the foot than with a real programming language. But once you
descend into jinja hell that's not very true either.

Also: newer isn't always better and all that.

~~~
zufallsheld
I always read that ansible can only be used for very simple things. However I
never read what things are too complicated with ansible. Can you share an
example?

~~~
Xophmeister
Ansible is good at putting configuration files in the right place and setting
their appropriate values to what you want — provided it’s relatively simple,
otherwise prepare yourself for complex templates — or installing software from
a package manager, etc., etc. As soon as you go outside what its modules
provide, it gets more difficult. Modules themselves are actually easy to
write, provided you maintain Ansible’s (completely reasonable) maxim of
idempotency. Sometimes, however, idempotency is hard to enforce.

The best example I’ve come across is building software from source. When
there’s no package, you have to do this. Some may argue that this is not
Ansible’s job, but I don’t see how it’s different from `apt install`, nor
where else it would fit into the pipeline. Anyway, the way I solved this for
example was by pinning the idempotency on the existence of build artefacts.
Those are often software-specific, so it’s quite fiddly/non-general to find
the right artefact.

~~~
notabee
I'd be interested in how you solved this, if you don't mind sharing. I tried
writing a role to build source RPMs and it kind of works after doing some
really dirty regex tricks, but only barely. I kept feeling that there must be
a better way that I wasn't seeing.

~~~
Xophmeister
Don’t say I didn’t warn you ;) This is not used any more — and the repo’s
history looks messed up — but it was my solution to the “configure, make, make
install” dance in Ansible:

[https://github.com/wtsi-hgi/hgi-
systems/blob/master/ansible/...](https://github.com/wtsi-hgi/hgi-
systems/blob/master/ansible/roles/irods-client/tasks/jansson.yml)

The replies to my original post say this is not something Ansible should be
doing. Fair enough :P I was young!

------
peterwwillis
I like the concept, but the problem with embedding random snippets of code
into a document is there are too many assumptions about the state of the
system they're running in. The operating system, the revision, the installed
software, how it's configured, where things are located, orders of operation,
what network you're on, what access your user has to what systems, etc.

Rather than embed snippets, write a single simple script that automates your
steps, and create an immutable artifact. The simplest way to do this is to use
Docker with a base image that mimics the target system, and performs all
necessary steps in one go. This allows you to perform a 'test run' in an empty
container to make sure the steps work. And you can embed in-line documentation
and generate it with Doxygen or some other tool. (I _highly_ recommend you get
all your teams to standardize on a tool like this, and use it literally
everywhere)

(Also, pet peeve, but what does DevOps have to do with this? Can we stop over-
using this term please? Not everything that's "sysadminny" is DevOps)

~~~
llarsson
Yes, much better. This person should use Docker instead and output the RPM
from the container. Reproducible as all assumptions are clearly stated and
much faster to execute again if the source changes (using cached layers for
most prerequisite steps).

Keeps comments close to the actual code, too.

------
lazyant
This is great. Now, there's quite a bit new to learn here.

I use Markdown notes files with quoted commands and outputs as first
exploratory documentation and before it goes into
Ansible/Dockerfile/Packer/Whatever. I can add links, images etc and anybody
can easily follow and edit/PR since well, it's Markdown.

Seems to me what I'm doing is basically the same idea as this, with total
flexibility (no schema), nothing new to learn only I can't run the .md file
(how hard would it be to parse a md file and ignore everything except the
triple ticks and execute that, in say Bash?).

~~~
IggleSniggle
Maybe even make your code blocks language aware to pick the execution context,
provide a language tidbit or even shebang, ie

    
    
        ```js
        console.log('yo, sup')
        ```
    

or

    
    
        ```#!/usr/bin/env node
        console.log('yo, sup')
        ```
    

That way, your md files could be both executable/interactive and be flexible
in regards to their execution environment (as long as you are on a unix-like).
Trying to resist the urge to go make a VSCode extension to do this right now.

~~~
c6401
atom with hydrogen plugin can run these

~~~
IggleSniggle
thanks for this

------
codygman
I've done this before, but more directed at general software development as a
sort of documented reproducible and resumable state. Ask me anything :)

I never got good at tangling my blocks together in the end though which is
what I think would have made this method really powerful.

~~~
t0astbread
Isn't that just literate programming? Or do you mean something else?

~~~
codygman
It was kind of more like coming up with organized snippets and notes while
understanding a problem.

I'm expressing the wish that were I better at organizing them better that I
could have tangled all of those snippets into a working (if ugly) source file
and work from there.

Does that make sense?

~~~
t0astbread
Ohh I see, yeah. I usually draw diagrams or write pseudo-code out on paper (or
both combined) when trying to grasp something difficult but "notebook-style"
exploration also sounds interesting.

~~~
codygman
> but "notebook-style" exploration also sounds interesting.

It is, but if the process and tooling isn't really polished and responsive it
feels really bad.

When everything is smooth though, it might be ideal. Jury still out on that
one.

------
hackerm0nkey
I do this day to day, nice to see that there's a name for it :)

Start every piece of work in a markdown with vim. and proceed to dump my
thoughts, what I have tried, what did work and what didn't. Also any useful
links that I've found useful.

And as they say "You only learn once you reflect", this proved invaluable for
me going forward when faced with "ah. I've done that before but don't quite
recall the details"

I move feel this to be like literate/log driven development

------
olefoo
If your config files don't have more comments than code; you're doing it
wrong.

Filed under things that have been true since the 1970's if not before.

------
bloopernova
It would be interesting to take the idea in some of these comments and turn
out some Ansible or Terraform code from literate org-babel documents. Kind of
a 2020 DevOps take on the linked article.

Most of my org-babel stuff is running AWS commands and collating the results
into tables for more documentation. I write my Terraform code outside of org-
babel.

------
JoelMcCracken
I was just doing something quite similar recently for my workstation:

[https://github.com/joelmccracken/workstation/blob/master/wor...](https://github.com/joelmccracken/workstation/blob/master/workstation.org)

------
sampo
Could you do same with Jupyter notebooks?

~~~
polyrand
Definitely, have a look at nbdev:

[https://github.com/fastai/nbdev](https://github.com/fastai/nbdev)

I think it's a very underrated way of programming that can have a great impact
in the future.

~~~
zelphirkalt
Does this have noweb with the same amount of functionality as Org-babel? What
about multi language support out of the box?

------
markus_zhang
Just curious as a Non developer who knows a bit of programming, is DevOps
always this complicated?

~~~
outtatime
It's not complicated per se, but the industry makes it complicated. Running a
basic web app isn't complicated if you throw up a server, but if you use
Kubernetes then you've suddenly made things much harder.

Things are only as hard as we make them.

~~~
oldsj
It depends on your scale of course. It’s not like “the industry” just wants to
make things complicated for the fun of it. There comes a point where you’ll
need more than one server to host your web app and when you do Kubernetes
provides a nice abstraction in addition to providing a ton of other useful
things at scale like infrastructure as code to zero downtime, continuous
deployments.

~~~
zeveb
> It’s not like “the industry” just wants to make things complicated for the
> fun of it.

Doesn't it? I think that there _are_ plenty of folks who want to make things
more complicated, because complicated things are fun, while just making things
work can be a pain but also boring.

------
viro
He legit lost me at "I pop into Emacs". I noped right out of that tab. Not
saying its right... but it is what it is.

Edit: Be gentle on the downvotes ... I was clearly trying to be funny.

------
jozzy-james
this all seems everything looks like a nail when all you have is a hammer
(emacs in this case)

------
knowhy
org-babel is a really cool extension of org-mode and Emacs. But it is only
useful for Emacs users and therefor useless for any professional use-cases. As
much as I love Emacs expecting people to learn it is a bit too much.

~~~
bloopernova
Well, it is possible to run Emacs in batch mode from the command line.
Therefore you could have someone create the files as documentation, with
instructions on how to run them as a single command.

I haven't tried this specifically with org-babel, and as other commenters in
the thread state, Terraform/Ansible may not lend themselves to org-babel quite
so easily, but it's certainly possible. Although maybe not worth the effort
unless you really push hard for a team to document their infrastructure code
in this way.

~~~
konjin
I have used emacs with make for this purpose. It works, but the main issue is
white space. Any language that has significant white space is liable to break
in org mode, and a ton of devops tools are python based, or python like.

I've been using guile and noweb, with a tangle I wrote myself, for literate
devops on top of Guix and it works better than anything else I've tried.

~~~
bloopernova
That's really cool, do you have any public examples of the guile/guix code?

