
Literate Devops with Emacs Org-Mode (2014) - tmstieff
http://howardism.org/Technical/Emacs/literate-devops.html
======
eadmund
Org-mode & Magit really are the secret superpowers of modern emacs. And
notmuch (full-text search of one's email, at least one order of magnitude
faster than Gmail's!). And elfeed. And gnus. And dired. And pretty much
everything else in emacs …

~~~
s4vi0r
Is there some way to tell org exactly how I want it to spit out HTML?

I was going to do my blog entirely with org-mode+org-publish, but it spits out
a mess of HTML and also a small bit of js for some reason, so I'm just writing
html directly myself now.

I don't really find writing html myself all that annoying or anything, but
it'd be nice if I could get org to generate really clean/simple HTML for me
that I could then write a small script to parse and wrap in divs I've defined
for styling.

~~~
btschaegg
I've been looking for a similar solution - albeit not for HTML - but haven't
found anything convincing so far. My train of thought was: It'd be great to
have a a canonical XML format for org files to export to, since after that,
you could essentially write XSLT transforms for almost any target format (even
back to org-files to go full circle). In my case, that e.g. would be odt-files
with branding for the organization I'm taking minutes for.

After a bit of googling, I also found an email[1] on the mailing list asking
for more or less exactly that, but things don't seem to have gotten far.

I've been thinking about adapting the existing HTML export for such an XML
format, but I really haven't had the time for it. :-/

[1]: [https://lists.gnu.org/archive/html/emacs-
orgmode/2009-08/msg...](https://lists.gnu.org/archive/html/emacs-
orgmode/2009-08/msg00189.html)

~~~
icebraining
Pandoc supports org-mode, so you should be able to use it to convert to
DocBook, which you can then use with XLST (pandoc also supports HTML directly,
by the way).

~~~
btschaegg
If I remember correctly, I had a look at the pandoc option, but got to the
conclusion that this way, I would throw away useful 'syntactical' information
(for example, I use todo-flags, tags and timestamps to track open tasks).

If you use org-mode without the need to extract such specific information (and
are only interested in generating a document), pandoc might of course be a
good way to go.

Edit: ...or maybe there even is a hack to identify and extract things like
timestamps anyway. I didn't look into it that much.

------
mjmein
Org-mode and babel sound like good ideas, but in practice I've found it very
finicky to use.

My main problem with it is that, unlike with executing commands in the shell,
with org you don't get any progress, and emacs seems to hang while running the
command.

For example, running apt update tends to freeze emacs without any progress
reporting for a few minutes.

I'm hoping that I'm doing something wrong and that someone can chime in with a
suggestion, because I really like org-mode, and I really like the idea of
doing devops using it.

~~~
Spivak
It's not 'freezing' Emacs so to speak, source block execution is synchronous.

Check out: [https://github.com/astahlman/ob-
async](https://github.com/astahlman/ob-async) if you want to change that.

~~~
mjmein
Thanks. I've tried that in the past. The problem is still that it doesn't give
any feedback on progress.

I still struggle to see how people actually use this with devops on a day to
day basis.

I think that some type of integration with tmux would work well - have a
running tmux session, and then just queue commands from emacs to that session,
instead of running directly through emacs.

~~~
gkya
If you want to see progress, it should be easy to add a process filter
somewhere (I bet there already is a hook for that, but not that hard to do
even if there isn't one) and make it display sth. on a secondary buffer.
That's the thing with Emacs: if you don't have it, it's very easy to add it.

------
matt_wulfeck
This is a cool use of emacs, but containers have made the process of iterative
development/system creation so much easier. If you have the deployment story
down, it’s even easier. And the configs are much more portable.

~~~
nine_k
The process demonstrated can be adapted to a wide variety of environments,
containers or not.

It's mostly about making a literate program, _executable on the spot_ , sort
of like a Jupyter notebook, only for shell commands (or nearly anything,
frankly).

~~~
jle17
FWIW, Jupyter notebooks also have a kernel to support shell commands.

------
ZenoArrow
So "Literate DevOps" is writing comments in your shell scripts and running the
commands line by line?

I agree it's a good idea. I do similar things when writing PowerShell scripts
(can run them line by line using PowerShell ISE).

EDIT: If you're going to downvote me, it'd be better to explain why.

~~~
elviejo
I didn't downvote but I'll try to explain.

Is the fact that you got the idea exactly backwards: is not about putting
comments on your scripts. Is about writing an article, intended to be read by
humans, that is also executable.

Imagine it this way: Suppose you are reading a blog post on how to install,
configure vagrant and install drupal on it. The article is pretty clear an
well written and you enjoy it. And you also know that is correct because every
single one of the instructions shown was executed while you were reading the
article and every output was given back to you.

That is literate programming, is closer to a jupyter notebook than to
'comments on scripts'.

~~~
ZenoArrow
> "That is literate programming, is closer to a jupyter notebook than to
> 'comments on scripts'."

Surely that's a matter of perspective? Imagine you have a text document
containing one command and one description of that command. Is this a literate
programming document or not?

Furthermore, does the mode change depending on which tool is used to open the
source file? For example, if I open a Bash script in a text editor then it's a
document, and if I open it in Bash then it's a script, but if I could open it
in a tool that allows me to both read and execute the script at my leisure
does it then become a literate programming document?

Aside from this, to use your Jupyter notebook example, is it necessary for the
results of execution to be captured in-line in order to follow the literate
programming methodology?

~~~
jonnycomputer
Answering questions one doesn't like is better than voting them down.

------
nimbius
much of whats covered in org mode can be handled by vim's folding option. if
you write ansible playbooks with coworkers who handle automation as a stream
of consciousness out of a single file, folding can save you some sanity.

[http://vim.wikia.com/wiki/Folding](http://vim.wikia.com/wiki/Folding)

~~~
quiq
Do vim's folds give you a full literate programming environment with support
for executing code, saving results in vars, WEB syntax, etc? From what I'm
familiar with, vim's folding is a lot closer to outline-mode[1], which org
relies on for folding. The folding is a _very_ small part of org-mode's
functionality.

[1]
[https://www.gnu.org/software/emacs/manual/html_node/emacs/Ou...](https://www.gnu.org/software/emacs/manual/html_node/emacs/Outline-
Mode.html)

~~~
pasabagi
I think you can run python inline in vim. But I obviously vim doesn't really
have something as full-featured as Org mode. I mean, there's vimwiki - and you
could probably get a majority of the rest of the features through messing
around with plugins. But at that point, you may as well just switch to
spacemacs.

