
Use Vim Inside a Unix Pipe Like Sed or AWK - signa11
http://blog.robertelder.org/use-vim-inside-a-unix-pipe-like-sed-or-awk/
======
jdp
This post hints toward vim's history: the ed family of editors.

[http://blog.sanctum.geek.nz/actually-using-
ed/](http://blog.sanctum.geek.nz/actually-using-ed/)
[http://blog.sanctum.geek.nz/using-more-of-
ex/](http://blog.sanctum.geek.nz/using-more-of-ex/)

A big use case for them was using them as scriptable editors in pipelines, and
a lot of vim's commands are inherited from them. The diff(1) utility actually
has an option to emit ed script, allowing files to be patched in ed pipelines:
[http://www.gnu.org/software/diffutils/manual/html_node/ed-
Sc...](http://www.gnu.org/software/diffutils/manual/html_node/ed-Scripts.html)

~~~
robertelder
It's common for people today to think of ancient Greeks as being very
unsophisticated and primitive, but then when you start to read what ancient
philosophers wrote you think "Wow, these guys really had things figured out,
maybe better than we do now." I kind of feel the same way comparing modern GUI
applications to ancient editors like ed.

~~~
freshhawk
I find learning this lesson often enough made me realize that it's difficult
to remember that people in the past were every bit as intelligent as people
are now. They simply had fewer, older tools.

There's another related lesson there that progress, like evolution, is
progress in a certain direction. No one said that direction is whatever you
call "good" at the moment.

~~~
inanutshellus
> people in the past were every bit as intelligent as people are now.

... and some evidence supports they were smarter...

[http://www.huffingtonpost.com/2013/05/22/people-getting-
dumb...](http://www.huffingtonpost.com/2013/05/22/people-getting-dumber-human-
intelligence-victoria-era_n_3293846.html)

~~~
freshhawk
Good point.This argument that domestication lowers intelligence (among other
powerful abilities) and that we are unquestionably domesticating ourselves is
pretty damn solid. The only argument against it seems to be "... but my ego!".

Obviously there is some interplay with the fact that we develop new mental
models and thinking tools to augment intelligence.

Also immersion as children in highly abstract ways of thinking further
augments/multiplies raw intelligence (most convincing explanation to the Flynn
effect imo).

I've lost the link, but there was an excellent article I read related to the
amazing Otzi discovery
([https://en.wikipedia.org/wiki/%C3%96tzi](https://en.wikipedia.org/wiki/%C3%96tzi))
describing how adults of that era (modern humans, primitive societies) would
likely have been terrifying to us now in just how much they outclassed us in
raw strength, intelligence and stamina. We would be relying a lot on the
benefits of childhood nutrition and education to feel superior. This isn't
completely convincing, there are a lot of factors in play, but those levels of
brutal competition and danger would have a profound effect, especially
epigenetically.

------
martanne
Vim is actually a relatively bad filter (some reasons are mentioned in the
caveats section of the article). I explicitly designed my vis editor[1] in
such a way that it can be used as an interactive filter. The following works
as expected, use :wq to write to stdout:

    
    
        printf "foo\nbar\nbaz\n"  | vis - | sort
    
    

dvtm uses this mechanism to implement its copy mode.

I also recently integrated sam's structural regular expression based command
language into the editor. This might be useful for people who want the power
of stream editors but with instant visual feedback.

However keep in mind that this feature is relatively new thus likely still
contains some bugs ...

[1] [https://github.com/martanne/vis](https://github.com/martanne/vis)

~~~
exDM69
Thanks for your efforts with the Vis editor...

You can do this with any other editor using "vipe" from "moreutils" package.
It just opens $EDITOR with stdin input (put to a tempfile) and then reads the
tempfile and dumps it in stdout.

    
    
        printf "foo\nbar\nbaz\n"  | vipe - | sort  # invokes $EDITOR

~~~
martanne
Yes vipe is indeed useful for editors which do not support it by themselves.
Personally I find a solution solely based on pipes without temporary files
more elegant.

I should probably let dvtm fall back to vipe if it is installed. Also I will
have to check whether there is an easy fix to make vipe+vis work.

------
robertelder
Hi, I'm the author. While I've got your attention, I would be interested in
hearing feedback about how I can improve the reading experience (text
colour/fonts/sizes/wording etc.) of my blog posts.

I used to have blinking text for the email signups at the bottom, but I got
rid of it since it didn't have a huge effect (I think it only doubled the
conversion rate). I have learned that some people _really_ hate blinking text.

~~~
ComputerGuru
Hey Robert! I've had your virtual memory article open in a tab since it made
the HN homepage recently - but I had to close it because every day it would
cause a massive memory leak in Safari for some really odd reason. I'd
repeatedly kill the process hosting the webpage (it'll respawn 3 times then
surrender) but every time I visited that tab by accident it would happen
again.

Just an FYI :)

~~~
robertelder
Yes that's not surprising, the code for that implementation is quite scary and
in order to make the page not lag terribly I realized at the end that I would
effectively have to re-invent some of what react.js does (because it would
have taken too long to completely re-write). I've considered re-writing an
open source version with react that people could fork and incorporate into
course material, but all that stuff takes time.

I can think of a few places where there would probably be circular object
references, so that's probably where the memory leaks come from.

Thanks for the feedback :)

------
andrewstuart2
Yikes, we can shorten that a bit. :-)

    
    
        function vimify() {
          (vim - -esbnN -c $@ -c 'w!/dev/fd/3|q!' >/dev/null) 3>&1
        }
    
        $ echo '<h1>Change Me</h1>' | vimify 'norm vitxiOK I Will'
    

(Don't forget '-c' if you have more than one transform.)

~~~
markmontymark
And further shorten that to

echo '<h1>Change Me</h1>' | vimify 'norm vitcOK I Will'

~~~
alexdme
And further shorten that to

echo '<h1>Change Me</h1>' | vimify 'norm citOK I Will'

------
eridius
Why is this article talking about idempotence? I don't see why that matters in
the slightest for a unix pipe. Idempotence only matters if you're going to
take the output of the pipe and run it back through the pipe again.

There's a bit later on about idempotence and newlines, but I don't see what
that has to do with idempotence. Vim will append a trailing newline if the
file doesn't have one, sure, but if I pass the output back through Vim again,
it won't append another trailing newline, it'll just preserve the existing
one, so if the vim command you run is idempotent then the whole thing will
still be idempotent.

I'm guessing here that the author is simply confused about what idempotence
means. Because of the newline bit, I'm going to guess that what the author
means by "idempotence" is "any text that the vim filter isn't supposed to
modify will be left exactly as found in the input", but that's not what
idempotence is, and I'd also say that that property is pretty much a given for
any kind of text transformation used in a pipe, because if it doesn't hold
then why are you even using that tool?

~~~
pjungwir
That bugged me too, although it was such a fun article I didn't want to
complain. :-)

What he really means is that his first snippet is the identity function. Like
x*1=x or x+0=x. Whatever it gets, it passes along unmodified. I think it's
actually kind of nice that he started with that. Sometimes it is not easy to
figure out how to do, for instance with XLST.....

~~~
robertelder
Yeah, an identity function is probably what I should be calling it instead.

------
super_mario
Why type vitxi (visual inner tag delete insert), when cit (change inner tag)
will do?

Indeed, why use visual anything in a pipe :D?

~~~
robertelder
That's a good point. For some reason, I never use 'change' in vim, and I was
just following muscle memory.

------
iopuy
I have to say the only time I ever pipe directly to vim in a real world
scenario is when I want to see a diff in color:

    
    
        diff file1 file2 | vim -
    

Gives you the diff but colorized with Vim. Handy little trick.

~~~
super_mario
Or just:

    
    
        vim -d file1 file2.

~~~
iopuy
This brings up a side by side comparison of the documents. This is different
than the pure diff.

~~~
super_mario
Sure, for that there is also colordiff.

~~~
iopuy
Yeah, there are many other external tools that can accomplish this. Piping
directly to Vim just allows us install one less.

------
sigmonsays
Why did I read this.

This is far from practical to use on a day to day basis. It'd make my eyes
bleed if some coworker used this anywhere.

Why not just pickup a scripting language like perk or Python. Use awk or sed.
They're available everywhere.

------
koolba
Would non-ASCII data and escape codes have the effect of vim-injection[1]?

 _[1]: Think SQL injection but with vim commands._

------
gglitch
How does this compare to piping through Elisp?

~~~
to3m
This isn't supported at all in emacs by default - or, if it is, I never
figured out how.

You can try my pmacs script to get something sort-of-like:
[https://github.com/tom-
seddon/bin/blob/master/pmacs.py](https://github.com/tom-
seddon/bin/blob/master/pmacs.py) (not productized, and some self-assembly may
be required - should work out of the box on Linux, though, and probably OS X
too)

Works pretty well, though only at the end of a pipeline. (It could probably be
extended to work mid-pipeline though; you can get emacsclient to block until
the file's buffer is closed in emacs - this is what the "-n" switch inhibits -
giving the script a chance to wait, load the (presumably modified) file and
print it to stdout.)

------
spinningarrow
The title is missing the word "inside" from the original title, i.e. "Use Vim
_Inside_ A Unix Pipe Like Sed Or AWK". Made me wonder for a bit...

~~~
dang
We put it back.

