
Explainshell – match command-line arguments to their help text - duck
http://explainshell.com/
======
gruseom
What a great idea. One of those clever ideas that are immediately obvious once
you see them (but not before).

I find man pages a chore to read, and that has definitely held back my skills,
so if this actually works—i.e., you can type in any command and get a readable
layout of what it does—I can definitely see using it.

Edit: It might be tricky to make this robust enough across all valid commands.
I tried a few variations and they didn't seem to work as well. Beware the
cherry-picked example.

Also, everyone asking for pipe support is dead on. It's how most commands
actually get used, so it's really needed. Plus it would take this tool to a
whole new level of coolness.

Edit 2: Who made this? I think if you wanted, you could turn it into a new
kind of learning environment for the shell. It already is that, in embryo, but
there are a thousand directions it could go. The trouble with learning the
command line is the overwhelming mass of information with no way of navigating
it or distinguishing what's important from arcane detail. What's badly needed
is a simple organizing principle that feels good to work with and lets you
learn what you care about right now (and hides the rest). You've got that
here.

An it-just-works interactive tool is a really promising way to go. There are
countless tutorial articles and books, but they're one-size-fits-all, and once
they get too complicated they suffer from the same problems that man pages do.
A tool that can actually be used to play with and learn interactively would be
a real contribution. But it might be a lot of work to turn this into that. I
doubt that the corpus of "all standard Unix commands plus all their options"
can be sucked up and re-presented by any generic script; the variations are
too unpredictable. There would probably have to be a lot of attention put into
important special cases, and a lot of careful design to get it to it-just-
worksness. And that is a must-have for beginners.

~~~
idank
Thanks for reviewing it.

I definitely went into this with a 'let's start with something good and useful
for as many commands as possible', so I haven't focused on accuracy for those
edge cases. But from what I played with it so far, results have been pretty
good.

Agree again on the pipe et al. support. It's not easy, hence why I haven't
added this right out of the gate. I basically have to build a parser for bash,
although I can probably get away with handling only a small subset of it that
is interesting in this context, such as pipes, redirections, etc. I still have
to figure out how to display multiple commands, because chunking them all in
one page isn't going work visually.

I'll be happy to hear more about turning this into a learning environment. It
might not work for this particular tool, but maybe it can use the foundations.

~~~
mtdewcmu
Is it possible to use the parsing in this to generate autocomplete rules?

~~~
idank
It's possible, but I'm not sure how robust it will be. There are a lot of
heuristics being used to guess what the options are, if they expect an
argument, etc. So when feeding it with a command that was typed by someone,
which is most likely correct, it's fairly easy to look up each option in all
the possibilities. Not to mention some commands have a very complicated set of
rules (e.g. find), and the position of some options is context sensitive,
which explainshell isn't (it could be, but that would require a lot of manual
work).

It's interesting because I had an idea once to do the opposite: see if
explainshell could use completion files from bash/zsh/fish as another source
of options besides the man page.

~~~
mtdewcmu
Completion scripts from bash are far too much of a kludge to extract
information from.

Most tools are written in C, and the standard way to parse options in C is
with getopt(3) or getopt_long(3). I actually looked into searching the
compiled binary for the getopt_long structure, but that wasn't so easy. Since
these functions have information on all the possible options, it would be nice
if there was a standard way to get them to spit out the option list.*

See my reply to your comment in another thread. It would be a killer app to
have better autocomplete support, but I can't offer assurance that it's
feasible.

* I had another idea: you could use ELF tricks (LD_PRELOAD) to load a different version of getopt/getopt_long that would spit out the specifications.

------
idank
Author here, thanks for the feedback guys!

\- adding support for shell syntax is definitely the next thing on my todo
list. It's a bit tricky since Python's shlex (which I'm using to parse the
command line) is fairly limited in functionality and pretty much only handles
quotes. So I'll have to write a mini bash parser which isn't trivial.

\- the automatic options extractor isn't perfect and requires manual
corrections here and there

\- add an API for command line tools

I'll be happy to answer any questions you might have.

~~~
DEinspanjer
I'm a pretty fair hand with regexes, if you would like any assistance, please
feel free to drop me a line.

~~~
idank
The regex is already getting out of hand ;)
[https://gist.github.com/idank/6382427](https://gist.github.com/idank/6382427)

At some point I wanted to convert it to a DFA because I thought it will give
me more control and flexibility and be more maintainable over time. The thing
to look out for is not make it too permissive so it starts catching things
that aren't really options.

~~~
mtdewcmu
The occasional tool, like find, is beyond the capabilities of a DFA, because
it's based on a context free grammar (e.g., it counts parentheses).

~~~
JelteF
True for official regular expressions, but not for the ones used in most
programming languages. [http://nikic.github.io/2012/06/15/The-true-power-of-
regular-...](http://nikic.github.io/2012/06/15/The-true-power-of-regular-
expressions.html)

~~~
mtdewcmu
True. Perl-style regexes have grown capabilities far beyond regexes in formal
language theory. Note that I was referring to DFAs.

------
ekyo777
For those like me who want to call it from their zsh, add the following to
your .zshrc:

explain(){ curl -s $(echo
"[http://explainshell.com/explain/$1?args=${@:2}"](http://explainshell.com/explain/$1?args=${@:2}")
| sed -e 's/ /+/g') | sed -n '/<pre/,/<\/pre>/p' | sed -s 's/<[^>] _> //g' |
sed -e 's/^ _//g;s/ *$//g' | grep '.' | cat }

and call it, for example like: explain git status -s

~~~
connerbryan
Perhaps I'm missing something, but this doesn't work for me:

% explain git status -s

explain:1: no such file or directory:
[http://explainshell.com/explain/git?args=status](http://explainshell.com/explain/git?args=status)

curl: no URL specified!

sed: -e expression #1, char 11: unterminated `s' command

~~~
ekyo777
It's probably hn's formatting. I just tried copy/pasting from my post and I
get the same error.

Try copying it from there:
[https://gist.github.com/ekyo/6383547](https://gist.github.com/ekyo/6383547)

I can't edit my previous post anymore, if someone can maybe add something like
"EDIT: copy/paste friendly version
[https://gist.github.com/ekyo/6383547"](https://gist.github.com/ekyo/6383547")
or better, fix the formatting so it works...

------
Pxtl
... I want this integrated into the shell on my ubuntu machine. Give me a big
freaking sidebar panel with this thing on it.

~~~
Bootvis
Agreed something like

    
    
        explain tar xzvf archive.tar.gz
    

would be great.

Author: Do you have an API?

~~~
BHSPitMonkey
Once the source is released, it should be possible to make such a client
without needing to connect to a web service. These breakdowns are figured out
by the Python code written by the author.

~~~
Bootvis
That's of course great but as a child of our time I want my explain command
_now_ ;)

------
Ecio78
Very nice!

p.s. it looks like pipe is not supported:

[http://explainshell.com/explain/cat?args=text.txt+%7C+grep+-...](http://explainshell.com/explain/cat?args=text.txt+%7C+grep+-v+%22this%22)

------
jongraehl
Nice. Positional arg descriptions are probably hard to automatically find -
e.g.
[http://explainshell.com/explain/tr?args=-sd+G-Z](http://explainshell.com/explain/tr?args=-sd+G-Z)

~~~
idank
Yes they are, and most of the time require manual tagging. I fixed 'tr', try
again ;).

------
extesy
Looks awesome. Is it possible to list explanations in the same order as they
are in the command line? For example, options for "tar xzv" are listed in the
"v, z, x" reverse order.

~~~
idank
The order of help blocks is determined by their position in the command and if
they're on the left or right side of the lines.

If you always keep the order, lines would have to cross each other which looks
less pretty and pretty confusing.

~~~
jeroen
When I resize my browser the display order switches back and forth from xzv to
vzx. Their lines connect on the left side.

In the original order the lines cross.

The f-argument is always on the bottom with its line on the right.

------
gwu78
There is no substitute for reading source code.

grep -A20 -ri "usage:" src-directory

should get you started. But if you rely on looking only at usage(), you'll
miss "undocumented" features. Try reading the OpenSSH source for example.
There is at least one undocumented option.

If the source code is so voluminous and the organization of the project so
convoluted that it is prohibitive to scanning through the source to figure out
what the program does, then that in itself tells you something.

The smaller the program (and its source code), the easier it is to master it,
in my humblest opinion.

Generally, the larger the program, the higher the probability of bugs and
vulnerabilities. This, you might say, is the cost of adding "features".

I prefer small programs where I can read the source code quickly and easily.
They need not have many features. The less output the better. As long as they
do one thing well enough, and reliably.

Not to mention what it does to ease of use every time you add more "features"
to a program. Look at the OpenSSL binary. It is like they are going for the
World Record on the number of options you can cram into a single command line
program. Insane.

------
Toenex
Any reason why this is stuck in browser? I've got most oif the raw data (man
pages) already on my machine and the closer this is tied to command line the
more likely I am to use it. For instance I'd love to be able to do;

    
    
      explain !!
    

in my shell to work out what just happened!;)

Good work by the way.

~~~
jbackus
This should work:

    
    
        #!/bin/bash
        cmd="$(cut -d ' ' -f 1 <<< "$@" )";
        args="$(cut -d ' ' -f 2- <<< "$@" )";
        url="http://explainshell.com/explain/$cmd?args=$args"
        explanation="$(curl -s $url | grep '<pre' | sed -E 's/<\/?[a-z]+(\ [a-z]+=\"[a-z0-9]+\")*>//g' | sed -E 's/^\ +//g')"
        echo "$explanation"
    

Example:

    
    
        ~$ tar xzvf archive.tar.gz
        ~$ explain !!
        explain tar xzvf archive.tar.gz
        The GNU version of the tar archiving utility
        -x, --extract, --get
        -z, --gzip, --gunzip --ungzip
        -v, --verbose
        -f, --file ARCHIVE
    

Save in /usr/local/bin. Gist:
[https://gist.github.com/backus/6387485](https://gist.github.com/backus/6387485)

~~~
Toenex
Beautiful. Thank you.

------
rkangel
Absolutely fantastic. That's going to be SO helpful.

I really like what you've done with the colour coding and the lines to link up
the boxes of explanation with each argument, but it is still a little
difficult to see what's going on when you've got lots of args. With an example
long command line [1] (the sort this is really useful for) I kept having to
scroll up and down from the whole command line to the explanation.

Perhaps when you do the hover over, rather than just greying out the other
options, you could hide all of them and only show the relevant one. It would
be right at the top, on the same screen as the argument (assuming you had the
res). You'd get explanation of the particular arg, along with it's position
and context.

[1] rsync -vzrc -e ssh --exclude .svn --exclude *~ code/
target:/root/application

------
talles
Loved it.

Just a tip: when the command got a long man page you can't really see the line
matching the option in the top of the page and the man section down on screen.

If you guys could keep the command on the top of the screen even when scrolled
down that would be superb.

~~~
idank
I actually experimented with that but getting the lines to display correctly
when scrolling was tricky.

------
jvandyke
This is amazing! Reminds me of
[http://www.regexper.com/](http://www.regexper.com/).

I hope you made it maintainable, there's going to be a lot of feature requests
coming your way :)

------
ekyo777
Awesome, thanks for that.

I almost never use my browser full screen, often it's snapped with 50% width.
In opera chrome and firefox, the width is a tad too big for that. It'd be nice
if it was more fluid.

Well that improvement wouldn't be that useful for me tough, I'm going to
integrate your website in zsh and emacs if someone doesn't do it before me.

Also, if you can make it so that the input area is still there when showing
results, it saves a frequent 'back' in the case someone wants to check
multiple commands.

~~~
idank
Will do, thanks!

------
mrcharles
This is really awesome, but what would be awesome as well would be the
inverse, which is the ability to find potentially relevant options from man
pages based on keywords.

This site is a great idea for when you want to find out what some command line
black magic is, but the inverse would be great for "I know this command line
tool can do this thing, but I don't know the options".

------
mratzloff
Hey, this is great! I've got a few ideas for improvements:

1\. Fix the layout. At least in Safari, you can only see the beginning and
ends of the colored lines.

2\. Pipe support (as mentioned by Ecio78)

3\. Additional Unix flavors. Red Hat and Mac OS X would probably be the top
suggestions.

~~~
idank
I don't have a Mac but I hope once the code is open someone will jump on it.

Re other flavors: do you know if there are easy ways to get a dump of all man
pages, similar to Ubuntu's man page archive?

~~~
mratzloff
It would probably be easiest to extract them directly by running the OS in a
VM. But I suppose crawling them is your next best option.

Fedora:
[http://linuxmanpages.net/manpages/](http://linuxmanpages.net/manpages/)

OS X:
[https://developer.apple.com/library/mac/documentation/Darwin...](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/)

------
ekyo777
Simple script to call it from command line. Uses bash, works from other shells
as long as its installed. Require curl to be installed.

[https://gist.github.com/ekyo/6389069](https://gist.github.com/ekyo/6389069)

------
mdisraeli
How well does it handle differences between various shells and their common
tools?

Since it's called "explain /shell/", I just had to try inputting some basic
powershell as a test - Sadly it's not supported yet :P

~~~
ygra
At least for PowerShell such a thing should be trivial as the shell knows
about its commands and parameter parsing is consistent for every command.

I'll probably try hacking something together on the weekend for PS.

~~~
mdisraeli
Awesome, thank you! :D

~~~
ygra
Not progressing as fast as I'd like. Stumbled over a few roadblocks regarding
parameter binding. The rules are complex (and underdocumented, I might add.
E.g. why man gci displays three parameter sets, or why -fi always binds to
-filter and not to -file is unknown to me. CmdletInfo.ResolveParameter sounds
like being great in theory but is unusable in practice. At least when I'm done
I might be able to revisit parameter binding in Pash. Preliminary repo is at
[https://github.com/ygra/ExplainPS](https://github.com/ygra/ExplainPS) – once
parameter binding is done the rest is just fetching information from Get-Help
which is the actual trivial part.

------
vsbuffalo
It appears to be incorrect with oddities like ps aux, which isn't the same as
ps -a -u -x since -u expects a user list (and then reads in -x as a user). But
this is a weird case because of backward compatibility.

------
taejo
A nice-to-have would be to have explanations for non-option arguments: for
example, for "sed -e s/foo/bar/", give "replace foo with bar", and even feed
foo into a regex explainer.

------
mixmastamyk
The fish shell does some (if not all) of this interactively at the terminal.

~~~
toupeira
What's the name of this functionality in fish? Might be worthwhile to install
it just for that.

~~~
mixmastamyk
Not sure, but you just hit tab, i.e.:

    
    
        echo -<tab>
        --help        (Display help and exit)  -E  (Do not use backslash escaped characters)  -n  (No newline)  
        --version  (Display version and exit)  -e         (Use backslash escaped characters)

------
lttlrck
Nice. It's a shame it can't break down the more complex ssh/scp parameters
such as -L [bind_address:]port:host:hostport Those are the commands I'm
_always_ Googling...

------
Groxx
Oh wow, this is great. Many thanks! I intend to use it :) I see you already
have most of my requests listed, so I'll just leave it at that and say
"awesome work".

------
caioariede
Such a very useful service, thanks! :)

It would be nice if it could detect inner commands and do a recursive parsing.
Example command:

find . -name ".txt" -exec iconv -f ISO-8859-1 -t UTF-8 {} \;

------
bnegreve
Cool I also like it!

A couple of things that it should also support (or simply ignore):

\- shell variables

\- shell constructs (for, while, | & > ...)

\- arguments followed by values without space (e.g. gcc -O3 -lmath ...)

~~~
idank
An argument to an option without a space does work. When they don't it's
because the option was incorrectly analyzed as having no arg.

------
idank
Which open source license would you recommend I use?

~~~
unnuun
The GNU GPL.

It allows you to share your code with other people who share their code, but
not people who don't. Afterall, why would you want to share your work with
someone who doesn't share theirs?

Great work, thouGh. Seriously, nice job!

------
cobbzilla
this is super cool.

curious where to submit bug reports? for the heck of it I put in a long ffmpeg
command; it detected the "-i" argument twice and the lines look a bit crazy.

here's the line to reproduce the error: ffmpeg -i infile -s 640x480 -vcodec
h264 -r 30 -b 450k -profile:v baseline -tune film -bufsize 2000k -maxrate 550k
-an -t 16.0 -passlogfile infile.mp4.passlog -pass 1 -y outfile.mp4

~~~
idank
Hi,

I'll open source the code sometime in the future so github would be the place
for bug reports.

As for long commands, yeah they're a bit crazy right now since they get
wrapped. I'm still trying to figure out the best way to display it.

------
fudged71
This is freaking amazing for educational purposes. I wish I had this years ago
in software engineering.

------
trolleibusov
Will show this tool to my students in the Shell course. Thanks!

------
peterwwillis
Wow! This is a genuinely new and very useful idea! Congrats

------
ArtDev
Useful, thanks!

I will send this to a friend who needs to learn the basics.

------
joleX
Great idea, nicely done. Went to my bookmarks!

------
tiziano88
brilliant!

------
tudborg
awesome! very good job!

