
Exploring CLI Best Practices - kddeisz
http://eng.localytics.com/exploring-cli-best-practices/
======
lucb1e
Good and useful advice. Just one minor thing I'd change (copied from a comment
that I posted there too):

Don't use -v for version. It's very uncommon (though I know software that does
it, it's confusing) and people will often blindly add -v for verbosity. A
better alternative is -V (capitalized) and of course --version. These should
both work, just like -h and --help should both work.

~~~
shanemhansen
Funny story about -v. I was using pkill to get rid of some hung processes on a
remote server. For some reason I decided to get verbose output and run pkill
-v.

Turns out -v negates the match so I killed every process on the box but one.

I had to email someone to reboot the server.

~~~
matt_wulfeck
Oh my gosh. I'm so glad I just read that. For the life of me I can't
understand why someone would want to say "let me send a kill to every single
process on this host but this one..."

~~~
__derek__
No idea, but there it is:

    
    
       -v          Reverse the sense of the matching; display processes that do
                     not match the given criteria.
    

I'm similarly happy that I read that before I went off and did something
stupid.

------
rossy
I'd expand on this and say every command line program should support GNU
getopt_long() syntax, since this is the most popular syntax for command line
interfaces and it saves having to guess the syntax when using a program for
the first time. This doesn't mean all programs should use getopt_long(), but
they should understand all the syntax that getopt_long() understands, such
that someone could reimplement the program's command line interface using
getopt_long() and get exactly the same behaviour.

This means, for example, supporting both space-separated and equals-separated
option values (--name value and --name=value). Some programs only support one
or the other, so you have to guess which will be supported when invoking a
program for the first time. Both forms have their uses too. --name=value is
more explicit and should throw an error if the --name option doesn't take a
value, so it's good for use in scripts. On the other hand, a lot of shells
don't tab-complete file names unless the file name is in a separate argument,
so for interactive use, --name value can be more convenient.

~~~
Annatar
_I 'd expand on this and say every command line program should support GNU
getopt_long() syntax, since this is the most popular syntax for command line
interfaces and it saves having to guess the syntax when using a program for
the first time._

One of the greatest crimes which GNU is not UNIX has committed are the --long-
options, and that horrible practice stems from the fact that command line
applications on GNU/Linux have poorly designed usage displays, and even worse
(often non-existent) manual pages, and when there is a manual page, it lacks
an EXAMPLES section, in stark contrast to UNIX systems (Solaris, HP-UX and
SmartOS are exemplary).

Ease of UNIX and efficiency thereof comes from brevity, and that particularly
concerns command line options: the lowest amount of typing possible is the
goal.

The solution is not to invent an arbitrary and horrible convention, but to
design better usage output, and to deliver high quality manual pages with lots
of examples.

Treat the root cause, not the symptom.

~~~
lobster_johnson
The quality of GNU documentation has absolutely nothing to do with the merits
or demerits of option styles. Not sure why you conflate the two.

The GNU style makes things more explicit and readable, which is why people
love it -- long options have been implemented far and wide outside of GNU.

And it has one nice feature: It lets you treat short options as concatenable
(so instead of "-v -f" you can just do "-vf") without ever accidentally
conflicting with long options.

~~~
Annatar
Because --long-option is way to cop out of writing a good quality usage output
and a comprehensive manual page, as well as putting undue burden on the user
to type more than is or should be necessary. Why do you think move is mv, list
is ls, copy is cp, and remove is rm? And how long did it take to learn cp -p,
ls -al, or rm -i? Ergonomy was and remains an extremely important topic in
UNIX usage, and --long-option something or --long-option=something completely
undermines that.

-vf usage has absolutely nothing whatsoever to do with GNU getopt; it is the core functionality of both getopt(1), getopts(1), getopt(3C), and getopts(3C).

~~~
Ar-Curunir
No, the names are shortened because the original Unix was meant to be used
over slow connections, and every character not typed was time saved in command
execution. We don't have those concerns now, and so to make code unreadable
for the sake of conciseness is not really useful.

~~~
Annatar
The names were shortened becuause the terminals had keyboards very similar to
a typewriter, and stdout / stderr was a built-in printer, so mistakes were
costly, and time consuming:

[http://www.tpsoft.com/museum_images/Teletype%20ASR-33%201.JP...](http://www.tpsoft.com/museum_images/Teletype%20ASR-33%201.JPG)

[https://en.wikipedia.org/wiki/Ken_Thompson#/media/File:Ken_T...](https://en.wikipedia.org/wiki/Ken_Thompson#/media/File:Ken_Thompson_\(sitting\)_and_Dennis_Ritchie_at_PDP-11_\(2876612463\).jpg)

the ergonomy came first and foremost, not the slow links, and the ergonomy
still plays an extremely important role on UNIX and UNIX-like systems, even
today, because more typing and mistakes are still time consuming; those
factors haven't gone away, and no link, however fast, can help with that.

Ergonomy is extremely important on UNIX. It's one of the major reasons for
preferring the command line as opposed to a graphical user interface.

To test this: run non-customized /bin/ksh and see how fast you're able to type
long command names without [TAB] completion and command history, and what your
input error rate will be.

You can even use a modern shell, like tcsh or zsh which have extremely
powerful command history and autocompletion facilities, and see how well you
fare with the shell completion of --some-long-option=/path/to/somewhere. You
can even use bash and see how well it will complete --some-option for you.

The message here is: if you're developing a command line application, please
do not use --long-options, and please consider ergonomy very carefully. Save
your users typing, and think about how you can reduce their error rates.
--long-option isn't the way to do either of those things, it actually makes
things more strenuous and slows one down.

------
okdana
>Provide long, readable option names with short aliases.

It's unnecessary, and potentially even burdensome, to provide short aliases
for less commonly used options. `ls --quoting-style` doesn't need a short
option — you're almost never going to need it.

>Provide a version command that can be accessed by version, --version or -v.

Don't use `-v` for that. It's true that some utilities do, but i think even
more use `-V` — including most GNU tools, Python, PostgreSQL, cURL, OpenSSH,
iptables, iproute2, procps-ng, just about any Symfony Console app, &c. Even if
you don't need a verbosity option right now, you might some day, and it's best
to have that work as expected.

>Sometimes your script will take longer to execute than people expect.
Outputting something like ’Processing…’ can go a long way toward reassuring
the user that their command went through.

If you do print status messages like that, make sure to send them to STDERR if
your utility outputs any actual data (like a report or file listing). Same for
progress meters.

>Conversely, don’t exit with a nonzero status code if your CLI didn’t
encounter an error. Your cleverness will end up confusing and frustrating your
users, especially if -e is set.

Using `set -e` is the 'cleverness' here. It's a bad idea in all but a few
cases. Also, i'm not sure what qualifies as 'encountering an error', but i
think it's perfectly reasonable to return a non-zero status in certain non-
error conditions, like when `grep` doesn't find a match, or in other cases
where output is valid but empty.

~~~
liw
Stderr should only get error messages, and, arguably, warning messages. It
shouldn't get status and progress messages. Those should go to /dev/tty, and
the program should be gracefully quiet if run without a terminal, e.g., from
cron.

~~~
restalis
Shouldn't the "verbose" option take care of the presence of the status or
progress messages? Also, arguably, the error and warning messages are status
messages too, so I would say all the status messages, regardless of gravity,
belong to the same channel.

~~~
liw
Error and warning messages are not status or progress messages, at least not
in the context of this discussions, which is that a long-running computation
should indicate to the user that it's doing something.

~~~
restalis
Example: "warning, the x is in y condition and thus z will take place
affecting the computation in progress" or "error, could not perform x (out of
many x-like things to do, but I'll continue because you instructed me to)".
These are status messages in a long-running computation context.

~~~
liw
I disagree. They're a warning and an error message, and should go to stderr.

It may _also_ be useful to show them on the terminal, in case the user would
like to see them now, and has redirected stderr somewhere other than the
terminal. In that case a program might show them on /dev/tty as well.

------
vkjv
This is missing something that I highly recommend:

"Start with supporting stdin/stdout as the only input and output. This ensures
that it is composable with other utilities. You may find you never need
anything else"

Need to read a file? `my-cli < file`

Need to write a file? `my-cli > file`

How about read from a URL? `curl url | my-cli`

~~~
jpitz
While I agree that v0.1 should support stdin/stdout, I don't know that you are
serving all of your users well by limiting i/o to ONLY stdin/out.

~~~
lucb1e
The two applications for specifying files on the command line is when: they
actually do something with that file (e.g. move the file) or when you operate
on multiple files and/or directories (e.g. backup application).

Otherwise it still _might_ be useful, but in general it's kind of unnecessary.
It adds logic to your application that it doesn't need, which violates "do one
thing, and one thing very well" (albeit only a very small violation).

~~~
jpitz
ISTM that you're arguing from a perspective of intrinsic necessity. Your
argument is that, anyone can cat a file into my utility, if they need that
functionality.

Sure. However, for example, GNU sort doesn't work that way. Most utilities
don't work that way. Most utilities accept a file as an source of input, and
most of those don't act on the inode. That's the status quo.

~~~
marvy
sort is a bad example: to support sorting in-place, it has to know the file
name. Pity the user who typed

    
    
        sort < bigfile > bigfile
    

for said user just lost a file.

~~~
shabble
This has bitten me occasionally, even though I know the workarounds (tempfiles
or pipe-consumers like sponge(1)).

I'm wondering if there's any practical use for the behaviour, or if it's worth
hacking a shell such that it produces a warning/interactive confirm prompt for
it (or transparently buffers to DWIM maybe?)

~~~
eichin
It's pretty common to `set -o noclobber` in beginner dotfiles; doing deferred-
open-if-exists is an interesting idea that would probably get a lot of
resistance :-)

------
andrewstuart2
> 12\. Write to stdout for useful information, stderr for warnings and errors.

I would constrain this even further to "Write to stdout only if information is
useful as input to another program."

Even useful information for a human reader can make downstream integration
overbearingly complex if the information is intermingled with a lot of
extraneous, albeit human-readable, information. Machine-readable layouts
(structured somehow: csv, tsv, json, xml, etc.) are vastly more useful for
integration.

~~~
niftich
I strongly disagree. A CLI is fundamentally for the benefit of humans
interacting with the application, and not for the benefit of interprocess
communication. When run interactively, a CLI should print human-useful
information on the console.

Conversely, a structured way of exposing program outputs and state should be
the preferred way of interprocess communication.

Because of convention and deliberate design choices, on the Unix command line,
these two often find themselves in conflict. In my opinion, the solution isn't
to compromise human usability to support machine-consumability of outputs.

~~~
new_hackers
Disagree.

A CLI is fundamentally for executing from the command line. Thus, embracing
the power of the command shell is best practice.

If you want a human to see it, put it on stderr.

If you want a machine to see it, put it on stdout. Also I strongly agree with
making it "structured" output. At minimum, a line-oriented record output is
fairly easy to process downstream.

EDIT: If it is an interactive console application (REPL), then stdout is okay
i guess.

~~~
smhenderson
Would a good compromise be to provide -q|--quiet to suppress extra, human
readable messages when piping out to a different app?

~~~
dllthomas
Another option is looking at isatty. I'm not sure how I feel about either.

~~~
prodigal_erik
Please provide a flag that bypasses the isatty check. I use nohup for anything
slow or hard to rerun and then tail and/or grep the nohup.out file.

~~~
dllthomas
Yeah, I think that's definitely good advice. isatty can be a good way of
deciding which defaults to use, but it shouldn't be the only way of accessing
functionality. Note that the flag might say "pretend isatty said yes/no", or
there might be a few flags that independently mediate all of the things set
based on isatty.

------
slantview
I used to use Ruby for CLIs. Then I started using Golang and delivering
compiled binaries for each OS that I supported. This has been a game changer
for me. With the CLIs that I've worked on that are open source
[https://github.com/RiotGamesMinions/motherbrain](https://github.com/RiotGamesMinions/motherbrain)
and some other ones in Ruby, we consistently had issues with rubygems and
having people be able to run our CLI from version to version.

Using something like Golang for distributing compiled binaries means that as
long as they have the CLI, it will continue to work. With Ruby, Python, PHP,
etc, there is absolutely no guarantee that your application will work in the
future.

~~~
yannis
Go has been a game changer for me also. There is also a very good package
[https://github.com/spf13/cobra](https://github.com/spf13/cobra) that can make
the development of complex CLIs easier.

~~~
aikah
Go default flag package is pure garbage though. That's a fact. However, Go
cross compilation makes it easy to develop multi-os CLI tools. The downside is
the size of the executable, it's easy to reach the 50MB bar with Go binaries.

~~~
skj
Yes. The default "flag" package is trash. I like the Go team a lot, and I
think they did a great job with many things, but it seems to me like they just
said, "meh, get something that can work, usability is not something that
matters."

~~~
ptman
consider the history, it probably makes sense if you're used to research unix
and plan9

------
beefsack
One trend I've really not appreciated in recent years is neglecting man pages
for command line utilities.

Having a `--help` option is great as a cheat sheet, but nowadays these help
outputs are much too large and hard to navigate because.

Alongside complete man pages, help commands can be helpful high level
overviews focusing on common use cases with examples.

~~~
dredmorbius
That's been a long-standing beef of mine for well over a decade. There are
several apparent sources of this.

The GNU project deprecates man in favour of info. This is a category error for
numerous reasons. The good news is that the documentation still exists, man
can be configured to fall back to 'pinfo' or similar, and projects (Debian are
particularly good about this) can schlep the Info format into a manpage.

There's an argument in favour of info: it was a hypertext document format
which is, arguably, more powerful than man. On the other hand, it has a
format-specific reader (info), there's a far more dominant hypertext document
format (HTML), and there are utilities to provide manpages in HTML format and
over a local HTML server
(dwww:[https://packages.debian.org/stable/doc/dwww](https://packages.debian.org/stable/doc/dwww)).
GNU should have bailed on this decades ago.

Several projects, notably GNOME (a GNU project) and many Red Hat utilities,
lack good / updated / any manpages. This is particularly frustrating. KDE
similarly fails frequently to provide manpages.

Again, Debian frequently will create and provide manpages, but they will
frequently run behind package development, so new features aren't clearly
documented.

Increasingly, standalone packages (say, imagemagick) don't fully document
themselves in manpages. Worse are vendor utilities which lack any manpage, a
useful --help or -? -h page, or anything else vaguely resembling sane
practices.

This is entirely inexcuseable.

------
fredrb
Very good article.

Another point that I think is valuable is: "Output one line per record data"

This is useful if your CLI that has to output data to the user. Makes it
easier to interface with programs like `grep`

~~~
gizmo686
I'd add to that an option to deliminate records with NUL instead of new line
and treat all other characters as literals.

------
rlpb
> 5\. Don’t have positional options.

But also, don't conflate options with arguments. Options should be _optional_
(the clue is in the name) and prefixed with - (single letter options) or --
(long form options). Arguments are typically positional, but as the author
says, they can become confusing if there are too many, or the ordering is non-
obvious, in which case the syntax may need a rethink.

A blanket "don't use positional arguments" would be tedious though. For
example, if I ignore what I say above, imagine having to type "cd
--directory=foo" or "cp --source=foo --destination=bar" all the time.

~~~
dllthomas
> "cp --source=foo --destination=bar"

Cf. `dd if=foo of=bar`

Not really disagreeing, just adding context.

~~~
slrz
Then maybe one should also add the bit of context that dd's command line
syntax was intended as a joke, a pun on some job control language of yore.

~~~
dllthomas
Oh, really? Interesting. While... odd, I always thought it appropriate to be
extra explicit with dd, where for some common uses you've the chance of nuking
the wrong _disk_.

Do you know which job control language of yore?

~~~
slrz
OS/360 JCL, apparently. At least Rob Pike wrote something to the effect on his
Google+.

 _dd is horrible on purpose. It 's a joke about OS/360 JCL. But today it's an
internationally standardized joke. I guess that says it all._

How the hell does one direct-link to a G+ comment? Well, cllck the thing that
says "View 384 bazillion previous comments" and then search for dd.

[https://plus.google.com/+RobPikeTheHuman/posts/R58WgWwN9jp](https://plus.google.com/+RobPikeTheHuman/posts/R58WgWwN9jp)

------
jakub_g
Semi-related: if you write CLI tools with nodejs, I highly recommend using
yargs and/or inquirer - yargs for robust command line arguments parsing, and
inquirer for interactively asking questions to the user.

[https://github.com/yargs/yargs](https://github.com/yargs/yargs)

[https://github.com/SBoudrias/Inquirer.js/](https://github.com/SBoudrias/Inquirer.js/)

When using these tools correctly (following readme), you get most of the stuff
written in the article out of the box.

~~~
paulddraper
Ew. Most programs shouldn't ask me questions.

They should print a usage statement if I haven't provided enough info.

~~~
flukus
Yep, they're commands, not conversations.

------
MereInterest
Most, I agree with, others, I don't.

> 1\. Every option that can have a default option should have a default
> option.

The exception to this are output files. A program should only output to a path
that has been explicitly given to it. Too many scripts litter their working
directory with output files.

> 8\. Don't go for a long period without output to the user.

If there nothing's wrong, then nothing needs to be printed. Especially once a
script gets called from another script, I don't want to have any output from a
script unless it requires my attention.

~~~
userbinator
_Most, I agree with, others, I don 't._

This is the fundamental reason why I've grown tiresome of the phrase "best
practices". They probably are the best you can come up with, but that doesn't
mean they're the best (or sometimes even close) for everyone else.

~~~
MereInterest
I think the discussions are useful, even if the conclusions aren't. Discussing
best practices gets everyone to bring up the pros and cons of each. Then, when
confronted with a new area, one can apply those arguments to figure out a
decent set of best practices for that particular field.

------
TheAceOfHearts
Related to CLI best practices, is there any good styleguide or "best
practices" list for formatting help and text output?

I've only written a handful of small CLI utils, and each time I end up looking
through tools I regularly use and adapt ideas from each. But this is more time
consuming than being able to run through a list of suggestions prepared by
someone with more experience in this domain.

~~~
okdana
POSIX talks about it a little bit:
[http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html)

Some argument-handling libraries like Python's argparse and PHP's Symfony
Console generate usage help for you, and they all use a relatively similar
formatting that i think was inspired by the formatting of man pages:
[http://man7.org/linux/man-pages/man7/man-
pages.7.html](http://man7.org/linux/man-pages/man7/man-pages.7.html)

docopt ([http://docopt.org/](http://docopt.org/)) is an attempt to standardise
it from the other way around — instead of you defining the options in the code
and it generating the usage help for you, you write the usage help and it
generates the code to define the options. But i don't know that it's super
widely used.

------
emmelaich
My favourite annoyance is using the option syntax for things that are not
optional.

If it really is a command, then require a command without the -- or -.

Today's examples:

1\. centos6/redhat6 /sbin/chkconfig; it needs one of --list or --del or ... so
why not just list or del or ?

2\. the kafka command line tools, e.g. kafka-topics. It requires --create or
--delete or --list ... So just use create/delete/list/... !

~~~
restalis
_" it needs one of --list or --del or ... so why not just list or del or ?"_

That's all good and everything but what about the parameters for which
optional is only the explicit digression from an existing default value?

------
chriswarbo
I find myself using environment variables much more than commandline arguments
when writing CLI programs:

\- Env vars provide a key/value interface, which is trickier to handle with
arguments (e.g. "get the argument immediately after '-f'").

\- Env vars are in-scope at the point they're needed, whereas non-trivial
arguments usually have to be parsed up-front all at once and passed into the
processing code somehow. Yes, it's good to check things up-front; we can check
mandatory/mutually-exclusive env vars up front too, with the bonus that we can
keep such 'invocation checking' separated from the rest of the processing.

\- Setting env vars in preparation for a subprocess call can be done
incrementally, by extending the environment over and over until we're ready.
In contrast, commandline arguments must be accumulated into a single
(correctly parsable) array.

\- Sharing env vars across a program and its children is easy (e.g. "VERBOSE=1
./foo.sh").

\- We can use env vars to control a program even when it's called deep in the
bowels of something else, without having to fiddle with all of the layers in
between.

I find this works very nicely, as long as we:

\- Treat the environment as append-only, i.e. no changing a variable once it's
been set. We can append new variables, e.g. "export FOO=bar", and we can
invoke processes with extra variables, e.g. "FOO=bar ./baz.sh", but we don't
unset or alter a variable.

\- Don't use env vars for communication within a process, i.e. we can use the
environment variables we're invoked with, and we can append new variables for
use by the programs we invoke, but within a program we should refrain from
setting variables which affect our own execution (since that causes the
coupling and spooky-action-at-a-distance which globals are notorious for).

What are others' thoughts on this?

~~~
vacri
> _(e.g. "get the argument immediately after '-f'")_

Is there a language commonly used for cli tools that doesn't have an inbuilt
options lib? Even bash has inbuilt 'getopts' to handle this for you.

    
    
        while getopts "ab:" option; do
            case $option in
                a) VAR="foo";;
                b) VAR2="$OPTARG";;
            esac
        done
        shift $((OPTIND-1))
    

which will allow you to "myscript -a -b myarg"

By all means, use env vars if it works for you, but it should be right tool
for the right job. Imagine trying to use 'ls' with env vars...

~~~
beefhash
> Is there a language commonly used for cli tools that doesn't have an inbuilt
> options lib?

Yes, namely C/C++ under the Visual Studio toolchain:
[https://stackoverflow.com/a/12689342](https://stackoverflow.com/a/12689342)

~~~
vacri
I stand corrected. I thought I might get burned by C, but didn't expect it to
bring Windows in on the action :)

------
iddogino
Reminds me of this super useful thread from SO
[http://programmers.stackexchange.com/questions/307467/what-a...](http://programmers.stackexchange.com/questions/307467/what-
are-good-habits-for-designing-command-line-arguments)

------
dredmorbius
As is typically the case in software, different standards tend to emerge from
different projects and their associated tools.

Two of the primary diverging Linux/Unix standards are FreeBSD (arguably the
older, from AT&T & BSD traditions) and GNU.

The GNU programming standards specifies standards for command line interfaces
in section 4.7, strongly influenced by GNU getopt(3).
[https://www.gnu.org/prep/standards/html_node/Command_002dLin...](https://www.gnu.org/prep/standards/html_node/Command_002dLine-
Interfaces.html)

This references the Table of Long Options (this needs to be worked in to Game
of Thrones): [https://www.gnu.org/prep/standards/html_node/Option-
Table.ht...](https://www.gnu.org/prep/standards/html_node/Option-
Table.html#Option-Table)

For FreeBSD, the equivalent guide appears to be (I'm not an expert at this, so
salt appropriately) the kernel source file style guide:
[https://www.freebsd.org/cgi/man.cgi?query=style&sektion=9&ma...](https://www.freebsd.org/cgi/man.cgi?query=style&sektion=9&manpath=freebsd-
release-ports)

There may be another source.

I've seen other significant projects impose their own argument styles. Among
those:

MIT/X11 has its own family of arguments.

Various toolkits seem to engender their own styles.

The GNOME and KDE projects have tended to both adopt idiosyncratic and largely
undocumented argument formats. I find both tremendously frustrating.

Major applications often develop yet more idiosyncratic command and argument
syntax. Chrome, Firefox, and LibreOffice come to mind.

I'm going to pretend Java doesn't exist at all. Nope. It's a myth.

The notorious 'dd' owes its syntax to mainframe JCL notation, from which it is
derived.

------
tomgagnier
Providing command line completion should be on this list!

~~~
Retra
How would you do that unless you were changing the CLI itself?

~~~
deathanatos
Modern shells, such as bash or zsh, support specifying completion through
external files or scripts that the shell can parse. Having never written one,
I'm not familiar with the exacts, but suffice it to say the right file in the
right location with the right contents can inform the shell as to how to auto
complete.

i.e., the auto-completion facilities are general / extensible.

Especially if many programs follow a general format of

    
    
      program subcommand arg arg arg --optional-flag --option
    

(my personal favorite, as I find it most clear; followed by e.g., argparse in
Python, git, many GNU utilities)

then it should be easy to see how a small specification of what subcommands
take what for args or options should be enough to enable a pretty powerful
auto-complete. (This is an example; I think zsh's autocompleters are actually
small scripts; see [https://github.com/zsh-users/zsh-
completions/blob/master/zsh...](https://github.com/zsh-users/zsh-
completions/blob/master/zsh-completions-howto.org))

(This, in zsh, combined with zsh's fuzzy autocomplete, is amazing.)

~~~
dllthomas
For bash:

    
    
        $ help complete
        complete: complete [-abcdefgjksuv] [-pr] [-DE] [-o option] [-A action] [-G globpat] [-W wordlist]  [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [name ...]
            Specify how arguments are to be completed by Readline.
            
            For each NAME, specify how arguments are to be completed.  If no options
            are supplied, existing completion specifications are printed in a way that
            allows them to be reused as input.
            
            Options:
              -p	print existing completion specifications in a reusable format
              -r	remove a completion specification for each NAME, or, if no
            	NAMEs are supplied, all completion specifications
              -D	apply the completions and actions as the default for commands
            	without any specific completion defined
              -E	apply the completions and actions to "empty" commands --
            	completion attempted on a blank line
            
            When completion is attempted, the actions are applied in the order the
            uppercase-letter options are listed above.  The -D option takes
            precedence over -E.
            
            Exit Status:
            Returns success unless an invalid option is supplied or an error occurs.
    
    

There are a lot of canned completion approaches, usually tweakable in small
ways, but for heavy lifting `complete -F foo bar` will call the function `foo`
when you are asking for completion of a command where the first word is `bar`.
Information on what words are already on the line, where the cursor is, etc is
passed in in shell variables starting with COMP_. See the "Programmable
Completion" section of the bash manpage for much more detail.

Note that it's settings for a specific shell process, not global across all
instances of bash. "The right file in the right location" is only relevant 1)
to set up your shells a particular way by default, and 2) if the function
called references a file.

~~~
dllthomas
Tying this back in to the earlier discussion, many distros provide standard
places for installed packages to drop scripts, which will be sourced during
shell startup and which can thus configure the shell appropriately each time.
"Providing command line completion" for your utility, therefore, means
providing any such scripts as are appropriate, and setting up your packaging
(or install scripts) to install them.

------
vacri
For in-house stuff, I also run by the rule that any tool should be safe to run
if there are no args provided. Run any of my tools, and if it's potentially
destructive, you'll only get a help screen if there are no args provided. Even
tools that don't need any args to function will still require a misc arg if
they might theoretically damage something.

The theory here is that these tools aren't well-known, so it's an extra level
of safety. It also means that colleagues don't have to hunt me down to ask
about what it does...

------
sigil
Great advice on the whole!

 _> 3\. Use common command line options._

The GNU options standards are good, but see also the "Command-Line Options"
chapter in "The Art Of Unix Programming" for some of the reasoning and history
behind these conventions. Actually -- if you're writing unix programs often,
do yourself a favor and read all of TAOUP. It really helps crystallize the
Unix Philosophy in a way that a grab bag of suggestions doesn't.

[http://www.catb.org/esr/writings/taoup/html/ch10s05.html](http://www.catb.org/esr/writings/taoup/html/ch10s05.html)

 _> 4\. Provide options for explicitly identifying the files to process._

Lots of options pointing to different files is a CLI smell. If your program
"does one thing and does it well" (the Unix Philosophy), it often works like a
filter: take one or more input files, apply the same transformation to them,
output the result. `cat`, `head`, `tail`, `grep`, `cut`, `sort`, `join`,
`sed`, `gzip`, `tar`, `cc`, `curl`, `md5sum` -- at the heart of all these
programs is a unix filter.

In this case, just have your program accept input files as positional
arguments beyond the last option. A nice side effect is that your program will
be `find | xargs` -friendly. That puts the user in total control of the
inputs. It also means they can parallelize your program with `xargs -P`!

Also consider whether you need input files at all! Can your unix filter
program just transform stdin to stdout? If so, it can be used to process
streams of data much larger than available disk space. Often you can trick
programs that require a file argument by passing `/dev/stdin`, but note that
`/dev/stdin` won't be accessible in some environments.

 _> 8\. Don't go for a long period without output to the user... Outputting
something like ’Processing…’ can go a long way toward reassuring the user that
their command went through._

I get the reasoning behind this, but as a heavy CLI user I'd prefer if
programs didn't crap progress all over the terminal by default. If I want
verbose progress, I'll pass `-v`. If I want even more insight into what's
happening, I'll pass `-v -v`.

See Rule Number 11 in TAOUP:

 _Rule of Silence: When a program has nothing surprising to say, it should say
nothing._

[http://www.catb.org/esr/writings/taoup/html/ch01s06.html](http://www.catb.org/esr/writings/taoup/html/ch01s06.html)

 _> 10\. For long running operations, allow the user to recover at a failure
point if possible._

One way to perform a sequence of expensive steps iff they haven't been done
yet ("recover at failure points"): orchestrate it with a Makefile.

~~~
vram22
>> 8\. Don't go for a long period without output to the user... Outputting
something like ’Processing…’ can go a long way toward reassuring the user that
their command went through.

>I get the reasoning behind this, but as a heavy CLI user I'd prefer if
programs didn't crap progress all over the terminal by default. If I want
verbose progress, I'll pass `-v`. If I want even more insight into what's
happening, I'll pass `-v -v`.

>See Rule Number 11 in TAOUP:

>Rule of Silence: When a program has nothing surprising to say, it should say
nothing.

Agreed. In fact, informally, that rule goes back to much before TAOUP was
written, I think. (TAOUP may have only formalized it.) I remember something
like it from classic The Unix Programming Environment book (UPE, I first read
it several years ago) which I referred to in another comment in this thread.

IIRC, in UPE it may have been phrased a bit differently, that's all -
something like:

if the program succeeds (in some cases, don't generate any output, at least on
stderr - of course if the program generates output as part of its normal
behavior, like filters and some other programs do, then it has to write to
stdout even if no errors). The behavior of the cmp command (compare two files)
is an example of that - it produces no output on a successful compare (the
files match), only on an unsuccessful one (the files differ).

Also, part of the reason for the brevity of Unix commands and terseness of
output, is supposed to be because the first Unix versions were actually
developed on teletypes (which were like the old telex machines) for output -
that actually printed the output as you typed commands, on rolls of paper.
Video screens came later. And the same is the reason for the Unix editor ed's
brevity, and even it's p(rint) command - you actually had to (physically)
print the changed lines after an edit, to even see them ... :-) All in all,
I'd say it was an even more incredible job to develop an OS like Unix under
such constraints.

There is also pipe viewer. Peteris Krumins wrote a post about it a while ago:

[http://www.catonmat.net/blog/unix-utilities-pipe-
viewer/](http://www.catonmat.net/blog/unix-utilities-pipe-viewer/)

You insert it at points in the pipeline and it lets you see the progress of
the pipe.

And totally agree with your recommendation of TAOUP. The wording can be a
little ornate / verbose at times, but that is just ESR's style. Worth
tolerating it for the content.

------
ozten
Great post. An amazing opt library is docopt[1]. Instead of writing a lot of
code, with docopt you write your usage doc based on long standing best
practices and docopt parses that USAGE block.

[1] [http://docopt.org/](http://docopt.org/)

------
dllthomas
I'd add "if you're going to output locations (within files), format it
conventionally - a line starting with "filename:line: " or
"filename:line:column: ". Like `grep -n` or most compile errors.

------
ryanlm
I'm hoping the angular cli will be able to send email soon.

------
endgame
Also: Write a manpage.

------
vram22
Also, -- (two dashes) in the command line means "end of the options", which
allows, e.g. rm to delete a file starting with dash.

------
inlined
Good feedback. I have one comment and one question:

1\. I personally prefer to present a terse message to stderr and stack traces
to a debug file. This helps a user give support more information when they're
hitting transient issues.

2\. I fully agree with the recommendation for a --dry_run option, but I think
the output needs to be actionable. Do people have good examples of actionable
--dry_run output that lets the user verify their intent? E.g. The list of
files that would be deleted by "rm -rf"

------
newman314
+1 to Option #1

I'd argue that good software has what I call "good factory defaults".

Pine was a good example of this. It allowed people to use email right out of
the box with plenty of options for customization in it's config settings.

~~~
dredmorbius
No idea why the hivemind is rejecting this, but yes, absolutely.

Defaults should be sane, non-destructive, intuitive, and suit the common case.
_People don 't change defaults_, and actions which _can_ be destructive should
_not_ be easy to invoke accidentally. Even (or especially) on a CLI.

------
ams6110
This was all figured out in the 1980s.

~~~
vacri
New techies are coming online all the time.

