
Help Message for Shell Scripts - reconquestio
https://samizdat.dev/help-message-for-shell-scripts/
======
gorgoiler
The power is out on your boat, again. It’s 3am. You suspect that, again, the
alternator housing has come loose.

You duct tape a flashlight to the bulkhead so you can work hands free and
actually see what you are doing. All you have on you is a broken pocket knife
but it’ll do because all you need to accomplish right now is to tighten the
housing screws _enough_. You know this for a fact because you’ve done it three
times already in the last 24 hours.

It’s not even a documented procedure — you’ll replace the housing mounts
entirely when you’re back at port in three days’ time. You guarantee it — this
is the first thing you’ll do even, when you get back to shore. You have my
word on that, captain!

The duct tape came unstuck. It was damp and doesn’t work so well (at all) when
it’s wet. The flashlight survived the fall. More tape this time should do the
job. Tape mount version 2 will still unstick of course, eventually. Nothing
stops the damp at sea, but if you use enough tape then you’ll have fixed the
power by the time the tape fails. That’s your plan B and you’re sticking to
it.

Sure, you could do this job better if you had an impact driver with an
automatically illuminated bit chuck, but buying one of those is further down
the _todo_ list than fixing the power on the boat, making it back to port, and
ensuring the power doesn’t fail this way again, as promised. Or at least won’t
fail for the next few shifts.

On your days off you relax by programming in Bash.

~~~
chrisweekly
+1 for the sailing analogy!

"Necessity is the mother of invention" must have been coined by a sailor.

------
adrianmonk
You can also use a "here document"

    
    
        help() {
          cat <<'EOH'
        my-script — does one thing well
        
        Usage:
          my-script <input> <output>
        
        Options:
          <input>   Input file to read.
          <output>  Output file to write. Use '-' for stdout.
          -h        Show this message.
        EOH
        }
    

If the indentation bugs you, you can use a simpler sed trick to remove leading
space so that you can indent it as desired:

    
    
        help() {
          sed -e 's/    //' <<'EOH'
            my-script — does one thing well
            
            Usage:
              my-script <input> <output>
            
            Options:
              <input>   Input file to read.
              <output>  Output file to write. Use '-' for stdout.
              -h        Show this message.
        EOH
        }

~~~
pavon
Or just a multiline string:

    
    
      #!/bin/bash
      USAGE="my-script — does one thing well
        
        Usage:
          my-script <input> <output>
        
        Options:
          <input>   Input file to read.
          <output>  Output file to write. Use '-' for stdout.
          -h        Show this message.
      "
    
      help() {
        echo "$USAGE"
      }
    

This is my standard approach which is cleaner for putting the documentation at
the very top of the file like the linked article.

~~~
sillysaurusx
Thank you! I had no idea that multiline strings were valid bash.

~~~
rkangel
It's the same logic that allows you to type:

git commit -m "First line of commit

Second line of commit"

That's a multi-line string in bash.

------
j1elo
I learnt the same trick some years ago, from an article called _Shell Scripts
Matter_ :

[https://dev.to/thiht/shell-scripts-matter](https://dev.to/thiht/shell-
scripts-matter)

So I took some of the advice and tips offered in there, and wrote a template
file to be used as a baseline when writing scripts for any project that might
need one:

[https://github.com/j1elo/shell-
snippets/blob/master/template...](https://github.com/j1elo/shell-
snippets/blob/master/template.sh)

Other resources that I link in the readme of that repo, because they were a
great guide to write better and more robust scripts, are:

\- _Writing Robust Bash Shell Scripts_ :
[https://www.davidpashley.com/articles/writing-robust-
shell-s...](https://www.davidpashley.com/articles/writing-robust-shell-
scripts/)

\- _Common shell script mistakes_ :
[http://www.pixelbeat.org/programming/shell_script_mistakes.h...](http://www.pixelbeat.org/programming/shell_script_mistakes.html)

\- _Bash Pitfalls_ :
[http://mywiki.wooledge.org/BashPitfalls](http://mywiki.wooledge.org/BashPitfalls)

\- _The Bash Hackers Wiki_ : [https://wiki.bash-
hackers.org/](https://wiki.bash-hackers.org/)

EDIT: -for anyone who would like to read some actual examples- I have to
manage a bunch of scripts so actually a slightly more up to date version of
the template is put into practice by means of a common bash.conf file that
then gets sourced by all scripts: [https://github.com/Kurento/adm-
scripts/blob/master/bash.conf...](https://github.com/Kurento/adm-
scripts/blob/master/bash.conf.sh)

~~~
themodelplumber
Thank you for this really helpful comment. It's like an encyclopedia's worth
of bash information in one go--much appreciated.

~~~
j1elo
You're welcome! Shell scripting has a _weird_ language, unsafe by default, and
very prone to mistakes... but knowing it well pays off.

People say that for complex things it's better to write Python, but that
doesn't fly in embedded or Docker environments. Python is not even present in
the default Ubuntu Docker images. Also if all you want to do is really write
glue code between CLI programs, shell scripting is the way to go.

Happy coding!

~~~
mercer
I'll second the thanks. Thanks!

------
mey
Handling of arguments is one of the reasons I reach for Python or Powershell
instead of a bash script when writing my own stuff.

[https://docs.python.org/3/library/argparse.html](https://docs.python.org/3/library/argparse.html)
is great.

Powershell has the Param keyword that functions like argparse in Python

[https://docs.microsoft.com/en-
us/powershell/module/microsoft...](https://docs.microsoft.com/en-
us/powershell/module/microsoft.powershell.core/about/about_functions_advanced_parameters?view=powershell-7)

~~~
Spivak
But handling args isn't that bad in bash.

    
    
        while [[ $# -gt 0 ]]; do
          case "$1" in
             -h|--help)
               do_help
               exit
               ;;
             -v|--version)
               do_version
               exit
               ;;
             -d|--debug)
               debug=true
               shift
               ;;
             -a|--arg)
               arg_value=$2
               shift 2
               ;;
          esac
        done

~~~
mey

        import argparse
        parser = argparse.ArgumentParser()
        parser.add_argument('-v','--version',action='version', version='demo',help='Print version information')
        parser.add_argument('-d','--debug', help='Enable Debug Mode')
        parser.add_argument('a','arg', help="Argument Documentation")
        args = parser.parse_args()
    

Personally I feel like this is more readable code, gets me better validation,
and help docs for "free". That's the attraction.

~~~
gitgud
Elegant, but then it's no longer a basic shell-script as it requires python
installed.

If you can live with additional dependencies, then I like the node [1]
commander package, which is very readable and nice to work with in my opinion.

    
    
            #!/usr/bin/env node
            const { program } = require('commander');
    
            program
              .command('clone <source> [destination]')
              .description('clone a repository')
              .action((source, destination) => {
                console.log('clone command called');
              });
    

It also automatically generates the --help output for ./script -h

[1] [https://github.com/tj/commander.js/](https://github.com/tj/commander.js/)

------
diablerouge
This seems like a neat sed trick, but I'm not sure that it's useful for this
particular case?

When I write a shell script, I often write a help function if it's not a
totally trivial script, but there's no need for this cryptic sed expression,
right? You can just call `echo` a few times and do it the obvious way. That
works better for maintainability and if you put it at the top of the file then
it's immediately visible when opening or using `head` on it.

Neat trick though - sed is super useful for all kinds of things. I had a co-
worker who bound a keybinding to a sed one-liner that would automagically
reconfigure some files that needed to be changed regularly. I ended up using
that trick to allow for changing my terminal console colorscheme with a single
command.

~~~
e12e
It's a little sad that standard shell here documents only support elliding
leading tabs (it wouldn't be so sad if the record separator hadn't been thrown
under the bus - having a character for indentation distinct from space is
good... In theory).

But at any rate my typical usage() is generally along these lines (warning
watch out for expansions):

    
    
      usage()
      {
        cat - <<-EOF
        `basename ${0}`: demonstrate here docs
         Usage:
         `basename ${0}` <required argument> [-o|--optional-param] 
    
           Etc. Possibly referencing
           default value: ${DEFAULT_VALUE}
        EOF
      }

~~~
jolmg
I think it'd be better without using basename, just $0. That way it matches
the way it was called, which is how the user chose to access it for whatever
reason. The bare filename might refer to a different command, even. Also, if
you include examples in the help text, they'll also be able to copy and paste,
instead of having to manually insert what basename stripped away.

~~~
e12e
True enough - I find it depends a bit on the nature of the script - if it's
something buried under misc/tools/extra/bin/util.sh - i tend to prefer brevity
- especially in the first paragraph/usage section (util.sh <required param>
[optional param]).

But for more concrete examples I'll often leave off the basename - for easier
cut and paste.

------
dougdonohoe
We do this for Makefile entries - looking for '##' that we put before each
make command.

    
    
      ## help: prints this help message
      help:
         @echo "Usage: \n"
         @egrep -h "^## [a-zA-Z0-9\-]*:" ${MAKEFILE_LIST} | sed -e 's/##//' | column -t -s ':' |  sed -e 's/^/ /'
    
      ## build: builds JAR with dependencies
      build:
         mvn compile

------
xvolter
I also posted to the github gist, this the sed command here is not cross-
plataform friendly. You can accomplish the same thing with an awk command
though:

awk '/^###/' "$0"

~~~
pwdisswordfish2
sed -n '/^###/p' is cross-platform friendly.

~~~
u801e
This will also work:

    
    
      sed '/^###/!d'

~~~
pwdisswordfish2
This will also work:

    
    
       sed '/^#\{3\}/!d'

------
account42
> $0 means a filename of a file that is being executed.

This is only a convention and is entirely up to the calling program.

For example in bash scripts you can use `exec -a name ...` to pass "name" as
the 0th argument.

If you are already using #!/bin/bash you might as well use ${BASH_SOURCE[0]}
to get the path to the current script.

------
xelxebar
This reminds me of a nice sed one-liner I recently happened to craft.

Do you ever collect families of functions in your shell scripts under
different sections? Here's a nice way of printing out all the functions under
a given section:

    
    
        funs(){ sed -n '/^## /h;x;/'"$1"'/{x;s/^\(\w\+\)().*/\1/p;x};x' "$0";}
    

Where "sections" are delimited by comments of the form "## Section Name" at
the beginning of a line. A particularly nice use case is when you write
scripts that expect "subcommand" arguments, like

    
    
        $ foo.sh bar baz
    

and wish to keep track of the available subcommands in the help documentation.
Simply collect all your subcommands under the heading "## Subcommands" and
stick a funs call in your documentation:

    
    
        usage=$(cat <<USAGE
        Usage: foo <subcommand>
        Subcommands: $(funs Subcommands)
        USAGE
        )
    

The sed one-liner above uses the oft-ignored "hold space" which lets you store
data that persists between lines. Here's the same sed but expanded with
comments:

    
    
        funs(){ sed -n '/^## /h  # Store header line in hold space
            x               # Swap out current line with header in hold space.
            /'"$1"'/{       # Run block if last encountered header matches $1
                x           # Return to processing current line (instead of header)
                s/^\(\w\+\)().*/\1/p    # Print function names
                x           # Whether or not this block runs, we want to return to
                            # processing the current line. If the block does not
                            # run, then the hold space contains our current line
                            # with the active line being our header. So we must
            }               # return to that state as whell when the block is run.
            x               # Restore current line from hold space' "$0"
        }

------
fomine3
I'm particular about: If I run a command and its arguments is wrong, it should
output error and help messages to STDERR. But if I run a command with --help
argument, it should output help messages to STDOUT.

------
ahnick
I like the idea of combining the header with the help documentation to reduce
the number of areas to maintain in smaller scripts. For larger scripts though,
I think I'd still prefer to have a separate function, so that the help
documentation doesn't overwhelm the initial viewing of the actual code.

I also like to feed a heredoc directly into man, which allows you to achieve
nicer formatting for the help documentation. Something like this...

    
    
      man -l - << EOF
      .\" Manpage for encpass.sh.
      .\" Email contact@plyint.com to correct errors or typos.
      .TH man 8 "06 March 2020" "1.0" "encpass.sh man page"
      .SH NAME
      encpass.sh \- Use encrypted passwords in shell scripts
      ...
      EOF
    

See encpass.sh for a working example of this ->
[https://github.com/plyint/encpass.sh/blob/master/encpass.sh](https://github.com/plyint/encpass.sh/blob/master/encpass.sh)

~~~
sicromoft
Note that this doesn't work on macOS, where the builtin `man` command doesn't
support the `-l` option.

~~~
ahnick
Ah interesting, is there any workaround for Mac? Otherwise, I may just have to
fallback to stripping the man page formatting and sending it to less.

~~~
gpanders
The actual `man` command does this:

    
    
        /usr/bin/tbl | /usr/bin/groff -Wall -mtty-char -Tascii -mandoc -c | /usr/bin/less -is
    

So you could do it "manually" that way. Not the cleanest or prettiest solution
but it's much lighter weight than using something like pandoc.

EDIT: Full example:

    
    
        { /usr/bin/tbl | /usr/bin/groff -Wall -mtty-char -Tascii -mandoc -c | /usr/bin/less -is; } <<EOF
        .\" Manpage for encpass.sh.
        .\" Email contact@plyint.com to correct errors or typos.
        .TH man 8 "06 March 2020" "1.0" "encpass.sh man page"
        .SH NAME
        encpass.sh \- Use encrypted passwords in shell scripts
        ...
        EOF

~~~
ahnick
Thanks, this worked well on Mac.

------
hansdieter1337
Even better: Don’t use bash. I started using Python instead of bash. It’s way
better to read and more maintainable. If I need the performance of native-Unix
commands, I can still use them using subprocess.

~~~
ben509
Python is surprisingly bad at managing subprocesses, though. Read the
subprocess docs closely, it's quite easy to get deadlocks if you try to
compose operations the way you can in bash.

~~~
aflag
There are libraries that fix that:
[https://pypi.org/project/sh/](https://pypi.org/project/sh/) if you are in
position to use external libraries.

------
raggi
I used this strategy in "fx" which is a development helper frontend for
fuchsia builds and tools. I used four # for the "short description" and three
for the long description. The reason I used the strategy there is that lot of
our scripts delegate arguments to native commands, and so adding help purely
to --help wasn't really a good ROI. Implementation:
[https://fuchsia.googlesource.com/fuchsia/+/refs/heads/master...](https://fuchsia.googlesource.com/fuchsia/+/refs/heads/master/scripts/fx#40)

------
jandrese
Hmm:

% sed -rn 's/^### ?//;T;p' testfile

sed: 1: "s/^### ?//;T;p": invalid command code T

Looks like it might need GNU Sed or something. But honestly if I want to read
the top of the file less works just as well.

~~~
adrianmonk
Yeah, "man sed" on my machine says, "This is a GNU extension."

You could do the same thing with awk instead:

    
    
        awk '{ if (sub("^### ?", "")) { print; } else { exit; } }'

~~~
bewuethr
Or

    
    
        sed -rn 's/^### ?//p'

~~~
e12e
Doesn't appear anyone has tried addressing before replacement - ie the
simplest sed work-a-like - if you don't mind the leading ### is just:

    
    
      sed -n '/^### /p' 
    

I believe? (equivalent to grep).

Then eg:

    
    
      sed -nr '/^### /s/^.{4}(.*)/\1/p'
    

(or without the redundant addressing, just:)

    
    
      sed -nr 's/^### (.*)/\1/p'

~~~
bewuethr
You can simplify

    
    
        sed -nr 's/^.{4}(.*)/\1/'
    

to

    
    
        sed -nr 's/^.{4}//
    

And if you use a pattern for the address, you can repeat it in the
substitution by using an empty pattern, so

    
    
        sed -nr '/^### /s/^.{4}(.*)/\1/p'
    

is the same as

    
    
        sed -nr '/^### /s/^.{4}//p'
    

is the same as

    
    
        sed -nr '/^### /s///p'
    

at which point I prefer just the substitution:

    
    
        sed -nr 's/^### //p'

------
heinrichhartman
I like to do:

    
    
        help() { cat $0 }
    

"May the source be with you." : )

------
OliverJones
Cool. I did this on 7th Edition UNIX in 1977. I forget how. It's interesting
that ....

* _NIX makes people do creative things

_ All that Bell Labs stuff still works the way it always did.

* People are still reinventing this particular wheel, and

* This embedded help stuff still somehow hasn't made it into the infrastructure along with autocomplete.

------
jeffrom
Something I've wanted to do for a while is build a library to parse and
generate use lines / documentation that adheres to the posix useline spec
(can't find the link at the moment) while also being able to idempotently
(de)serialize and be descriptive enough to define arguments and flags in a way
a human could easily understand. iirc the spec seemed probably too vague to
just work with all the currently existing man pages, but it would be nice to
have a spec all programs can follow that machines can parse on my os.

~~~
layoutIfNeeded
[http://docopt.org/](http://docopt.org/) ?

------
andsens
My time to shine! I built an argument parser that uses a POSIX compliant help
message as the input. It's a parser generator really. It generates minified
bash that is inlined in your script, so no dependencies. The work is based off
of docopt (and is docopt compliant). Check it out:
[https://github.com/andsens/docopt.sh](https://github.com/andsens/docopt.sh)

------
flaxton
Didn't work on macOS (multiple sed errors - switches differ from Linux) but
prompted me to write a help function ;-)

~~~
gfosco
I have a project that makes heavy use of sed.. On macOS, I `brew install gnu-
sed` and use `gsed` instead, so it works like other platforms.

~~~
flaxton
Nice! I will check that out.

------
owenshen24
Unrelated: Is there any connection between the author and the other sam[]zdat
who writes about society and other intriguing topics?

[https://samzdat.com/](https://samzdat.com/)

~~~
krick
I wouldn't know, but there is no reason for me to be thinking something like
that. "Samizdat" is not really a name or something, it's a transliteration of
"самиздат", which is a short/colloquial for "самостоятельное издательство",
which literally means "self-publishing" (this was a thing during the USSR,
where "self-publishing" was basically opposed to "real, official government-
approved publishing"). I believe it's just a "clever" domain somebody was able
to acquire, nothing more.

~~~
owenshen24
Ah, thanks for the explanation.

------
7786655
Pretty sure this won't work if the script is called via $PATH

~~~
pwdisswordfish2

        x=$(command -v $0 2>/dev/null)
        sed -rn 's/^### ?//;T;p' $x
    

Personally I would not use the author's chosen sed commands.

    
    
        exec sed -n '/^###/p' $x
    

would work fine.

------
sneak
Why is “./script.sh -h” better than “less script.sh”?

~~~
soraminazuki
If you use zsh, you can use the help output to generate completions for your
own scripts.

[https://github.com/zsh-users/zsh-
completions/blob/master/zsh...](https://github.com/zsh-users/zsh-
completions/blob/master/zsh-completions-howto.org#completing-generic-gnu-
commands)

------
stkai
Handy! Now, can it author the help text as well? ;)

------
thangalin
There's an endless variation on how shell scripts can present help
information. Here's another, consider this array:

    
    
        ARGUMENTS+=(
          "a,arch,Target operating system architecture (amd64)"
          "b,build,Suppress building application"
          "o,os,Target operating system (linux, windows, mac)"
          "u,update,Java update version number (${ARG_JRE_UPDATE})"
          "v,version,Full Java version (${ARG_JRE_VERSION})"
        )
    

The lines are machine-readable and alignment is computed by the template:

[https://github.com/DaveJarvis/scrivenvar/blob/master/build-t...](https://github.com/DaveJarvis/scrivenvar/blob/master/build-
template#L186)

When install script[0] help is requested, the following is produced:

    
    
        $ ./installer -h
        Usage: installer [OPTIONS...]
    
          -V, --verbose  Log messages while processing
          -h, --help     Show this help message then exit
          -a, --arch     Target operating system architecture (amd64)
          -b, --build    Suppress building application
          -o, --os       Target operating system (linux, windows, mac)
          -u, --update   Java update version number (8)
          -v, --version  Full Java version (14.0.1)
    

Using an array reduces some duplication, though more can be eliminated.
Scripts typically have two places where the arguments are referenced: help and
switch statements. The switch statements resemble:

[https://github.com/DaveJarvis/scrivenvar/blob/master/install...](https://github.com/DaveJarvis/scrivenvar/blob/master/installer#L191)

Usually parsing arguments entails either assigning a variable or (not)
performing an action later. Introducing another convention would allow
hoisting the switch statement out of the installer script and into the
template. Off the cuff, this could resemble:

    
    
        ARGUMENTS+=(
          "ARG_ARCH,a,arch,Target operating system architecture (amd64)"
          "do_build=noop,b,build,Suppress building application"
          "ARG_JRE_OS,o,os,Target operating system (linux, windows, mac)"
          "ARG_JRE_UPDATE,u,update,Java update version number (${ARG_JRE_UPDATE})"
          "ARG_JRE_VERSION,v,version,Full Java version (${ARG_JRE_VERSION})"
        )
    

The instructions to execute when arguments are parsed are thus associated with
the arguments themselves, in a quasi-FP style. This approach, not including
the FP convention, is discussed at length in my Typesetting Markdown
series[1].

[0]:
[https://github.com/DaveJarvis/scrivenvar/blob/master/install...](https://github.com/DaveJarvis/scrivenvar/blob/master/installer)

[1]: [https://dave.autonoma.ca/blog/2019/05/22/typesetting-
markdow...](https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdown-
part-1/)

------
oweiler
This is not elegant, this is an ugly hack at best.

