
Defensive BASH programming - jdkanani
http://www.kfirlavi.com/blog/2012/11/14/defensive-bash-programming
======
catern
The only thing in this post that can be accurately called defensive is the use
of "local" and "readonly". The rest is all just style preferences, which are
rather subjective, and none of which are very appealing to me.

Three real defensive bash programming tips are:

\- Quote all uses of variables

\- set -o nounset

\- set -o errexit

And many others can be found in and around
[http://mywiki.wooledge.org/BashFAQ](http://mywiki.wooledge.org/BashFAQ)

~~~
krallin
While not as common as nounset and errexit, pipefail is a useful option as
well (set -o pipefail).

Using pipefail, if _any_ program in a pipeline fails (i.e. exit code != 0),
then the exit code for the pipeline will be != 0.

E.g. pipefail can be useful to ensure `curl does-not-exist-aaaaaaa.com | wc
-c` doesn't exit with exit code 0..!

~~~
michaelhoffman
You can set all three of them in a single line. Set up your Bash template with
this today:

    
    
        set -o nounset -o pipefail -o errexit

~~~
vog
I usually shorten this to:

    
    
        set -eu -o pipefail

~~~
michaelhoffman
I used to do that but the long versions are more understandable by other
people who will look at the script.

------
reacweb
Very interesting article and I agree with most of the points. One point I
strongly disagree is what is called "Code clarity" where the author replaces
conditional expressions
([https://www.gnu.org/software/bash/manual/html_node/Bash-
Cond...](https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-
Expressions.html#Bash-Conditional-Expressions)) by function calls.

I agree that the function call introduces a better name. The problem is that
this name is specific to the author of the script. It replaces a reusable
tricky knowledge of the language by a simple knowledge of the author usages.

If we take into account the increased verbosity, increased typing, increased
number of lines, there is a clear loss in using these functions.

For me, adding a comment would be far enough and far less annoying.

~~~
peeters
I disagreed with that too, but mostly because those tests are a core part of
Bash programming. If you don't know what they mean by reading them, then you
just need to learn.

It'd be like defining an "and" function and writing

    
    
        if ( and(conditionA, conditionB) ) {
            //do something
        }
    

because "&&" is too confusing. It's just syntax. Learn it.

And on the subject of &&, it's interesting that in his example about clarity
he chooses to use short-circuit and for brevity instead of the more readable
if block.

------
c0l0
A guy publishing a guide for "defensive bash programming", who, in the
process, provides this listing as an example for anything, is not fit for
publishing said guide in the first place:

    
    
      main() {
          local files=$(ls /tmp | grep pid | grep -v daemon)
      }

~~~
brianzelip
You might have been better off reading the text after the code example that
came _after_ the code you quoted.

The text read:

\- Second example is much better. Finding files is the problem of
temporary_files() and not of main()’s. This code is also testable, by unit
testing of temporary_files().

\- If you try to test the first example, you will mish mash finding temporary
files with main algorithm.

~~~
potatosareok2
The problem isn't where it is - using "ls" to find files is never a good idea
from my understanding. I think it's because of ls garbling file names but I
might be mistaken there.

That the author is using that to find files is reasonable enough to me to
disregard the rest of the blog.

edit: take that back...looked through rest of blog but don't see anything
useful. I hate his idea for functions for builtins, using local is ok, please
don't break up shell pipelines over N lines for simple stuff

~~~
peeters
Google also mandates that you shouldn't assign to a local var at declaration,
because the exit code of the function producing the value is overwritten by
the exit code to `local`.

[https://google.github.io/styleguide/shell.xml?showone=Use_Lo...](https://google.github.io/styleguide/shell.xml?showone=Use_Local_Variables#Use_Local_Variables)

------
Gnouc
It's said that all the example was missing quoting variables. It made all
others defensive things useless.

------
jakub_g
Slightly off-topic but maybe some people will find it useful:

I used to write my various glue-things-together scripts in bash, but this
quickly becomes a nightmare as the script grows, due to bash's corner cases,
syntax, portability issues etc.

Recently I wrote my massive glue-things-together script with nodejs (since I
already use node for many things) and it's much more maintainable and I
couldn't be more happy. Node 0.12 has execSync which was the missing piece for
making node the proper shell scripting platform.

If you are interested, you may want to check shelljs [1] and my snippets for
robust `exec()` and `cd()` in node.

[1] [https://github.com/shelljs/shelljs](https://github.com/shelljs/shelljs)
[2]
[https://gist.github.com/jakub-g/a128174fc135eb773631](https://gist.github.com/jakub-g/a128174fc135eb773631)

~~~
vacri
Is it really a 'script' if you need to add a dir full of modules? I've always
thought of 'scripts' as all-in-ones.

~~~
jakub_g
Well, you're right, it's a tradeoff. The 'script' is not standalone anymore,
but it's a (script + package.json) and needs `npm install` to work properly. I
still use bash for simple things, but if it grows too much and logic starts
getting non-trivial, IMO it's a valid use case to switch.

------
voltagex_
Does anyone know if these ideas conflict with
[http://mywiki.wooledge.org/BashFAQ](http://mywiki.wooledge.org/BashFAQ) or
[http://mywiki.wooledge.org/BashGuide](http://mywiki.wooledge.org/BashGuide)?
These are the resources that are drummed into you on #bash on Freenode, with
the strong suggestion that All Other Resources Are Bad And You Should Feel Bad

~~~
moviuro
The #bash users on freenode are right on that one. Everything in the wiki you
linked to is documented and everything is explained clearly.

Another very good resource is the O'Reilly book:
[http://shop.oreilly.com/product/9780596009656.do](http://shop.oreilly.com/product/9780596009656.do)

------
makecheck
At some point, "defense" against a language's faults should translate to "use
the right language for the job".

Any sufficiently-complex shell script can usually be written clearly as a
Python or Perl program for instance, without having to worry about how the
code might be misinterpreted.

Yes, I write shell scripts sometimes. I just make sure they're doing something
pretty straightforward.

------
ehartsuyker
I think the best "defensive" piece of advice you left out that everyone abuses
is never pipe `find` results to `xargs.` One should always do `find ...
-print0` to a read-while loop because of filenames with whitespace.

~~~
_ZeD_
avoid read-while loop... there are problems with filenames starting/ending
with spaces... better use

    
    
        find ... -print0 | xargs -0 ... [eventually with -n1]

~~~
Gnouc
And you need non-standard feature `-print0` and `xargs -0` there. Better way
should be `find ... -exec cmd {} +`.

~~~
potatosareok2
Is there a place you can use `-exec cmd {} +` but not `print0`? My experience
has been I can use both or I can use none (Solaris 10 boxes).

~~~
dumol
Systems with a non-GNU find, including the default one in Solaris 10 (I've
just double-checked). Perhaps you were using the GNU find in Solaris?

By the way, there are systems where 'print0' is a recognized parameter for the
default find, but not '+'. RHEL 4.x comes to mind.

~~~
potatosareok2
Sorry I think we're saying same thing. On systems with non-GNU find do either
`-exec cmd {} +` or `print0` work? My exp was both did NOT work. So either
both work or both don't.

But if I understand you correctly on RHEL 4.X `print0` works but `-exec cmd {}
+` doesn't.

Which is to say I disagree with op that it's better to rely on `exec cmd {} +`
when it seems you're more likely to have `print0 and xargs -0` then that.

------
emmelaich
The sloshes are redundant after a pipe symbol. i.e.

    
    
        ls $dir \
            | grep something
    

is the same as

    
    
        ls $dir |
            grep something

~~~
Tepix
However, the author argues that the pipe symbol should be moved to be
beginning of the next line, so the backslash isn't optional.

------
falsedan
All these recommendation are excellent (except omitting the necessary
obsessive quoting of all variables everywhere (just in case they contain a
space).

I've been enjoying BATS [0] for my bash testing.

[1] [https://github.com/sstephenson/bats](https://github.com/sstephenson/bats)

~~~
lisivka
Why BATS? Why not something *unit-like? Example:
[https://github.com/vlisivka/bash-
modules/blob/master/main/ba...](https://github.com/vlisivka/bash-
modules/blob/master/main/bash-modules/test/test_arguments.sh) .

~~~
falsedan
TAP output is killer, plus I prefer to write my tests in bash -e, not Java
thanks.

------
ceruleus
I have always tried to avoid using functions in my bash scripts. That said,
it's always nice to other people's bash programming techniques and style. This
one had the biggest impact for me, although I can't imagine it's news to any
of you on here: [http://redsymbol.net/articles/unofficial-bash-strict-
mode/](http://redsymbol.net/articles/unofficial-bash-strict-mode/)

------
labianchin
Remembering best practices for BASH might be hard. I recommend using
[http://www.shellcheck.net/](http://www.shellcheck.net/). I use with vim and
syntastic. This article talks more about it: [http://jezenthomas.com/shell-
script-static-analysis-in-vim/](http://jezenthomas.com/shell-script-static-
analysis-in-vim/)

------
chmielewski
Content from so long ago, posted here on HN less than two weeks ago... here
are some recent reddit comments about the post:
[https://www.reddit.com/r/bash/comments/3w354v/defensive_bash...](https://www.reddit.com/r/bash/comments/3w354v/defensive_bash_programming/)

------
lisivka
IMHO, it is much easier to parse arguments using bash_modules args module
(disclosure: I am author).

------
matobago
I'm not sure if scripting can be consider programming. Excessive scripting
means lack of software architecture, no to mention the least efficient way to
work with the FS

~~~
darkr
Depends on how you define scripting vs programming. I'd argue that you can
"program" in a shell language; bash is Turing-complete after all.

Whether doing so or not is a good idea is another question entirely..

------
dang
Discussed last year:
[https://news.ycombinator.com/item?id=7815190](https://news.ycombinator.com/item?id=7815190).

------
peteridah
I find this useful
[https://github.com/progrium/bashstyle](https://github.com/progrium/bashstyle)

------
moviuro
This guide is all but ready for prime time. Make it go down, as it is a pile
of bad habits. Sure, there are one or two good ideas, but nothing
revolutionnary.

~~~
moviuro
Use www.shellcheck.net as a complement

------
IshKebab
Step 1: Choose a better language. Almost anything will do.

------
emmelaich
I quite like

    
    
        set -u
        set -e
    

with

    
    
        trap 'echo $0 internal error at $LINENO' ERR

------
fergie
I kind of like that BASH allows you to make unconventional programs.

------
unixhero
Yes, I applied some of these in my bash program.

------
Grexception
To me, BASH is very offensive

