The only thing in this post that can be accurately called defensive is the use of "local" and "readonly". The rest is all just style preferences, which are rather subjective, and none of which are very appealing to me.
Very interesting article and I agree with most of the points. One point I strongly disagree is what is called "Code clarity" where the author replaces conditional expressions (https://www.gnu.org/software/bash/manual/html_node/Bash-Cond...) by function calls.
I agree that the function call introduces a better name. The problem is that this name is specific to the author of the script. It replaces a reusable tricky knowledge of the language by a simple knowledge of the author usages.
If we take into account the increased verbosity, increased typing, increased number of lines, there is a clear loss in using these functions.
For me, adding a comment would be far enough and far less annoying.
I disagreed with that too, but mostly because those tests are a core part of Bash programming. If you don't know what they mean by reading them, then you just need to learn.
It'd be like defining an "and" function and writing
if ( and(conditionA, conditionB) ) {
//do something
}
because "&&" is too confusing. It's just syntax. Learn it.
And on the subject of &&, it's interesting that in his example about clarity he chooses to use short-circuit and for brevity instead of the more readable if block.
A guy publishing a guide for "defensive bash programming", who, in the process, provides this listing as an example for anything, is not fit for publishing said guide in the first place:
You might have been better off reading the text after the code example that came after the code you quoted.
The text read:
- Second example is much better. Finding files is the problem of temporary_files() and not of main()’s. This code is also testable, by unit testing of temporary_files().
- If you try to test the first example, you will mish mash finding temporary files with main algorithm.
The problem isn't where it is - using "ls" to find files is never a good idea from my understanding. I think it's because of ls garbling file names but I might be mistaken there.
That the author is using that to find files is reasonable enough to me to disregard the rest of the blog.
edit: take that back...looked through rest of blog but don't see anything useful. I hate his idea for functions for builtins, using local is ok, please don't break up shell pipelines over N lines for simple stuff
Google also mandates that you shouldn't assign to a local var at declaration, because the exit code of the function producing the value is overwritten by the exit code to `local`.
Slightly off-topic but maybe some people will find it useful:
I used to write my various glue-things-together scripts in bash, but this quickly becomes a nightmare as the script grows, due to bash's corner cases, syntax, portability issues etc.
Recently I wrote my massive glue-things-together script with nodejs (since I already use node for many things) and it's much more maintainable and I couldn't be more happy. Node 0.12 has execSync which was the missing piece for making node the proper shell scripting platform.
If you are interested, you may want to check shelljs [1] and my snippets for robust `exec()` and `cd()` in node.
Well, you're right, it's a tradeoff. The 'script' is not standalone anymore, but it's a (script + package.json) and needs `npm install` to work properly. I still use bash for simple things, but if it grows too much and logic starts getting non-trivial, IMO it's a valid use case to switch.
I'm assuming this isn't a real example of a thing you would want to do, because it doesn't make sense. (redirecting stderr to stdout, stdout to a file, then pipe to tee and echo?)
At any rate, you'd:
1. use some sort of popen()
2. use pipes and pass the same pipe to stdout as stderr
3. pass stdout of one popen to another's stdin
4. just use bash, because this gets so hairy and bash does it really well. When I want to use a complex pipeline of programs in non-bash programs, I'll frequently popen() to a bash script. This is ugly, but if you're really careful (shellshock?) it can be ok.
At some point, "defense" against a language's faults should translate to "use the right language for the job".
Any sufficiently-complex shell script can usually be written clearly as a Python or Perl program for instance, without having to worry about how the code might be misinterpreted.
Yes, I write shell scripts sometimes. I just make sure they're doing something pretty straightforward.
I think the best "defensive" piece of advice you left out that everyone abuses is never pipe `find` results to `xargs.` One should always do `find ... -print0` to a read-while loop because of filenames with whitespace.
Sorry I think we're saying same thing. On systems with non-GNU find do either `-exec cmd {} +` or `print0` work? My exp was both did NOT work. So either both work or both don't.
But if I understand you correctly on RHEL 4.X `print0` works but `-exec cmd {} +` doesn't.
Which is to say I disagree with op that it's better to rely on `exec cmd {} +` when it seems you're more likely to have `print0 and xargs -0` then that.
All these recommendation are excellent (except omitting the necessary obsessive quoting of all variables everywhere (just in case they contain a space).
I have always tried to avoid using functions in my bash scripts. That said, it's always nice to other people's bash programming techniques and style. This one had the biggest impact for me, although I can't imagine it's news to any of you on here: http://redsymbol.net/articles/unofficial-bash-strict-mode/
I'm not sure if scripting can be consider programming. Excessive scripting means lack of software architecture, no to mention the least efficient way to work with the FS
This guide is all but ready for prime time. Make it go down, as it is a pile of bad habits. Sure, there are one or two good ideas, but nothing revolutionnary.
Three real defensive bash programming tips are:
- Quote all uses of variables
- set -o nounset
- set -o errexit
And many others can be found in and around http://mywiki.wooledge.org/BashFAQ