What you should do to avoid the problem of mishandling spaces is use proper quoting (for i in "$@"; do ...), not changing IFS; setting IFS to \n\t will still break embedded tabs and newlines.
In general, in bash scripts any of use of $ should always be between double quotes unless you have a reason to do otherwise.
And also, the newline and tab is not explained! What is with that?
"We don't want accidental field splitting of interpolated expansions on spaces, ... but we do want it on embedded tabs or newlines?"
If you don't want field splitting, set IFS to empty! (And then you don't need the dollar sign Bash extension for \t and \n):
$ VAR="a b c d"
$ for x in $VAR ; do echo $x ; done
$ IFS='' ; for x in $VAR ; do echo $x ; done
a b c d
Also, an unquoted variable that happens to be null will essentially vanish from the argument list of any command it's used with, which can cause another class of weird bugs. Consider the shell statement:
if [ -n $var ]; then
... which looks like it should execute the condition if $var is nonblank, but in fact will execute it even if $var is blank (the reason is complex, I'll leave it as a puzzle for the reader).
Setting IFS is a crutch that only partly solves the problem; putting double-quotes around variable references fully solves it.
In this case
[ -n $var ]
test -n $var
Therefore, always quote your variables.
[ -n ]
what the author is doing is like this in Python:
stuff=["a b", "c d", "e f"]
for thing in '\n'.join(stuff).split('\n'):
for filename in "*.txt" ; do...
for filename in *.txt ; do ...
You don't need play games with IFS to correctly process filesystem entry names expanded from a pattern.
./foo.sh one "a b" two
for x in "$*"; do
for x in "$@"; do
Quoted "$* " separates the parameters using the first character stored in IFS, or with nothing if IFS is unset/null. Usually, the first character of IFS is space, giving the impression that "$* " means "separate with spaces"; i.e. that it's just an ordinary quote job around $* (i.e. that it is "regular", in your words).
Quoted "$@" does ... what you clearly understand.
That's hardy the case. Most other variables do not represent the positional parameters, and don't have the logic of "$@" which effectively produces "$1" "$2" "$3" ...
Heck the shell language itself should have a declaration for this!
typeset -w foo # foo is not expected to contain spaces
Now if you have an unquoted $foo that undergoes field-splitting, bash produces an error if that splitting actually breaks the contents of foo into two or more pieces.)
Furthermore, a way could be provided to declare that a variable requires splitting. Maybe "typeset -W". This could even assert into how many pieces "typeset -W3 foo" means that expansions of foo are expected to undergo splitting, and it must be into three fields.
Then there could be a global diagnostic option (similar to set -u and set -e) which diagnoses all unquoted expansions of variables, except for the -W and -w ones. The -w ones are diagnosed if they are subject to splitting, and splitting actually occurs. The -W ones are diagnosed if they quoted, or if they are unquoted and splitting doesn't produce the required number of pieces, if specified.
and `brew install shellcheck`
> Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?
> set -e was an attempt to add "automatic error detection" to the shell. Its goal was to cause the shell to abort any time an error occurred, so you don't have to put || exit 1 after each important command. That goal is non-trivial, because many commands intentionally return non-zero.
> What are the advantages and disadvantages of using set -u (or set -o nounset)?
> Bash (like all other Bourne shell derivatives) has a feature activated by the command set -u (or set -o nounset). When this feature is in effect, any command which attempts to expand an unset variable will cause a fatal error (the shell immediately exits, unless it is interactive).
pipefail is not quite as bad, but is nevertheless incompatible with most other shells.
grep some-string /some/file | sort
is a good example of why -e and pipefail are dangerous. grep will return an error status if it gets an error (e.g. file not found) or if it simply fails to find any matches. With -e and pipefail, this command will terminate the script if there happen to be no matches, so you have to use something like || true at the end... which completely breaks the exit-on-error behavior that was the point of the exercise.
Solution: do proper error checking.
> GreyCat's personal recommendation is simple: don't use set -e. Add your own error checking instead.
> rking's personal recommendation is to go ahead and use set -e, but beware of possible gotchas. It has useful semantics, so to exclude it from the toolbox is to give into FUD.
If I actually do want to guard against every case imaginable, I immediately switch to Python or some other language that at least knows how to quote things unambiguously without a lot of effort.
Also, as an alternative to the proposed `set +e; ...; set -e` wrapper for retrieving the exit status of something expected to exit non-zero (generally cleaner in my opinion, if slightly "clever"):
count=$(grep -c some-string some-file) || retval=$?
And even despite more free licenses (AFAIR, IANAL), you can't depend on actual Korn shells being available on Unices. At least the dependent app situation has been getting a lot better, mostly by the death of workstations and their proprietary OSs (try depending on almost any grep/awk/sed option/switch when it has to run on Solaris/AIDX/HP-UX).
Although "all the world's a GNU/Linux" seems the new plague upon our lands here…
So after all these years, I'd say we're still in pretty much the same situation that birthed Perl. Which still would be my preferred choice if I'd actually have to distribute scripts and we're not talking about my own private, context-specific shortcuts, scripts and functions.
I almost never use -e, and as a result I have to stay vigilant and test return codes all over the place. I prefer that to the kludge of forcing everything to return zero and I think it produces overall better results. Ultimately you want to handle an error, not just abort, and -e doesn't do much to promote handling properly.
I've had mixed luck with set -e combining with trap ERR though - they seem to conflict with each other and I don't have the shell knowledge to sort it out.
Since “|| :” is always written at the end of lines, it should be short and visually unobtrusive.
user@host ~> :
fish: Unknown command ':'
I use it all the time when I have to write shell scripts.
Instead of handling non-zero exit statuses in a correct way,the article suggests to interrupt the script right in the middle, with probably some temporary files and processes hanging around which can't be cleaned up if something goes wrong.
The same BS goes though the entire article.
Has the author actually written anything bigger than echo "Hello world!" in Bash?
trap mycleanupfunc EXIT
P.S. No, temporarily disabling the option is not a solution, it's another workaround for the problem created out of nothing.
I'm not sure what scenario you're imagining with your other concern. This does what you would expect:
set -o errexit
if [ -f somefile ]
echo "File exists."
echo "File does not exist."
I'm not sure that writing scripts that rely on bash-specific features is such a great idea.
I am genuinely curious how you would write a command with a pipe in plain POSIX /bin/sh such that a non-zero exit status from the program that writes into the pipe is detected (as can be done in bash with "set -o pipefail" or "$PIPESTATUS").