
What I learned from others' shell scripts - fizerkhan
http://www.fizerkhan.com/blog/posts/What-I-learned-from-other-s-shell-scripts.html
======
chrismorgan
I don't like the `require_curl` example.

Here's what is done there:

    
    
        OK=0
        FAIL=1
    
        function require_curl() {
            which curl 2>&1 > /dev/null
            if [ $? -eq 0 ]
            then
              return $OK
            fi
    
            return $FAIL
        }
    

(`2>&1 > /dev/null` drops stdout and writes stderr to stdout—not what was
meant. `which` doesn't write to stderr, so I drop that part.)

That can be shortened significantly by using the return code directly in the
if branch:

    
    
        function require_curl() {
            if which curl > /dev/null
            then
              return $OK
            fi
    
            return $FAIL
        }
    

Or by just using the return code directly:

    
    
        function require_curl() {
            which curl > /dev/null
            return $?
        }
    

And as it will return the return code of the last statement executed:

    
    
        function require_curl() {
            which curl > /dev/null
        }

~~~
cbsmith
As I mentioned elsewhere
([https://news.ycombinator.com/item?id=6210440](https://news.ycombinator.com/item?id=6210440)),
it's just wrong to use "which" in general.

~~~
wazari972
Indeed, which does not recognize aliases and functions, so `command -v` seems
better:

    
    
        command -v llh >/dev/null 2>&1 ; echo $?
        0 # good
        usr/bin/which llh >/dev/null 2>&1 ; echo $? 
        1  # good
    

however (zsh) shell internal which does:

    
    
        which llh ; echo $?  
        llh='ls -lh'
        	/usr/bin/ls
        0 #good

(I'm not sure however when ZSH chooses to use its internal function or the one
from $PATH)

------
mordae
My eyes bleed. So many unquoted expansions!

    
    
      APP_ROOT=`dirname $0`
      filename=`basename $filepath .html`
    

should read:

    
    
      APP_ROOT=`dirname "$0"`
      filename=`basename "$filepath" .html`
    

Plus if your script supports both `--version` and `--help` with proper
formats, you can easily generate a manual page with `help2man`. The help
output example is far from that. Also, he does not mention getopt at all.

The magic line for that is:

    
    
      eval "set -- $(getopt -o hV -l help,version -- "${@}")" || exit $?
      echo "${*}"
    

It does this:

    
    
      $ ./test foo bar --help --version
      --help --version -- foo bar
    

Which can be parsed using a `while` loop with `shift`.

For other tips on shell scripting best practices I recommend Gentoo ebuilds.
They tend to be quite well written. And always read scripts from other people
before running them yourself.

~~~
616c
I always wanted a shell best practices book (for formatting, beyond the TLDP
Advanced Bash Guide, or dare I even say the POSIX spec). I will definitely
checkout the Gentoo ebuild scripts. I used to look at Debian in my youth to
try and find someone to imitate.

However, are there are any people writing lint or prettify or Perl Critic like
progs or web services that will properly wrap and do things you mention?

Would there be interest in such a thing?

UPDATE: I found this stuff. Does anyone use these things?

[https://trac.id.ethz.ch/projects/bashcritic/](https://trac.id.ethz.ch/projects/bashcritic/)

[http://man.he.net/man1/checkbashisms](http://man.he.net/man1/checkbashisms)

[http://stackoverflow.com/questions/3668665/is-there-a-
static...](http://stackoverflow.com/questions/3668665/is-there-a-static-
analysis-tool-like-lint-or-perlcritic-for-shell-scripts)

~~~
dmckeon
A (cheap) trick for formatting shell scripts is approximately:

    
    
      wrap script into a shell function
      source function definition
      diff defined function against script
    

with something like:

    
    
      { printf 'dummy (){ '
        cat script
        printf "; };\ntype -a dummy\n"
      } > dummy
      source dummy > defined
      rm dummy; unset -f dummy
      diff script defined
    

Issues include:

    
    
      comments are absent from defined function listing
      trailing semicolons appear after each shell command / line
      the first semi in the trailing "; };" may be a syntax error
    

The default function listing format is fairly generic, so personal formatting
preferences must be left as an exercise for the reader.

MHO: people who insist on specific code formats should provide tools that can
convert arbitrary code into their preferred format.

------
praptak
Shell scripts have their use but at the point when your script needs output
coloring or printing debug information, it is time to switch to Python.

Edit: Place a general disclaimer to weaken the absolute tone of the above
statement. Rule of thumb, exceptions apply, yadda, yadda. In other words, I
somewhat agree with most responses that disagreed with the statement above :-)

~~~
kamaal
Python is an ideal replacement for small Java programs, not Shell.

The moment your script needs any more than 10 regular expressions, or dealing
with >3 files all in a complex interplay- use of Perl becomes inevitable. And
that is something like the very utmost basic thing you can do with Perl.

Python is more like tried-to-be-scripting-but-is-a-web-framework language.

~~~
rubinelli
I agree that Python is a general-purpose language that could replace much of
the Java code you see around, but I never got the feeling that it is
particularly well-suited for the web. For one thing, with significant
whitespace, it's harder to embed snippets of code in HTML or vice versa, so
you have to work with a templating library right from the beginning. Some will
see this as a blessing in disguise, but for very small projects or mostly
self-contained components, having logic and presentation in one file is much
more productive.

~~~
kamaal
Ironically bulk of all the Python out there is web code.

In fact Django, Twisted and Zope is all the Python code there is.

Scripting was never Python's forte. Scripting is all about succinctness, power
and providing as much power with fewer constructs and restrictions. Which
happens to be exactly the very opposite of Python goals, and some thing which
more or less as a mission statement Python tries to achieve. This is why
Python will likely never be a very successful scripting language.

Python was always a web language for frustrated java programmers who couldn't
put with java's problems anymore. Much of Python's success is in web frame
work area. Which was previously Java's territory.

~~~
PommeDeTerre
Yes, Python is obviously used for web development. But it's very short-
sighted, and even absurd, to claim that the "bulk of all the Python out there
is web code".

It's even more absurd to claim that "Django, Twisted and Zope is all the
Python code there is." That's utter nonsense, in fact.

There are numerous non-web applications that use Python extensively, whether
they're partially or fully implemented in Python, or whether they can be
scripted using Python.

Then there are the numerous libraries and frameworks for Python, from GUI
toolkits through to scientific computation packages.

Many, many organizations use Python internally for a very wide variety of
tasks and systems, without broadcasting such use loudly, if at all. This
ranges from one-off scripts up to entire multi-million-line software systems.

I sure hope that you're joking, but it just isn't coming across as a joke.
Nobody can seriously claim that "Python will likely never be a very successful
scripting language" when it has undoubtedly been one of the most successful,
and versatile, programming and scripting languages around for many years now.

~~~
kamaal
The biggest reason why Python 3 is/was seeing such late adoption is purely
because web frameworks/libraries were not migrated to Python 3. Goes on to
tell how many people use Python for what purposes, that absence of a domain of
frameworks/libraries slows down the adoption of the language itself for years.

Of course Python is used for scripting purposes. But that amount of code is no
where close the web code that is written in Python. It all depends how much
code in ratio is written for what purposes and not the total amount of code
written for that purposes.

Python's glory days came with web frameworks and continue to be the reason for
its fame and wide spread adoption.

I will take Python seriously if it offers the same capabilities as Perl at
least on the command line.

Python is a awesome general purpose language. But so far as scripting is
concerned it is still no match to Perl

~~~
PommeDeTerre
Like I tried to explain to you earlier, those who use Django and the other
Python web frameworks that may not really support Python 3 yet are a very
small portion of the entire Python user community.

Many of the rest of us have been happily and very successfully using Python 3
for years now, including for the scripting tasks that you incorrectly claim we
don't use Python for.

Your other claims like "Python's glory days came with web frameworks and
continue to be the reason for its fame and wide spread adoption." are truly
absurd and outright wrong. Python was very popular and widely used well before
the mid-2000s, when many of the web frameworks you're referring to were first
released.

I think you vastly overestimate the amount of Python used for web
applications, as well. This is understandable, as such uses are often more
visible than the other behind-the-scenes uses. But there's a staggering amount
of Python code that you don't see, and that isn't used for web applications.

------
arnehormann
Huh, no mention of "set -e -u"? Do yourself a favor and follow this, too:
[http://www.davidpashley.com/articles/writing-robust-shell-
sc...](http://www.davidpashley.com/articles/writing-robust-shell-scripts/)

~~~
unhammer
Also, regarding the lockfile examples, Linux users get a much simpler method
using flock:
[http://mywiki.wooledge.org/BashFAQ/045?highlight=%28flock%29](http://mywiki.wooledge.org/BashFAQ/045?highlight=%28flock%29)

------
captn3m0
A good gem I found recently was to use the big version of the command line
flags. So instead of seeing -s -q 1, you should use arguments like --max-
depth. Increases the readability of scripts by a huge margin.

~~~
unhammer
Note that some of those long arguments don't work on OS X or BSD, so do some
tests if you want to be portable.

~~~
nonchalance
To be fair, with some commands there are flags which do different things in
OSX/BSD and in GNU. So you should verify that all arguments, both short and
long, do the same thing

As an example, `sed -i` is used for in-place editing. BSD/OSX variants expect
an extension argument to be provided after the flag, with an empty string for
no backup (sed -i '' 's/foo/bar/' foo.bar)

On the other hand, gnu expects the backup extension in the flag (like -W in C
compilers), so the command is parsed as if 's/foo/bar/' was the file to open

------
derekp7
My favorite discovery is how to efficiently do RPC style calls in BASH. Let's
say you have some functions and variables you want to execute on a remote
server. Just do the following:

    
    
        ssh remotehost "
            $(declare -p var1 var2 var3)
            $(declare -f func1 func2 remotemain)
            remotemain"
    

In this example, var1, var2, var3 and func1, func2 are support
variables/functions for the function "remotemain". This pushes all those to
the remote side, then calls remotemain.

~~~
dmytrish
And don't forget that you can still set the remote environment variables, use
stderr output of the remote command, pipe it and so on:

    
    
        $ ssh remotehost "arecord | gzip -c" | gunzip -c | aplay
    

records raw pcm stream an a remote machine, gzip's it and gunzip's and plays
on the local machine.

------
louwrentius
A while ago I was working on a bash function library. The library itself may
not be that usefull but it also contains some examples on how to color output
and move the cursor around.
[http://code.google.com/p/bsfl/](http://code.google.com/p/bsfl/)

------
lelf

      function require_curl() { which "curl" > /dev/null; }
      function require_curl() { which -s "curl"; }
      function debug { ((DEBUG)) && echo ">>> $*"; }
    

Last one is bash, which -s is BSD'ish

~~~
cbsmith
Really, just say no to which:
[https://news.ycombinator.com/item?id=6210440](https://news.ycombinator.com/item?id=6210440)

------
pavs
As someone who is learning PHP as his first programming language, I am
surprised to see similarity in terms to syntax with shell scripts. Are there
other popular language with php-like sytax?

~~~
username42
The syntax of php was inspired by Perl (and C). Perl was inspired by
sh/sed/awk.

------
essrinn
Shell scripts frequently fail to handle directory names with spaces in them.
Example from the linked article: APP_ROOT=`dirname $0`

This is not going to do what you expect if the current script is run via a
command with spaces in one of the preceding directories' name.

It should be: APP_ROOT="`dirname "$0"`"

Even this would fail if your immediate parent directory's name ended with a
newline character, but should handle any other whitespace without issue.

~~~
damncabbage
The outer quotes aren't required:

    
    
      APP_ROOT=`dirname "$0"`
    

(See
[http://www.tldp.org/LDP/abs/html/varassignment.html#EX16](http://www.tldp.org/LDP/abs/html/varassignment.html#EX16))

Also, even if ^J or ^M were present in a parent directory's name, the command
above would still work. Whitespace-like characters take some getting used to
with Bash.

------
louwrentius
Browse this website and you will learn a lot about bash. Opinionated at times,
but that's what I like. If you use Bash scripting you've might encountered
this site already, but for what it's worth:
[http://mywiki.wooledge.org/BashFAQ](http://mywiki.wooledge.org/BashFAQ)

------
ballard
I wrote a silly script for noob openbsders like yours' truly: it builds the
usage from functions' comments in the script on-the-fly.
[https://gist.github.com/6120072](https://gist.github.com/6120072)

------
jalcine
Might inject those `tput` color commands into my shell

To those curious:
[https://github.com/jalcine/dotfiles](https://github.com/jalcine/dotfiles)

------
jfb
All this article does is make me wish there were an alternative to typical
Borne shell garbage. No sane person would ask for a wretched pile of hacks
like this.

~~~
pekk
I guess you'll just have to wait for someone to develop Perl, then...

------
arek2
Shouldn't it be:

which curl >/dev/null 2>&1

?

(what I learned from man bash)

~~~
cbsmith
Sadly, much of this article is an example of why you _shouldn 't_ learn from
other people's shell scripts. Too often shell scripts are quickly written
without much attention to best practices, so they have a lot of bad habits in
them. For much the same reason bad code tends to get copy and pasted about
until it becomes common such common practice no one questions it.

One shouldn't use which at all. It forks an external process, it is vulnerable
to PATH issues unless you specify what will surely be a non-portable path to
the executable, often isn't included in chroot'd environments, and doesn't
actually return an exit status on many platforms. In short: it sucks in almost
all possible ways you could hope to suck.

The proper expression to use is:

    
    
        command -v curl >/dev/null 2>&1
    

If you know you are in bash, type -P and possibly hash become acceptable
alternatives, but then... why not just use command -v right?

~~~
susi22
More about it here:

[http://stackoverflow.com/questions/592620/check-if-a-
program...](http://stackoverflow.com/questions/592620/check-if-a-program-
exists-from-a-bash-script)

------
xxtjaxx
[http://mywiki.wooledge.org/Bashism](http://mywiki.wooledge.org/Bashism)

------
herbig
You've got a trailing comma in your about json.

