
Shell Style Guide from Google - grhmc
https://google.github.io/styleguide/shell.xml
======
ronjouch
Tangentially, [http://www.shellcheck.net/](http://www.shellcheck.net/) is an
awesome static linter for shell code. Available online, as a command-line
tool, but most importantly as a plugin for your favorite $EDITOR.

~~~
careersuicide
Ah! Thank you so much for posting this!

Whenever I attempt to learn a new language I find that static analysis tools
and linters, if available, help me get over any initial confusion about what
constitutes good idiomatic code. Guides like OP help too, but nothing beats a
good linter. The thing is though, I've been writing Bash shell scripts for
going on 15 years now, and I still overlook little details [0] all the time.

I've been mulling in the back of my head for a while a theory about the need
for style guides and linters: if a language makes them a near necessity for
reasons other than aesthetics then the language is probably best avoided. I'm
not sure if agree totally with that. However, I think the most frustrating
experiences I've had while programming are the result of languages which allow
for multiple ways, in terms of syntax, of doing the same thing which behave
the same _most_ of the time.

0:
[https://github.com/koalaman/shellcheck/wiki/SC2086](https://github.com/koalaman/shellcheck/wiki/SC2086)

~~~
ronjouch
100% agreed!

When writing bash, there are so many hoops to jump through (e.g. syntactic
traps, bashisms to avoid) that people who like me who use these languages only
occasionally _will shoot their foot repeatedly_.

For this population (and I think it's big, given the glue nature of *sh), a
guide won't help much because occasional practice won't give them any time to
sink in; interactive linters are a godsend.

------
the_common_man
Good rules overall, we follow similar guidelines. Any comments on:

* readonly - describing these in library scripts is a bit dangerous since the readonly is not specific to the file. So, unless you have some namespacing pattern for globals, this will bite you

* I didn't see any recommendation but always start with set -eux -o pipefail. set -e has it's own set of pitfalls but it's best to learn what the pitfalls are and work with it :-) It helps in the long run.

* When using set -u, use the FOO={FOO:-} syntax to initialize defaults for all your env vars.

~~~
codys
On point 2: indeed, given that they are restricting themselves to bash, I
would have expected a note of `set -eu -o pipefail`, at least. `-x` (tracing),
isn't appropriate for many scripts, though.

On point 3: `: ${FOO:-some default value}` is the typical pattern for ensuring
a default value is set.

~~~
ymse
I tend to avoid `set -e`. To borrow the Python adage: "explicit is better than
implicit". In shell it translates to "if you need error handling, add it where
necessary".

See
[http://mywiki.wooledge.org/BashFAQ/105](http://mywiki.wooledge.org/BashFAQ/105)
for some (most) of the pitfalls.

~~~
d0mine
"Errors should never pass silently." is more relevant for `set -e`

~~~
ymse
Except they do even with `set -e`. Especially if you forget to/can't set
`pipefail` (which is bash-specific).

Or consider the last example from the wiki link above:

    
    
      set -e
      f() { local var=$(somecommand that fails); }
      f    # will not exit

~~~
lisivka
Yep, it is inconsistent:

    
    
        $ ( set -e; foo() { f=$(false); }; foo ; ); echo $?
        1
        $ ( set -e; foo() { local f=$(false); }; foo ; ); echo $?
        0
        $ ( set -e; foo() { local f; f=$(false); }; foo ; ); echo $?
        1
    

However, it is still better to use -e to catch errors early.

------
nickysielicki

        > Bash is the only shell scripting language permitted for
        > executables.
    
        > [...]
    
        > The only exception to this is where you're forced to by whatever
        > you're coding for. One example of this is Solaris SVR4 packages
        > which require plain Bourne shell for any scripts. 
    
        > [...]
    
        > When to use Shell
        > 
        > * If you're mostly calling other utilities and are doing
        > relatively little data manipulation, shell is an acceptable choice
        > for the task.
        > 
        > * If performance matters, use something other than shell.
        >
        > * If you find you need to use arrays for anything more than
        > assignment of ${PIPESTATUS}, you should use Python.
        > 
        > * If you are writing a script that is more than 100 lines long,
        > you should probably be writing it in Python instead.  Bear in mind
        > that scripts grow.
        > 
        > * Rewrite your script in another language early to avoid a
        > time-consuming rewrite at a later date.
    

Limiting the domain of shell scripts is the best advice in here. Shell scripts
are terrible to maintain, and you should use perl or python if it's
complicated.

But limiting the domain of shell scripts also probably means that you're not
going to be using any advanced features of bash, so you should write it in sh
instead. It's undoubtedly more portable, and it also serves to enforce the
idea that you shouldn't write complicated shell scripts. I'll take a leap and
say it outright, if you're using a feature of bash that isn't in bourne shell,
you shouldn't be writing it in either of them.

The internet is littered with bash scripts that are compatible with bourne
shell, but the shebang says '#!/bin/bash'. Well some systems don't have bash!
But everyone has bourne shell.

I'm disappointed by their choice of bash and how this guide might reinforce
that behaviour, but other than that, the actual style guidelines are great.

[Edit: rewording]

~~~
Zardoz84

        > * If you are writing a script that is more than 100 lines long,
        > you should probably be writing it in Python instead.  Bear in mind
        > that scripts grow.
        > 
        > * Rewrite your script in another language early to avoid a
        > time-consuming rewrite at a later date.
    

Hehe. I did on DLang with a script that we have to update some instances of
our application on develop machines. Our old bash scripts was becoming bigger
and can't handle any more some few cases.

~~~
nickysielicki
What made you choose D?

------
ymse
> _Executables must start with #! /bin/bash_

I've started using _#! /usr/bin/env bash_ for compatibility with non-FHS
distros such as NixOS.

Happy to see arrays omitted; if you find yourself in need of bash arrays you
should definitely consider a proper language.

I tend to stick to POSIX shell to the extent possible, but readonly variables
and [[ ]] syntactic sugar makes sense when maintainability is more important
than portability.

~~~
jxy
Those are exactly two things I don't agree.

> Executables must start with #!/bin/bash

Either they never use *BSD, or they put their bash there. Unbelievable choice.

> If you find you need to use arrays for anything more than assignment of
> ${PIPESTATUS}, you should use

I use array all the time and never used PIPESTATUS. So what?

~~~
ymse
Frankly, I find hard-coding #!/bin/bash sensible in a large corporate setting
such as Google. It prevents PATH manipulation and they obviously have a
uniform and controlled environment.

Bash arrays can be convenient, but at that level of complexity Python or Ruby
are much easier to read and understand (and necessarily maintain).

------
giovannibajo1
Any reason why they don't mandate something like "set -euo pipefail" at the
beginning? I find it invaluable for writing and debugging scripts, and in
general for avoiding weird (or dangerous) errors.

------
jdgiese
Love this! If you think you are a BASH pro, check out your knowledge with some
challenge problems:

[http://innolitics.com/10x/advanced-bash-
exercises/](http://innolitics.com/10x/advanced-bash-exercises/)

Also has some other useful BASH links and comments. Here is a quote:

> BASH is the most widely-used and widely-supported shell for Linux. There are
> other shells that are better than BASH in various ways, but we feel that
> none of these other shells are better enough to warrant replacing BASH as
> the de-facto standard when writing shell scripts. BASH is installed by
> default on almost all Unix-based operating systems, and the majority of the
> world’s shell scripts are written in BASH. For this reason, we suggest that
> all of our developers learn BASH.

> BASH scripts are a domain-specific programming language that is well-suited
> to managing processes and files. That being said, the large number of
> special characters appropriated for process management, its text expansions,
> and its unusual syntax make BASH poorly-suited for general purpose
> programming. Accordingly, we think that BASH should only be used for scripts
> that are predominantly concerned with processes and files.

------
pwd_mkdb
i try to write my scripts in such a way that if all newlines were lost, the
script would still run. semicolons even where optional.

requiring the use of bash for non-interactive use? good grief.

is it possible this company has a linux bias?

the usefulness of a minimal scripting shell cannot be denied. even with linux
distribs that use bash, we almost always see busybox in use. busybox is more
like the almquist sh than bash.

with sh, bash-only features may not work.

for example, shellshock did not work with almquist sh.

why are bash script not given a .bash file extension to distinguish them from
sh scripts (.sh extension)?

.ksh extension is often used for korn shell scripts.

~~~
chubot
Given that every server and most desktops at Google are Linux, and that it
created at least two operating systems based on Linux (Android and ChromeOS)
-- yes, I think it would be safe to say there's a Linux bias :)

------
sytse
Inspired by this we made a guide to shell out from ruby in a safe way:
[https://github.com/gitlabhq/gitlabhq/blob/master/doc/develop...](https://github.com/gitlabhq/gitlabhq/blob/master/doc/development/shell_commands.md)
(on mobile and only seeing the GitHub link in Google)

------
giantninja
Are there any good example scripts that show actual examples of most of these
cases? I think that would be a great supplement supplement to this guide

~~~
ronjouch
Yes but they're hidden behind the little "Play" button at the left of each
point. Clicking it reveals further explanation and, often, examples.

Not very discoverable; even though this feature is documented at the top right
of the page, I skipped it too and only found it by accidentally clicking one
of the buttons while obsessively-compulsively selecting text I was reading :D

~~~
Razengan
The "disclosure triangle", I believe. :)

------
pyed
is [https://github.com/mvdan/sh](https://github.com/mvdan/sh) relevant ?

------
LukeShu

        > SUID and SGID are forbidden on shell scripts.
    

Kinda funny, the Linux kernel just ignores SUID/SGID on script files.

~~~
superuser2
Why can't we have setuid on shell scripts which are non-writeable?

I have often wished for a facility to allow unprivileged users to execute
specific predefined tasks, and writing C programs for such things would be a
huge pain.

~~~
ekimekim
The issue is that there's an unavoidable race condition which renders SUID for
interpreted (ie. #!) executables insecure.

Suppose there's some SUID bash script owned by root in the system that starts
with "#!/bin/bash".

I (an unprivileged user) create a symlink "./foo" to "/path/to/SUID_script". I
execute "./foo".

The kernel follows foo, reads the SUID script's #!, and so runs "/bin/bash
./foo". Honoring the SUID, bash is run as root.

Here's the race. In between the kernel doing this and bash finishing
initialisation and reading its script arg, I swap out the "foo" symlink to
instead point at "./my_evil_code".

So bash reads "./foo" and executes the content as root. I now have arbitrary
code execution as root.

Are there ways this could be fixed? Possibly. But not easily.

It's not enough to just say "SUID can't work through a symlink" \- I could've
symlinked to the SUID script's parent directory, for example.

You can't say "the kernel should pass the fully resolved path to the
interpreter", this will break a million scripts that rely on changing
behaviour based on $0.

I share your pain about wanting an easy, non-compiled means of creating a
small SUID program. But I don't see any good way unless something changes.
Maybe a util that takes a #! interpreted file, adds an ELF header and some
machine code that execs the desired interpreter and feeds it static script
content?

~~~
zem
> I share your pain about wanting an easy, non-compiled means of creating a
> small SUID program. But I don't see any good way unless something changes.

i think the way to go is to just to have a compiled language that makes it
easy to write bash-like programs. julia has a pretty nice-looking subprocess
module, for instance:
[http://docs.julialang.org/en/release-0.4/manual/running-
exte...](http://docs.julialang.org/en/release-0.4/manual/running-external-
programs/)

------
hobarrera
> Bash is the only shell scripting language permitted for executables.

Why? That seems completely arbitrary.

I already lost interest in following this in the first line.

> Executables should have no extension (strongly preferred) or a .sh
> extension. Libraries must have a .sh extension and should not be executable.

So, not only are they bash-only, but you add an extension that makes one
assume the contrary. UGH!

~~~
ams6110
The answer to your question is obvious if you've ever worked in a large
development organization.

~~~
hobarrera
You could have bothered sharing it, if it's so obvious.

------
SoapSeller
I was expecting for style guide on how to _design_ shell user interactions.

Nontheless, Shell _Coding_ Style Guid is interesting.

~~~
xufi
That's a good point. I've been trying to find a good guide on make a nice MOTD
(Message of the day Script) and some ASCII art since I've been playing around
with my envrioment more.

------
paule89
Google seems to prefer spaces over tabs

~~~
e40
And 2 spaces is way too little. 4 is the sweet spot.

Reason: with 2 the visual lines at different indent levels are just too close,
in many fonts/sizes.

~~~
rossjudson
Wrong. 3 spaces is the sweet spot. What is it with you people and powers of
two? ;)

~~~
lisivka
It is easy to catch an indentation error with two spaces, while it is harder
to make an indentation error with just two spaces. It also easier to type two
spaces than 3 or 4 spaces, which is important when program is typed without
help of an autoindentation tool. Two spaces are enough for fixed length font,
typical in terminal, but also saves screen space, which is just 80 characters
in width.

------
gman83

      Indentation
    
        Indent 2 spaces. No tabs.
        Use blank lines between blocks to improve readability.
        Indentation is two spaces. 
        Whatever you do, don't use tabs. 
        For existing files, stay faithful to the existing indentation.
    

Richard Hendricks is not going to like that!

~~~
andrepd
I don't wish to dig up a holy war, but why would you want to use spaces
instead of tabs? That's what they're for, isn't it? One tab = one indentation
level. No need for messing around with spaces, which serve other purposes.

~~~
mwfunk
I would be happy with 100% tabs or 100% spaces, but in practice it seems like
allowing tabs inevitably leads to source files indented with a combination of
both, either by accident or because someone wants to have intermediate levels
of indentation (for multiline conditionals for example).

Once you have mixed tabs and spaces, everything goes to hell because now that
code's formatting is tied to a specific person's idea of how many spaces
should a tab display as, or whether tabs refer to tab stops or just a fixed
number of spaces.

I never had strong feelings about it until encountering some really
pathological code bases, which had gotten to the point where there was
literally no valid tab->spaces setting that would make everything look
correct.

Ultimately, though, these things really just don't matter. I use 4 spaces per
indent level, but it's not like I'm incapable of reading and writing code that
uses tabs, or 2 spaces, or 8 spaces. Really the only thing that matters is
that there is a standard and that it is consistently applied within a project.

~~~
etwigg
[https://github.com/diffplug/spotless](https://github.com/diffplug/spotless)

------
darekdk
XML. Woah.

------
codemac
Hard tabs for life.

Frankly, wish they had chosen a better shell like es or rc for all to be
written in.

