
Shell Style Guide - gkfasdfasdf
https://google.github.io/styleguide/shell.xml
======
no_wizard
I don’t understand all the hate for bash or python in this thread.

These are programming languages. Like all programming languages one must
understand how to deploy them and use them properly. Yes bash has faults
absolutely. It can be very arcane and esoteric and I think this due to the
fact that it’s creators are very much of the old 1970s and 1980s Unix mindset
and largely bash tools still feel like they work the way they did back then

Python is also a tool. I have written scripts in python to manage processes do
forking etc and it too has some warts. It is also used by companies tohat
deploy thousands of lines of code or more. Yes it isn’t statically typed and
yes you can get into a box but guys no single language is objectively superior
to another.

So learn them like you would any other tool and move on.

~~~
arca_vorago
It's because so many here on hn are the compsci theoritical programmer,
sysadmins are dying devops is everything, kind of people who don't actually
support production systems other than maybe their particular depts single page
webapp, etc...

Hn has a problem with people living in the SV filter bubble seeing everything
through that lens... in the real world, bash is still a king.

When your devs shit out some bad code that causes your sysadmin to get a call
at 3am... he's using bash to fix your fucking nodejs or whatever
fadofthemoment bullshit you decided to run with. Of course entire applications
generally shouldn't be in bash... it's like a kit of duct tape, epoxy, and
rivets. Of course you should have made the product better, but when it does
break, bash can keep it together until you get to the shop.

I also think there is a certain amount of eliteism. It has such a low barrier
to entry, it's like some people hate the idea of people being able to program
without being programmers.

I've heard it all so much here it just goes in one ear and out the other.

/end bofh rant

~~~
tomc1985
Shell code can run in a whole bunch of environments that even getting python
into can be tricky. initramfs before your drives are mounted, for example...

A lot of the discussion on here is mind-blowing, so many pushing to throw out
perfectly good tech because it doesn't fit their (limited) worldview.

~~~
cma
Micropython can fit :P

~~~
tomc1985
Yeah but its another thing to bundle

/bin/sh will always be there

------
tootie
I absolutely hate coding in bash or looking at bash or suddenly being in
Windows and being up a tree because bash isn't supported well (it is now if
you install WSL). I only use bash to set environment variables and maybe
string together 2 or 3 build commands. If I need so much as an if statement,
I'm going to switch to a real programming language.

~~~
bitlax
This document agrees with you.

\- If you find you need to use arrays for anything more than assignment of
${PIPESTATUS}, you should use Python.

\- If you are writing a script that is more than 100 lines long, you should
probably be writing it in Python instead. Bear in mind that scripts grow.
Rewrite your script in another language early to avoid a time-consuming
rewrite at a later date.

~~~
pas
Exchanging Bash for Python is a folly.

Python seems to handle complexity better, but only on the surface. (Basically
Python has nice data structures, great string formatting, and .. that's it.)

Setting up the Python environment requires something, because Python comes in
many flavors (2 vs 3, system installed, user installed, /usr/local) and there
are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current directory with
modules, requirement for python-dev headers, and GCC to compile native parts
of packages, and then of course maybe extra shared libraries too) that can
affect the "script".

Yet Python provides no way to sanity check itself. Mypy is wonderful, but
doesn't really help with scripts, where the enemy is raised exceptions, not
simple type errors.

And Python lacks efficient, quick and dirty process control. Sure, there are
nice packages for handling subprocesses, but that's really a lot more
batteries-included in Bash.

Do I like Bash? No, but debugging a bash script is -x, even the most vile many
thousand lines of Bash will eventually stop somewhere with a clean-ish error
or finish running.

I have no great recommendation for alternative (maybe Ammonite, maybe
scripting in Rust), but I'm interested in what people think about this.

~~~
simias
It's really not about data structure or anything like that. The big problem
with any large shell script is that it's utterly difficult to do proper error
handling. Usually the best you can do is check return values to detect an
error and 'exit' immediately, hoping that your "trap" works correctly to clean
up behind you.

>Python lacks efficient, quick and dirty process control.

Yeah, quick and dirty, that's kind of the problem really. It's great for
20-line scripts but it turns into a huge mess for larger projects unless you
enforce super strict guidelines and everybody has their doctorate in bash.

Python, perl and friends make it harder to spawn 3rd party programs and pipe
them together for instance, but it also makes it massively easier to handle
errors and give decent diagnostics to the user.

~~~
pas
> trap

I don't have more faith in Python's except than in Bash's trap.

If you use 3rd party code or a subprocess, then both are pretty weak compared
to something kernel enforced.

> makes it massively easier to handle errors and give decent diagnostics to
> the user.

I don't really find that. You have to fight for input/output for subprocesses
in Python.

Sure, logging is easier in Python/perl/PHP/whatever, than in Bash, but error
handling is about the same. You have to do everything manually, and there's no
compiler, nor type (or other magical) system, that'd warn you if you have
missed something.

~~~
chronial
> You have to do everything manually

You are aware that error handling is done via exceptions python?

~~~
pas
You are aware that there's no compiler, and that you have to write the try-
except-finally?

I'm talking about how weak guarantees Python gives compared to Bash about the
correctness of the program. It's easy to get None-s, it's too easy to forget
to handle file not found or lack of permission.

~~~
xapata
Maybe you should start raising exceptions instead of returning None.

I've almost never wanted to have my program do anything but crash upon file
not found or permission error.

------
benmmurphy
i'm surprised it doesn't recommend using set -e /other flags.
[[http://redsymbol.net/articles/unofficial-bash-strict-
mode/](http://redsymbol.net/articles/unofficial-bash-strict-mode/)] maybe it
is not covered because it is not considered 'style' but i think those flags
are some of the most important things you can set when writing a bash script.
i guess if you are writing scripts that start other things, and then check
their error codes it can be annoying because you have turn them off/on.

~~~
falsedan
`set -e` is basically a way to find where you need better error-handling, not
a way to make scripts safe. I'd much rather a crummy script run into trouble
calling some new featuref, shrug, and move on to the core business-critical
code, instead of erroring out and triggering a bunch of pages.

~~~
ianbicking
Do you normally protect every command with || exit?

    
    
        cd foobar || exit
    

It's those really small failures that cause shell scripts to do crazy things
without set -e.

~~~
falsedan
Why wouldn't I rewrite the script to avoid the cd instead? The point of `set
-e` isn't to insert all those pointless guards, it's to point out "hey this
might fail, so think of a way to do this that doesn't rely on cd".

For your extremely thin example, if it (say) removed some files, I would only
remove the files from the current directory or an absolute path.

------
_arvin
I'm really surprised they went with:

#!/bin/bash

as opposed to:

#!/usr/bin/env bash

the latter feels more flexible and dependable for a script to be passed
around.

~~~
nvarsj
I really don't like the `/usr/bin/env bash` approach (it came from the ruby
world I think?). It's less portable than /bin/bash in my experience - I've
been in environments, usually older ones, where it doesn't work at all. But
/bin/bash works everywhere. You also can't pass command line arguments. And
it's slightly more complex than just invoking /bin/bash.

~~~
davidcuddeback
> _It 's less portable than /bin/bash in my experience_

/bin/bash breaks on FreeBSD, which installs Bash at /usr/local/bin/bash.
/usr/bin/env bash works though.

> _You also can 't pass command line arguments._

Are you sure about that? Given this script:

    
    
        #!/usr/bin/env bash
        echo "args: $@"
    

it outputs:

    
    
        $ ./foo 1 2 3
        args: 1 2 3
    

Is that what you meant?

~~~
nvarsj
You can do “/bin/bash -e” but not “/usr/bin/env bash -e”. Same for python etc.

Good point about freebsd - I remember running into that exact problem. But I
think I just symlinked it.

[https://unix.stackexchange.com/questions/29608/why-is-it-
bet...](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-
use-usr-bin-env-name-instead-of-path-to-name-as-my/29620#29620) has more info
on the pitfalls.

In my experience env causes problems, whereas explicit binary call always
works (assuming the path exists). Not sure why I was downvoted.

~~~
jessaustin
Is this a problem with env? What does the "-e" flag do anyway? "man bash"
proved unenlightening.

~~~
jolmg
It's not because of env. It's just that shebangs are limited to passing one
optional argument to the executable specified. The way they're parsed is the
executable path, and if there's a space, then the rest of the line (including
any additional spaces) is interpreted as 1 argument to the executable.

EDIT: -e causes bash to exit on the first command that returns with an error
(returns with a non-zero return code). It's documented in `man bash` where it
documents the `set` builtin command.

~~~
jessaustin
Thanks for clarifying. In that case, one could imagine an _env_ capable of
parsing its single argument into the multiple intended arguments? I admit that
I never dreamed that the "set" builtin would have the information about "bash
-e" while the initial "OPTIONS" section did not. Why not look under the "cd",
"echo", "fc", "read", or "ulimit" builtins? Somewhat helpfully, the "COMMAND
EXECUTION ENVIRONMENT" section talks about how "-e" is inherited but not what
it means. Ah, man pages.

~~~
jolmg
Well, to be fair, the first sentence in OPTIONS is:

> All of the single-character shell options documented in the description of
> the set builtin command can be used as options when the shell is invoked.

It's easy to miss though, when one's accustomed to quickly searching with "/".

EDIT: On the imagined env, it might be possible. I wonder what aspects of
shebangs are portable across the various unix derived OSs. It may be that
there's some systems where it only takes non-whitespace characters as the
first argument and drops everything else after encountering another space.
There might be also other limitations that one should consider when making
more use of shebangs, like the character limit of the lines. For example, I
think linux only takes the first 127 characters of the shebang and ignores
everything else; other systems might take less. I encountered that limit when
writing this answer:

[https://unix.stackexchange.com/questions/365436/choose-
inter...](https://unix.stackexchange.com/questions/365436/choose-interpreter-
after-script-start-e-g-if-else-inside-hashbang/365751#365751)

EDIT 2: If you were to do such a program, it will probably also need to
implement quoting and escaping syntax. The difficult part is that, since it
wouldn't be a standard program to have, it'd have to be listed as a dependency
of whatever projects you use it in, and I wonder if the benefit really
outweighs the cost of having yet another dependency. I can't see something
like this gaining wide adoption.

~~~
jessaustin
Haha thanks that is why I missed it.

------
deeg
This quote on whether to always use braces with variable names (e.g. "${var}"
or "$var"):

> These are meant to be guidelines, as the topic seems too controversial for a
> mandatory regulation.

It's nice to see that even Google has bike-shedding arguments.

------
dkns
> If you find you need to use arrays for anything more than assignment of
> ${PIPESTATUS}, you should use Python.

This is exactly why I wrote my last script in Python. I started googling for
how to do some simple stuff with arrays in Bash and after 10 minutes decided
"I'll write it in Python".

------
assafmo
When breaking a pipe into two lines, I find it cleaner to end the first line
with pipe instead of backslash. Then I indent the second line. Bash knows a
command cannot end with pipe, therefore it's syntactically ok.

E.g.

    
    
      cmd1 |
        cmd2
    

Instead of:

    
    
      cmd1 \
        | cmd2

~~~
hi41
very nice!

------
koolba
I've never seen this guide before today but skimming through I absolutely love
it. It aligns with my _exact_ bash scripting style. Everything from the
variable and function naming rules, stderr logging, pipeline indentation, and
even having a "main () { ...}" wrapper and invocation. It's almost creepy!

~~~
enriquto
It's not creepy if you wrote it while working at google (as a mathematician, i
couldn't help but notice that you are not excluding this possibility).

~~~
nileshtrivedi
S/he did exclude this possibility by saying "I've never seen this guide before
today" :-)

~~~
JepZ
Maybe koolba was blind and has been cured.

------
geofft
The only thing I'd quibble with is the recommendation of [[ ... ]] over [ ...
], because shell programmers used to [ will be surprised that [[ "foo" == "f*"
]] does pattern matching. But that is more or less an arbitrary style
preference.

One thing I'm curious about is whether Google machines have followed Debian
etc. in making /bin/sh a faster non-bash shell, or if /bin/sh is bash.

~~~
dualbus
> will be surprised that [[ "foo" == "f*" ]] does pattern matching

It will not do pattern matching, because you quoted the right-hand side.

~~~
geofft
... ... okay, now I extremely object to [[ "foo" == f* ]] _not_ doing globbing
and doing pattern matching. But, I guess Google folks have experience that
this is non-confusing?

------
empath75
> Shell should only be used for small utilities or simple wrapper scripts.

Oh you mean we weren’t supposed to write an etl solution in bash?

~~~
jrs95
I’ve actually seen this happen. It’s terrifying.

~~~
enriquto
Sure, but not as terrifying as people rewriting shell one-liners in python or
(gasp) java.

~~~
hrktb
Then a lot of people don’t understand the limits of what they’re doing.

For instance piping though commands works fine until one character sequence is
interpreted as EOF. The fun part is it will work most of the time, and when it
fails nobody will understand why (“we didn’t touch anything”), and rewriting
the thing will be a political nightmare (“it was working before and was only 3
lines, why you need so much time to redo it ?”)

I think most people (me included) don’t do enough shell programming to really
know the trade-off of the one-liner vs doing it in groovy, so the latter
becomes the safer option (rightly so IMO)

~~~
icebraining
_one character sequence is interpreted as EOF_

I know this is just an example, but how would that happen?

~~~
hrktb
You can hit EOF or EOT in a stream of UTF-8 or other multi-byte encoding as
part of a character. There must be ways to have it handled through something
that understands character encodings and workaround the issue, but I know that
dumb shell piping will fumble on those.

~~~
JdeBP

        % printf '\x04\necho EOT is not EOF in a pipe.\n' | bash
        bash: line 1: $'\004': command not found
        EOT is not EOF in a pipe.
        %
    

Pipes do not contain line disciplines and thus do not do special character
processing.

------
gre
> If performance matters, use something other than shell.

I wrote a script that wrapped compiler calls to make dummy output files to
debug a huge compilation. Since startup time was the most important metric,
bash actually had by far the best performance.

~~~
nameless912
People rag on bash a lot, but it now forms the core of a major deployment
system I built at work. I'm not _happy_ that it's in bash, but every other
solution was worse, and it calls out to other tools (e.g. Ansible and
Terraform) when they're appropriate.

People have forgotten the subtle art of realizing what tool is best for the
job. If all you're doing is stitching together other tools and doing some
logging, bash is hard to beat.

~~~
adrianhel
^ this!

Bash is the best stitching together language you'll get. Sure it sucks at
scripting.. So stitch a script in there for God's sake!

~~~
h1d
Fish shell?

~~~
nameless912
Not portable enough. The point of bash (well, really sh if we want to be
pedantic) is that it's a stitching tool available on just about every
conceivable platform.

------
LinuxBender
I write long bash and perl scripts. I keep things neat and tidy most of the
time. My "temporary workarounds" are still in use today, in production, so I
must have written them well enough for people to understand. I see a lot of
debate around preferences at the risk of becoming a funny title on n-gate.
Just use whatever languages you know best. If it follows the best practices of
that language, someone can port it to their preferred language.

Maybe it is just my browser, but that xml file should be renamed to .txt or
have the mime type set differently. There is no formatting for me. It is just
one really long line.

~~~
occams_chainsaw
_My "temporary workarounds" are still in use today, in production, so I must
have written them well enough for people to understand._

Maybe the reason the temporary workarounds persist is because nobody
understands them well enough to replace them ;)

~~~
LinuxBender
I completely understand. In my case, every one of my scripts has my name and
email in it. They won't hesitate to reach out to me if something is broken.
:-)

------
dvfjsdhgfv
> Indent 2 spaces

It's interesting how quickly the 2-space indent became the new norm

~~~
andrewmcwatters
I'm very curious about why. In 8-space tab sizes, you have to very, very
carefully think about what you're fitting on that 80-col line after having in
most cases, already removed 10% of your column space. A lot of C resulted in
single character variable names and abbreviations that sacrificed immediate
readability, but with domain knowledge that gets resolved with time.

On the flip side of things, you have developers who universally write 240+ col
lines in HTML and don't know what a punched card is.

~~~
saagarjha
…I'm not seeing a strong argument for either in your response.

------
amptorn

        # Not this:
        if [[ "${my_var}X" = "some_stringX" ]]; then
          do_something
        fi
    

Why would anybody write this?

~~~
tobylane
They learned that $null = test is bad, that $a = $b is bad, but didn't learn
that you do either x$null = xtest or "$a" = "$b", not both.

------
textmode
Original HN title had the word "Google" in it.

According to this document, Google _requires_ Bash.

I do not use Bash. I use Almquist shell. I try to avoid using uncommon
"features" or utilities.

I am not a frequent Linux user but whenever I have to use it, all of the
hundreds of scripts I wrote using another OS and Almquist shell still work.
They all work in Bash.

Over the years, I used this site as a reference:

[https://www.in-ulm.de/~mascheck/](https://www.in-ulm.de/~mascheck/)

~~~
kevincox
They require bash for scripts.

You can use whatever shell you like on your machines.

~~~
textmode
"They require Bash for scripts."

I do not write scripts to run with Bash. I write them to run with an Almquist-
derived shell, which also means I can run them with Bash and other shells
without any problems. _However_ a script written in Bash may not run correctly
under Almquist shell and other shells.

------
raverbashing
"One example of this is Solaris SVR4 packages which require plain Bourne shell
for any scripts."

Such as?

------
jroseattle
" Indent 2 spaces. No tabs."

Oh dear, here we go....

------
Sir_Cmpwn
>Bash is the only shell scripting language permitted for executables.

Whelp, wrong right off the bat.

I'm gonna get down my my knees here and beg everyone reading: use POSIX shell.
Do not write scripts with bash. Do not write scripts with zsh. Do not write
scripts with fish. Use POSIX shell.

sh has a really bad interactive mode (so does bash), so I'm not gonna give
anyone a hard time for using another shell as their day-to-day interactive
shell. But, for the love of god, write your scripts with POSIX sh.

~~~
thankthunk
> Do not write scripts with bash.

It's a bit too late for that. Bash is pretty much the de facto default shell
for linux. Telling people not to use bash scripts is like telling web devs not
to use javascript or OS devs not to use C.

It may change in the future, but bash is so entrenched, it's going to take an
extraordinary use case for people to move from bash.

~~~
orzen
Bash is nowhere close for being de facto default shell for Linux. This is a
bad analogy. In this case it is more like telling people that want to code
Javascript to only write it in React. You got to go from something generic to
something specific. Scripts written with bash will often rely on bash-
specifics and can therefore only be interpreted by bash. There are entire
projects dedicated to stop this plague. If you want to write portable shell
scripts, use dash.

[https://linux.die.net/man/1/checkbashisms](https://linux.die.net/man/1/checkbashisms)
[https://mywiki.wooledge.org/Bashism](https://mywiki.wooledge.org/Bashism)
[https://wiki.ubuntu.com/DashAsBinSh](https://wiki.ubuntu.com/DashAsBinSh)
[https://en.wiktionary.org/wiki/bashism](https://en.wiktionary.org/wiki/bashism)

~~~
raarts
> Bash is nowhere close for being de facto default shell for Linux.

Oh, yes it is. Every important distribution uses bash as the default.

~~~
umanwizard
Ubuntu has dash as /bin/sh.

~~~
jhasse
But bash as the login shell.

~~~
gkya
Login shell means your interactive shell. It does not necessarily mean that
that shell should be used for scripting.

