
Testing Bash with BATS - samebreath
https://opensource.com/article/19/2/testing-bash-bats
======
zenlikethat
My experience doing integration testing with BATS was that it was complete
misery (not the BATS project’s fault, I think it’s just inherent in the nature
of the thing).

You REALLY want a real programming language for any kind of non trivial
integration testing. Things like set up and tear down, checking outputs, and
testing many-similar-but-not-identical cases gets hairy fast. Sure you CAN do
it in bash but just because you can doesn't mean you should. When we rewrote
our BATS tests in Go was a huge step forward for the project.

~~~
dnautics
One time I had to write a Json marshaller in bash (don't ask), and bats was
invaluable. BATS is great when you need to unit test bash. And for the most
basic integration tests on something that is meant to be a CLI.

------
orestes910
Outside of the scope of elaborate CI pipelines, I wonder how useful this can
really be.

Big CI pipelines are one of the few instances where I can think of Bash being
both an appropriate choice AND the resulting product being large, elaborate,
and sensitive to failure - which would benefit from being tested. Most other
applications of Bash are generally just so simple that fundamentally altering
how you write scripts ("Bash scripts must also be broken down into multiple
functions, which the main part of the script should call when the script is
executed.") for the sake of testing them seems like it could easily fall into
the category of over engineering.

Beyond the CI pipeline use case, wouldn't the tools in which this would
actually be properly useful be better off written in a proper programming
language?

~~~
LukeShu
Whenever someone says "most people don't write large Bash scripts", I have to
chime in and say that I would consider most GNU/Linux distros to be giant
piles of shell scripts.

A year or two ago, Arch Linux migrated the tests for dbscripts (the server-
side of how package releases happen) from shUnit2 to BATS.
[https://git.archlinux.org/dbscripts.git/](https://git.archlinux.org/dbscripts.git/)

~~~
chubot
_I would consider most GNU /Linux distros to be giant piles of shell scripts._

Yes! That was one of the primary motivations for my Oil project [1]. I was
building containers from scratch with shell scripts (in 2012 or so, pre-
Docker), and I was horrified when I discovered how Debian actually works.

 _Why Create a New Unix Shell?_
[http://www.oilshell.org/blog/2018/01/28.html](http://www.oilshell.org/blog/2018/01/28.html)

And of course it's not just Debian. Red Hat, Fedora, Alpine, etc. are all big
packages of shell scripts, often mixed with other ad hoc macro processing or
Makefiles. Alpine does this funny hack where their metadata is in APKBUILD
shell scripts, which is limiting when you want to read metadata without
executing shell.

I also point out in that post that Kubernetes is pretty new (2014) and it has
48,000 lines of shell in its repo.

That's true of most cloud infrastructure. If you deal with Heroku, OpenStack,
Cloud Foundry, etc. there is a ton of shell all over the place. With
buildpacks, Travis CI, etc.

And that's obviously not because the authors of those projects don't know what
they're doing. Shell is still the best tool for that job (bringing up Unix
systems), despite all its flaws.

The world now runs on clusters of Unix machines, and in turn big piles of
shell scripts :)

\-----

New release here if anyone wants to help me test: _OSH 0.6.pre15_
[http://www.oilshell.org/blog/2019/02/18.html](http://www.oilshell.org/blog/2019/02/18.html)

Caveat: it's still too slow; I'm mainly looking for people to help test it.

Running bats tests with it would be great. I don't know how bats works with
the @test annotation? That doesn't look like valid shell syntax.

OSH also opens a lot of new opportunities for people interested in testing /
static analysis and shell.

There was some discussion here, even related to bats, but I'm not sure where
it went:
[https://github.com/oilshell/oil/issues/200](https://github.com/oilshell/oil/issues/200)

[1] The other being how hard it is to write an autocompletion script!

~~~
jacques_chester
> _If you deal with Heroku, OpenStack, Cloud Foundry, etc. there is a ton of
> shell all over the place. With buildpacks, Travis CI, etc._

Cloud Foundry Buildpacks were rewritten in Go and the (quite elaborate)
automation for manufacturing them[0] is mostly test-driven Ruby these days.

I can't claim credit for the former but I had a hand in the latter. Bash is
not a sane production language.

[0] [https://github.com/cloudfoundry/buildpacks-
ci](https://github.com/cloudfoundry/buildpacks-ci) (though I note someone
snuck in some Crystal and I suspect it was Dave).

~~~
chubot
Also, I looked at the buildpacks-ci repo you linked, and it still has ~2000
lines of shell in ~80 files.

For comparison, there's ~12,500 lines of Ruby.

Another issue I have is that if Kubernetes replaces its 44,000 lines of shell
with 100,000 lines of Go (or probably more), that's a mixed result at best.
There's just a lot of logic to express and shell does it concisely.

Of course, without a better shell, I don't blame them if they switch, but it's
still a suboptimal state of affairs.

------
faitswulff
Tangential question, but are there any efforts to treat shell scripts as a
compilation target? It seems like such an insane language, I don't know why
another nicer language hasn't emerged on top of it.

~~~
freedomben
> _It seems like such an insane language_

I can understand that because I thought that way too before I really learned
it.

Bash is a language like any other: it takes reading and practice to get good
at it. Also most of the WTFs in Bash actually make decent sense when you
consider that _whitespace is the delimiter_ in bash (for example setting a
variable is VAR='value' instead of VAR = 'value'. That trips newer people up
pretty badly at first).

Most people also abandon good software practices when they write bash for some
reason. For example, you'd never write 100 lines of ruby or javascript without
declaring functions, yet people do that all the time in bash (they shouldn't).
If people followed good practice, I think bash would seem a lot less insane.

~~~
nicoburns
I think lack of proper arrays and structured data types is a big issue with
bash. Basically everything is a string, with all the issues around escaping
and corner cases that that implies

~~~
JetSpiegel
Bash (4 at least) has proper arrays and hash maps, but granted, the syntax is
hideous. The arrays are also limited, but hash maps are somewhat sane.

> declare -a ARRAY > declare -A MAP

------
jasonpeacock
Yes! So much yes. Many developers write crappy Bash scripts because they don't
see it as a real language, but it's an interpreted language just like Ruby,
Python, Perl, etc.

If you're going to design modular libraries and scripts with best practices,
unit tests, etc, for those languages, then you should be doing the same with
Bash.

If you're writing production scripts using Bash, you should be following
software best practices - don't give me any "it's just Bash" excuses.

 _Mocking_ In addition to bats-mock (part of the bats library in the article),
there is another mocking approach:
[https://pbrisbin.com/posts/mocking_bash/](https://pbrisbin.com/posts/mocking_bash/)
They both have their uses.

 _Documentation_ NaturalDocs (
[https://www.naturaldocs.org/](https://www.naturaldocs.org/) ) can happily
parse & extract doc comments from your bash scripts & libraries:
[https://www.naturaldocs.org/](https://www.naturaldocs.org/) Just define your
own language:

[https://www.naturaldocs.org/reference/languages.txt/#adding_...](https://www.naturaldocs.org/reference/languages.txt/#adding_languages)

    
    
      Format: 1.4
      
      Language: bash
      Shebang Strings: bash
      Extensions: bash sh
      Ignore Extensions: bats
      Line Comments: #
    

_Linting_ Use shellcheck, it's amazing:
[https://www.shellcheck.net/](https://www.shellcheck.net/) If you use Vim,
w0rp/ale knows how to integrate with it for real-time feedback:
[https://github.com/w0rp/ale/tree/master/ale_linters/sh](https://github.com/w0rp/ale/tree/master/ale_linters/sh)

 _Argument Parsing_ Use Docopt ( [http://docopt.org/](http://docopt.org/) ) to
get modern arg-parsing in your Bash scripts. Don't waste your time trying to
roll your own or use the built-in anemic arg-parsing. Shells implementation:
[https://github.com/docopt/docopts](https://github.com/docopt/docopts)

 _Use Bash built-ins_ Stop invoking external processes for features already
available in Bash: [https://github.com/dylanaraps/pure-bash-
bible](https://github.com/dylanaraps/pure-bash-bible)

More good resources:

* [http://www.tldp.org/LDP/abs/html/](http://www.tldp.org/LDP/abs/html/)

* [https://www.gnu.org/software/bash/manual/bashref.html](https://www.gnu.org/software/bash/manual/bashref.html)

* [http://mywiki.wooledge.org/BashGuide](http://mywiki.wooledge.org/BashGuide)

BUT why are you using Bash? Go use a modern language and don't suffer the
idiosyncrasies of Bash where you have to re-invent many common libraries
already available like JSON parsers, etc.

My favorite Bash replacement is Python + Plumbum:
[https://plumbum.readthedocs.io/en/latest/](https://plumbum.readthedocs.io/en/latest/)
All the convenience of Bash when invoking commands (no more subprocess()!)
with the joy of Python.

~~~
nomel
> Linting Use shellcheck

Shellcheck has saved me countless times, but I noticed it's hard to get people
to adopt it into their workflow.

I've introduced probably a dozen people to shellcheck and warn them ahead of
time that their scripts are broken. They try it, see a flood of errors and
warnings, close the window, and never use it again.

The logic surrounding their undefined variables and improper conditions happen
to work as hoped, by luck, combined with their strict adherence to "it's best
practice to not use spaces" both keep their scripts crawling along "just
fine".

~~~
yesenadam
I just went to install it with macports, it says:

"The following dependencies will be installed: autoconf automake bzip2
clang-3.3 clang-4.0 clang_select cmake curl curl-ca-bundle db48 expat gcc6
gdbm ghc ghc-bootstrap glib2 gmake gmp help2man hs-json hs-mtl hs-parsec hs-
primitive hs-quickcheck-devel hs-random hs-regex-base hs-regex-tdfa hs-syb hs-
text hs-tf-random icu isl ld64 ld64-97 legacy-support libarchive libcxx
libedit libffi libgcc libgcc6 libgcc7 libidn2 libmpc libomp libpsl libtool
libunistring libuv libxml2 libxslt llvm-3.3 llvm-3.5 llvm-4.0 llvm_select lz4
lzo2 m4 mpfr openssl p5.28-locale-gettext pcre perl5 perl5.26 perl5.28
pkgconfig python27 python2_select python_select readline sqlite3 texinfo xar
xattr zlib zstd Continue [Y/n]:" ..uh maybe not right now.

~~~
woodruffw
It looks like the MacPorts version of shellcheck is substantially out-of-
date[1].

`brew info shellcheck` shows just three (build-time) dependencies: cabal-
install, ghc, and pandoc.

[1]: [https://github.com/macports/macports-
ports/commits/master/de...](https://github.com/macports/macports-
ports/commits/master/devel/shellcheck/Portfile)

~~~
l2dy
Yes, it hasn't been updated because dependencies have to be updated first. See
recent discussion on updating ghc:
[https://trac.macports.org/ticket/48899#comment:43](https://trac.macports.org/ticket/48899#comment:43).

~~~
yesenadam
Oh thanks! That's a fascinating discussion.

------
fxfan
Im probably in minority but I use powershell on my arch and love it. Finding
duplicates has never been so easy-

`Get-childitem -Recurse |% {[System.Tuple]::Create( $_.FullName, (md5sum
$_.FullName).split( " " )[ 0 ] ) }) | Group-Object Item2 | Where-Object {
$_.Count -ge 1}`

~~~
samebreath
You win for enthusiasm when it comes to that syntax. I appreciate I pipe
output to uniq (MacOS) to get uniques, proving that easy is an extremely
relative value :)

~~~
fxfan
Use powershell ;)

It's wonderful.

