
Stronger Shell - mattbowen
http://m.odul.us/blog/2015/8/12/stronger-shell
======
heydonovan
Just about everything I've learned about bash has been from the #bash IRC
channel on Freenode. You'll see the same repeated warnings of not learning
from the public web, due to the fact that misinformation spreads like a wild
fire, and some articles out there are just flat out wrong (sort of like
w3schools in the #css circle).

There is just one thing I will _never_ understand. The tired argument of
"...but it isn't portable". Features that are new, and make programming bash
easier (such as "[" vs "[["), are looked down upon in some circles because it
isn't portable. If I'm programming for bash, then upon deployment, I'll be
using bash. To use another shell, and hope it just works is a bit insane.

I'd also like to second using the Fish shell. Far better than bash/zsh for
everyday use, and I just can't go back to other shells.

~~~
StavrosK
Seconded. I can't live without fish, and I hear this "but what about my bash
scripts?!" argument a lot. Unless you're actually sourcing the script, it has
a magic "#!/bin/bash" line on top that makes it work correctly!

~~~
baghira
It's not that simple. Changes in the syntax for strings and environmental
variables are going to bite precisely newcomers who would benefit the most
from fish. And sourcing happens.

I do know that all these problems result from fish having a saner syntax
compared to bash (hell, FWIW I'm still mad that globs are not regular
expressions and have a different syntax) but everytime someone points out
these problems the reaction is "#!/bin/bash", which kinda misses the point.

------
roneesh
I'd be remiss if I didn't use the comments to mention the amazing and totally
free, "Unix for the Beginning Mage". The author of the article also didn't
mention this great resource. It's an amazing book and a short read. It took me
about 3 hours to work through completely. www.unixmages.com

If you don't know how to use your shell, block yourself off an evening with
this book and change how you forever use you computer.

~~~
faitswulff
I, too, was looking for this in his list. It's a really great resource.

------
wereHamster
> You must have spaces inside your test expressions (so [[-z $FOO]] won't
> work; [[ -z $FOO ]] is correct).

That's because '[[' is a special bash extension which started out as a better
'[' which is actually a binary (/usr/bin/[). And, well, you have to have a
space between an executable and its first argument.

(nowadays '[' and '[[' are builtins in most shells, but the external binary
still exists).

~~~
vodik
The core difference is that:

\- `[` is treated as a statement

\- `[[` is treated as an expression

Which, for example, is why you can't use `>` in `[` based expressions. Bash
thinks you want to redirect the output of `[` to somewhere else. This
restriction is lifted for `[[` and makes for much more natural looking code.

------
realharo
I personally can't stand bash for writing scripts (it's fine for typing one-
liners into terminal though). This question and its top answer capture the
insanity of bash quite nicely [http://stackoverflow.com/questions/3601515/how-
to-check-if-a...](http://stackoverflow.com/questions/3601515/how-to-check-if-
a-variable-is-set-in-bash) .

~~~
david-given
Is there a better shell? By which I mean more flexible, more consistent, and
simpler. (Anyone who says 'zsh' is disqualified.)

I've briefly looked at rc, but while it's significantly simpler and more
orthogonal than sh it's got it's own weirdnesses; but the 'Design Principles'
section here is worth reading: [http://plan9.bell-
labs.com/sys/doc/rc.html](http://plan9.bell-labs.com/sys/doc/rc.html)

Surely there must be a usable shell wrapped around an actual modern language?

~~~
useerup
> Is there a better shell? By which I mean more flexible, more consistent, and
> simpler.

Yes, PowerShell. And it may actually arrive at Linux/Unix soon, due to
CoreCLR. The specification is already open, but until now not much progress
has been made towards bringing it uncrippled to Linux/Unix

PowerShell is extremely consistent. Examples: All commands are strictly on the
_verb-noun_ form where there are only 40 or so "approved" verbs. Noun
represent the topic, so if you are working with ACLs, the commands are Get-
Acl, Set-Acl, and if you are working with network network adapters, the
commands are Disable-NetAdapter, Enable-NetAdapter, Get-NetAdapter, Rename-
NetAdapter, Restart-NetAdapter, Set-NetAdapter. All (PowerShell) commands
require the parameters in the same way (no - and -- confusion) as it is
actually the _shell_ that does the parameter parsing.

The grammar is "modern" so it has no surprises for anyone used to context-free
grammars (like C, Python, Java, ...).

PowerShell is certainly flexible as it comes with the ability to invoke
anything that has an exposed .NET API, or even has a WBEN/CIM standard
interface (like some network switches etc).

~~~
felixgallo
I had an opportunity to use PowerShell recently. The task was simple: on a
bunch of freshly installed windows 8 boxes, kill a running graphical program
of a certain name, copy over new executables, and restart that graphical
program. The kind of thing you do essentially constantly on Unix boxes.

I urge anyone at all with any interest in PowerShell to give it a shot.
Spoiler: it's not possible without an epic level of hackery; but along the
way, you will uncover that PS uses the same parameter to mean very different
things for different commands, that the 'everything is an object' conceit
doesn't work when the object you want doesn't exist, that PS is chock full of
hacky, revolting cruft _already_ despite its youth, hacky revolting cruft that
makes oh-my-zsh look like /bin/rc, that most windows commands will fill your
screen with banner(8) style output even on success, and that no two windows
commands are alike.

Given which decade it was developed in, PowerShell represents the most
absolute and fundamental failure of software I've had the pleasure of
encountering in the last five years; and I've seen Windows 8.0. Anyone who
thinks PowerShell is at all any good has, axiomatically, never seen any other
shell besides cmd.exe before. You may think I'm exaggerating. Please, try the
example above, then report back here with a list of what you had to do.

~~~
useerup
Using the non-aliased verbose form of the cmdlets:

    
    
        $hosts = 'host1','host2','host3'
        $source = '\\host0\c$\source'
        $dest = 'c:\dest'
        $executable = 'C:\Program Files (x86)\Notepad++\notepad++.exe'
    
        Invoke-Command -Computer $hosts -ScriptBlock { 
            Get-Process | Where-Object Path -eq $using:executable | Stop-Process -Force
            Copy-Item -Path $using:source -Destination $using:dest
        }
    

Explanation:

Line 1-4: Set up variables to make the script more explanatory. Line 1 defines
an array (the "," operator)

Line 6: Invoke-Command takes an array of (remote) hosts (the -Computer
parameter) to execute the script block (the -ScriptBlock parameter).

Lines 7-9: The script to execute at each host simultaneously (the Invoke-
Command executes the scripts in parallel at each host)

Line 7: Get the process list (Get-Process), pipe through the filter (Where-
Object) which selects only the process(es) executing the desired executable,
pipe those processes to the Stop-Process cmdlet which will forcably stop the
process.

Line 8: Copy files from the desired source at an UNC path to the local machine
at the destination

Now, the above script was the _canonical_ way, using the long form. For casual
scription, I could have written just this:

    
    
        $hosts = 'host1','host2','host3'
        $executable = 'C:\Program Files (x86)\Notepad++\notepad++.exe'
        icm $hosts { ps | ? Path -eq $using:executable | kill -f; cp \\host0\c$\source c:\dest }

~~~
warfangle
While likely to work (I don't use powershell), I think you missed the main
constraint posed: kill a running GUI with a certain name.

Your solution finds the process to kill only if the executable path is at a
known location.

How would you do this if you only know what the process name will be -- e.g.,
what if the executable path is on D:\stuff\gui.exe on host1 and
E:\secretstuff\gui.exe on host2, if gui.exe always runs as a process named
"Updatable GUI Thing"?

~~~
useerup
Silly me, I assumed that the executable path was the requirement. My bad. If
you need to find a process just by it's name, it is even simpler: Just
indicate the process name (possibly with wildcards) to Get-Process (alias _ps_
):

ps Notepad++ | kill

Actually, _kill_ (alias for _Stop-Process_ ) takes a name parameter directly,
so the "kill" line from the script could be written as simply

kill notepad++

But your question is actually really good, because what if we did not know the
neither the process name nor the executable, but -say- only the _Window
title_? Get-process (alias _ps_ ) will produce a sequence objects describing
the running processes. If I want to know what properties those objects have
that I can possibly filter on, I can pipe the objects through the Get-Member
cmdlet (alias _gm_ ):

    
    
        ps | gm
    

This produces a table-formatted list like this (shortened):

    
    
        TypeName: System.Diagnostics.Process
    
        Name                       MemberType     Definition
        ----                       ----------     ----------
        Handles                    AliasProperty  Handles = Handlecount
        Name                       AliasProperty  Name = ProcessName
    	...
        MainModule                 Property       System.Diagnostics.ProcessModule MainModule {get;}
        MainWindowHandle           Property       System.IntPtr MainWindowHandle {get;}
        MainWindowTitle            Property       string MainWindowTitle {get;}
        MaxWorkingSet              Property       System.IntPtr MaxWorkingSet {get;set;}
    	...
        Site                       Property       System.ComponentModel.ISite Site {get;set;}
        StandardError              Property       System.IO.StreamReader StandardError {get;}
        StandardInput              Property       System.IO.StreamWriter StandardInput {get;}
        StandardOutput             Property       System.IO.StreamReader StandardOutput {get;}
        StartInfo                  Property       System.Diagnostics.ProcessStartInfo StartInfo {get;set;}
    	...
        Product                    ScriptProperty System.Object Product {get=$this.Mainmodule.FileVersionInfo.ProductName;}
        ProductVersion             ScriptProperty System.Object ProductVersion {get=$this.Mainmodule.FileVersionInfo.ProductVersion;}
    
    

Lo and behold, there is a property called _MainWindowTitle_. So to stop a
process by it's main window title I could write:

    
    
        ps | ? MainWindowTitle -eq 'Deepthought Main Console' | kill
    

That is, find all processes, pipe them through a filter selecting only those
where the _MainWindowTitle_ equals the desired text, and pipe those processes
to the Stop-Process cmdlet

------
bryanlarsen
How many people learned shell scripting while engaged in morally ambiguous
activities? For me it was scraping for porn ~25 years ago. I imagine lots of
people learned scripting while "penetration testing".

------
alpsgolden
I've found that as soon as bash script goes to more than a couple lines, or as
soon as it needs anything modestly complex, like "if" statements or functions,
then it is almost always more efficient to write it in ipython. If you know
python, then ipython is really superior way to do any ops and administration
tasks. I've also found it easier to use for server setup than alternatives
like ansible or puppet. I can just use standard, reusable classes and
functions to do all my various activities.

~~~
falcolas
Perhaps it's just me, but flow control statements and functions in Bash have
yet to scare me off. It's still faster, and readable, to write even moderately
complex scripts in bash.

This is doubly true if you have to chain together external scripts - the
"easy" ways to do it in Python can have some very major limitations.

I liken Bash scripting to Perl scripting. You can write safe, beautiful, and
readable code in both. However, you can also create a summoning circle to the
5th circle of hell if you don't spend the effort to make it safe, beautiful
and readable.

~~~
alpsgolden
Part of this was that I became an expert in python before learning much bash,
so it was just much easier to write in the language I already knew, and I
found the bash syntax strange with a bunch of gotchas. I'm sure though, that
if I did a ton of bash scripting I would not even notice the gotchas.

You are correct that the main weakness of ipython is that it doesn't have
pipes, and that chaining is very limited. But usually this does not impact me.
And if all your scripting is in python, then you just import functions and run
them, rather than chaining scripts together.

------
tachion
What suprises me is how often Bash is being used as 'the shell' and software
projects use it, even when they could simply call /bin/sh instead. This is
particularly bad for people who port software to other platforms (where BSD
systems dont come with Bash, for example) and have to deal with pure shell
scripts calling Bash. One recent example is CoreOS/etcd that's currently
dropping Bash in favor of Sh, because they simply didnt need it.

Bash is a shell, but Bash is not the shell!

~~~
philh
It may be better for those people to call /bin/bash, and fail early and
obviously for anyone who doesn't have that; than to call /bin/sh, attempt to
write portably without really knowing the quirks of this variant of the
language, and possibly fail subtly in obscure situations.

------
danharaj
Confession: The horrors of bash set back my programming hobby until high
school. Even then, I didn't actually like interacting with my computer
programmatically until I found a comfortable set of tools sometime in college.
I'm lucky that my day job lets me use the tools I like and avoid systems like
the bash shell.

~~~
ParadigmBlender
Interestingly my experience has been opposite. My first programming adventure
was automating some file validation using bash. I agree though that the silly
and inconsistent syntax made me really appreciate python when I started
learning it.

------
_yy
If you're writing shell scripts, you might as well write Ansible playbooks.
You can use Ansible playbooks locally without the SSH layer.

Bash/sh is a horrible programming language.

------
TazeTSchnitzel
You don't need to learn Bash to write shell scripts. Many other languages
support executing commands with special syntax (PHP and Perl have backticks
for example).

I write my scripts in PHP if they're moderately complex, and only use Bash for
dumb lists of commands.

------
enibundo
actually _[_ is same as _test_ and the last _]_ is just the last (useless)
argument of _[_ , iirc

~~~
leni536
Actually you most probably have /bin/[ on your filesystem as a standalone
'test' executable or a link to 'test'. [ is a shell builtin in bash though.

------
crimsonalucard
interviewers tend not to test for shell skill even though your shell skills
are usually more relevant to daily tasks. I'm debating whether to improve on
my shell skills or algorithm skills.

~~~
gtk40
My interview for the job I have now actually had me do some hands-on shell
activities and had very little algorithms, but that was partly because they
were wanting me to work with code deployment and other tasks where it would
come in handy.

------
CJefferson
One this which greatly helped me was understanding that [ is a program, like
any other, which takes arguments. It's a small thing, but made the shall less
magic.

------
anh79
Bash is good as a system shell. For interactive shell, zsh/fish/... wins.

That's why I only learn Bash. One lesson for everything for both purposes ;)

~~~
qznc
fish would be better than bash as a system shell as well. More consistent, for
example. However, bash is available and usually preinstalled everywhere, in
contrast to fish.

~~~
anh79
I agree. But "fish" is quite strange to me, especially its way to recall
history. In "Bash", I can type "^ R" and browser the history very fast. In
"fish", I have to type and select with up/down array; it's not easy to browser
the history randomly (FIXME).

My #Bash" history has > 96k entries (woh, believe it or not; because of this
too big number, "xterm" \+ "screen" always stuck when exitting; but "urxvt" \+
"tmux" work perfectly thanks to "urxvt" daemon mode.)

Porting this huge history database from my daily "bash" to "fish" is just a
nightmare ... :D

~~~
ridiculous_fish
Actually fish will import your ~/.bash_history automatically!

------
chriswarbo
Was worth reading if only to find out about ShellCheck.

My current project has a lot of bash. The program itself is just `foo | bar |
baz | ...`, but the associated test script has grown quite long.

I've just added an extra test which calls shellcheck on each script, and it's
spotted a bunch of redundant code for me :)

------
estrabd
I generally avoid aliases. I prefer to rote learn complex commands; if it's
fitting for an actual shell script then I'll just bang one of those little
guys out in no time.

------
IshKebab
Please don't write shell code that you _ever_ intend to distribute to
_anyone_. The future will thank you.

------
jchomali
Nice! Thanks for sharing!

