
Command line tools for the novice - commadelimited
http://andymatthews.net/read/2012/07/24/Command-line-tools-for-the-novice
======
spudlyo
I knew before I even clicked on this article, that something inside was going
to cause me nerd rage; it's almost always the case when I read these _Baby's
first UNIX_ type articles. I thought this one might be different when I saw
Andy's formidable neck beard, and for the most part it was decent but then I
saw it.

 _..plus Ack is 4 or 5 times faster than grep._

Ack is implemented in Perl, which has a full featured regex engine, but is not
generally known for being super fast. It's going up against GNU grep, which is
a highly tuned regex engine, which is very fast. The ack guys make the
argument that ack is faster because it doesn't search stuff you, the
developer, don't care about, which in my estimation is horseshit. Try it
yourself, I threw a regex at my email spool from the 90s which is about 128M.
I made sure it was in disk cache first and then did:

    
    
        chunky:~$ time egrep -c -i '(d|m|j)ingle' RMAIL
        23
    
        real    0m3.129s
        user    0m3.075s
        sys     0m0.051s
    
        chunky:~$ time ack -c -i '(d|m|j)ingle' RMAIL
        RMAIL:23
    
        real    0m26.208s
        user    0m26.060s
        sys     0m0.121s

~~~
shurane
Yeah, I don't like it when people spout speed comparisons without backing it
up either.

A lot of people do want/need PCRE, which ack relieves the pain point for.

~~~
spudlyo
GNU grep also supports PCRE.

    
    
        chunky:~$ otool -L /usr/bin/grep | grep pcre; grep --help | grep perl
          /usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
          -P, --perl-regexp         PATTERN is a Perl regular expression
    

Many users choose ack over grep for the same reason they choose htop over top,
tmux over screen, zsh over bash, and postgres over mysql -- the perception
that one is significantly better than the other. This advocacy often comes
from folks whose use and understanding of either tool is superficial at best.

~~~
ralph
I often see ack recommended by word of mouth but unfortunately I feel it's
between programmers that don't understand the Unix command-line enough to
instruct grep, find, xargs, etc., sufficiently accurately to do what they
intend and ack has the allure of doing the right thing, though often slowly,
as you say. GNU grep's -I option to ignore binary files for example, or -boa
to search binary files.

Once the standard commands are learnt then by all means use ack if you
understand the trade-offs and can fall back to the normal way when on another
system or when ack's insufficient.

~~~
pydave
How would you exclude the same kinds of files that ack does in grep without
typing out the exclude patterns?

As described here ( <http://blog.sanctum.geek.nz/default-grep-options/> ), you
could use GREP_OPTIONS, but then you run into the problems here (
<http://brainstorm.ubuntu.com/idea/24141/> ) -- scripts that use grep without
clearing GREP_OPTIONS will fail.

Do you have an alias? Or another script that wraps grep?

~~~
spudlyo

        find . -type f -name \*.[ch] | grep -v \\.git | xargs grep something
    

Normally my directories aren't going to have .bzr, .git, .hg, and .svn in the
same directory tree, but if they did it wouldn't be that much harder.

    
    
        alias nocrap="egrep -v \\.\(git\|bzr\|hg\|svn\)"
        find . -type f -name \*.[ch]| nocrap | xargs grep whatever
    

This is how I do it because I'm lazy. I suppose I could tell find to exclude
directories so it wouldn't have to recurse into them, but the list of
directories and files in my source tree is almost always in buffer cache
anyway, so I never really worry about it.

~~~
ralph
You might want to consider using single quotes to protect your globs and
regexps more rather than backslashes. \✳.[ch] will match the ./✳.c that may
have been accidentally created by an earlier error, then find will only look
for a file called precisely ✳.c and nothing else. '✳.[ch]' or \✳.\\[ch] is
what's meant.

(Edited to use ✳ to workaround unescapable mark-up.)

~~~
spudlyo
Good point, thanks!

------
ralph
> In Grep grep -R 'some term' * , in Ack ack 'some term'. The -R tells grep to
> search through all directories and subdirectories while the * tells it what
> types of files.

No, that's not how it works with grep. The shell is expanding the glob * to
all the things in the current directory that don't start with a dot and that's
what grep sees. It then reads each of them in turn or if they're a directory
it descends into them recursively; the * plays no further part in the descent.

~~~
dhimes
I was under the impression that * expanded to all things not starting with a
dot in _all_ directories-- or better: in each directory that -R dictates. I
understand that "* tells it what types of files," isn't correct, but I don't
understand what you mean by "plays no further part in the descent."

~~~
noselasd
just run echo * in your directory. That's the same argument grep will
get(instead of the * ) when you pass it * .

grep will run through all those files, if any, and recurse through all the
directories, if any. grep recursing down a directory will process all the
files/subdirs in that directory. (which includes . files).

If the shell can't expand the * , grep will receive a literal * and try to
open a file/directory named * .

~~~
ralph
Personally, I `shopt -s failglob' in my .bashrc so any glob that fails to
expand is treated as an error by the shell and the command never gets started.
I then quote all special globbing characters that I intend to pass to the
command as it avoids the sometimes hard to spot errors where a glob expanded
unintentionally and the command didn't see the wildcards. Relying on non-
matching globbing to reach the command will bite one day.

------
jerhinesmith
I've wanted to try zsh for a long time now, but I worry about the mental
context switching I'll have to do when developing locally and then ssh'ing
into another server to perform other tasks (servers, I should add, where I
can't install zsh).

Is anyone in a similar situation? If you switched, was it worth it?

~~~
SkyMarshal
I switched cold turkey from bash to zsh a few months ago, and going back to
bash on servers hasn't been a problem. In my experience, if you've used bash
for a long time you don't forget it.

Minor tip - you can use the same .aliases file for both shells, I just source
it in .bashrc and .zshrc on my dev machine. So if I ever need to switch to
bash for some reason, my most used aliases are still available.

If you switch, don't forget oh-my-zsh [1].

1\. <https://github.com/robbyrussell/oh-my-zsh>

~~~
jvm
I'm actually not very taken with oh-my-zsh. It seems to add a layer of
complexity while not really adding anything you couldn't just paste into your
.zshrc. If it eliminated tinkering overhead that would be one thing, but in my
experience it just gave me a new layer of software that needed tinkering.

~~~
shurane
It's along similar lines to copying someone else's rc file willy hilly,
without understanding what you're getting. Extra layer of quirks and features
that you need to figure out, on top of the already existing layer.

------
dredmorbius
Grey text on grey background: <http://contrastrebellion.com/>

~~~
morsch
Contrast rebellion refers to _W3C's Web Content Accessibility Guidelines_ ,
which specify a contrast ratio of at least 4.5. Just for fun, I plugged in the
numbers, the site has a contrast ratio of about 4.9.

W3C: [http://www.w3.org/TR/UNDERSTANDING-WCAG20/visual-audio-
contr...](http://www.w3.org/TR/UNDERSTANDING-WCAG20/visual-audio-contrast-
contrast.html)

~~~
dredmorbius
It's not nearly enough for my liking.

I'll take a look when I've got a few more moments to see where various of the
W3C's recommendation levels fall on my own assessment of readability.

Speaking as a 40-something with decent vision, the site isn't impossible to
read, but it's most definitely annoying. For someone with compromised or old
eyes, it would be much worse.

~~~
ralph
I agree, annoying, and I didn't realise many others thought the same. Thanks
for the link, I've tried to spread the word.
[https://plus.google.com/115649437518703495227/posts/iWtSJhR8...](https://plus.google.com/115649437518703495227/posts/iWtSJhR8QpA)

------
commadelimited
I'd love to hear other tools that you guys are using for your daily workflow.

~~~
sthatipamala
This was a good list for the intermediate command line user looking to
customize. For the absolute newbie I'd reduce it down to git/hg, ssh, grep/ack
and curl.

For crazy power users, I'm a huge fan of vim+tmux+zsh combo as described in
<http://www.drbunsen.org/text-triumvirate.html>

~~~
commadelimited
That's a good list. I considered including Git but that could be an entire
article by itself. I've looked at TMux but I just don't I'd use it.

~~~
devgutt
What's the difference between Tmux and iTerm2?

~~~
johncoltrane
tmux is a CLI terminal multiplexer. As an interactive CLI program, you need a
shell to use it. To run such a shell, you need a terminal emulator. iTerm2 is
a Mac OS X only GUI terminal emulator.

You can run tmux in iTerm2.

------
clvv
Tools similar to z:

1\. [Autojump](<https://github.com/joelthelion/autojump>), first
implementation and what z was based on, written in python. Functionality
similar to z.

2\. [Fasd](<https://github.com/clvv/fasd>), feature-rich fork of z, support
for files, support for more shells and platforms (BSDs, Android).

------
jasonlotito
tmux - Without a doubt, the biggest upgrade to my workflow in years. If you
aren't using tmux (or at least screen), you are doing yourself a big
disservice.

Also, I keep seeing iTerm2 get mention. You mentioned split panes and what
not, which are not critical when I have that with tmux (as well as the tabs).
And while before Lion, Terminal's color support was weak, that's changed. So,
I really wonder what the big reason for using iTerm2 is over Terminal?

~~~
tocomment
What's the benefit from tmux? Why is it better than just using multiple
windows?

~~~
jasonlotito
Because I can shut down my computer, move to another one, and pick up right
where I left off. This means even if I leave a process running, it doesn't
stop.

tmux is like screen, only it has more features.

Which is really why I asked. What does iTerm2 offer that I can't get from tmux
and current terminal? People always refer to iTerms ability to do multiple
windows/tabs. For me, this is useless if it's just available client side.

There is an amazing amount of freedom being able to just shut down, move to
another computer, and be able to pick up exactly where I left off.

~~~
tocomment
So you have to install tmux on the server?

------
cltnoven
So, first (and most obvious) point of interest. Your function "server" will
not work. It contains "python -m SImpleHTTPServer .." .. Notice the typo? It
should be SimpleHTTPServer. In addition, that other fluff around it is
unneeded in general. Just "python -m SimpleHTTPServer $port" is almost always
"good enough", though the SimpleHTTPServer module contains severe bugs.
Secondly, your link right above that snipped to Paul Irish contains an href=""
which, obviously, links it to your own page. It makes it seem like you're
trying to take credit for his work. This is all sloppy and bad form.

Honestly, anyone who puts code into a blog post without even bothering to RUN
IT THEMSELVES first should get out of the business of writing even slightly
technical posts. Feel free to review food without tasting it first instead; at
least then it's subjective so you can't be wrong. Here, the "/usr/bin/python:
No module named SImpleHTTPServer" you'd get if you had arsed yourself to
actually check your code to be valid easily clues the reader into your
carelessness.

In addition to all that, there's the audacity of posting a post for the
"novice" to hacker news when the post is clearly neither relevant for any
hacker who has performed any real work or has any real intelligence (they
would already know all the mentioned tools) and neither is news for all the
tools mentioned are fairly old and well known. You might as well just call us
all stupid and dilute the front page further with crap.

~~~
commadelimited
Citnoven...

I appreciate you taking the time not only to read the post, but to offer your
opinion and corrections to my article. I've updated those two mistakes you
pointed out.

As I understand it, SimpleHTTPServer offers basic functionality to the user,
which is all I use it for...a quick way to view pages using AJAX requests. I
attempted to use http.server instead, but as I'm not running Python 3.0 I'm
not able to use that option. Since Lion ships with Python 2.7, and many people
are using that OS, I felt safe leaving the reference in place.

I do run the server function myself, many times a day in fact, but I made the
mistake of typing it in manually rather than copying and pasting from my
.zshrc file.

This post isn't intended for hackers, thus the title "...for the novice". The
comments I've received about the article thus far, most likely from people not
as advanced as yourself, have been positive and appreciative for the
information. I'd love to hear some of the tools that improve your workflow
daily...would you care to share them?

