
Ask HN: Best Unix Tricks? - scottjackson
HN, what are your killer Unix tips? I've been using and learning about Unix both at university and at home for almost a year, and I was wondering -- what are the things that, when some wizard showed them to you, you wrote down as soon as you could so you wouldn't forget them? Is there some trick that saves you a buttload of time? Some script that makes your job way easier? Some command that made you smack your forehead in frustration when you found out it could <i>be</i> that simple?<p>Note: I'm personally looking for tips for working in bash (specifically OS X stuff if that makes any difference), but if there's some amazing thing that only works in zsh under Debian or something, feel free to suggest those too -- it's sure to be useful for someone.<p>I had a quick search for previous topics on this, but all I found was http://news.ycombinator.com/item?id=103725 which had a couple of badass things in the discussion.
======
jacquesm
It's a great kill-ur unix tip but you should never fall for it:

    
    
        :(){ :|:& };
    

Oh, the temptation... if you're thinking of cutting & pasting that into the
shell of your machine google for 'bash fork bomb' first, I've left off the
final ':' to set it off.

Anyway, on a more serious note, after an installation procedure that was a lot
of work, if you want to document the whole thing use:

    
    
        history | cut -d ' ' -f 5-
    

To give you all the commands you executed in the last little while, cut &
paste that and you have a pretty good start for a how-to.

It's not a 'great killer unix tip', but it works for me.

Edit: Another one just came to mind, the ps command has a very useful option
that is quite undervalued imo:

    
    
        ps ax --forest

~~~
etherealG
I usually use ps -auxf for the best output on linux. unfortunately the OSX
version of ps doesn't support foresting, so sad.

------
acg
On OS X pbcopy and pbpaste are useful for working with clipboard data. Copy
data from a web-page or whatever, manipulate it on the command-line to get
what you want. Putting reasonably large files in the clipboard can be useful
too.

For example:

    
    
      curl http://news.bbc.co.uk/ | pbcopy # copies a web-page to clipboard
    

Depending on what you are doing it can be very useful:

    
    
      pbpaste | sort | pbcopy # sort the contents of the clipboard.
    

Another useful feature of OS X is open, which uses the finder to open a file:

    
    
      open file.pdf
    
      open music.mp3

~~~
rudd
You can also specify the app to open using the -a flag and specifying the full
name of the app (with correct spacing and capitalization), e.g.:

    
    
        open -a 'TextEdit' foobar.txt
        open -a 'TextMate' foobar.txt

~~~
zefhous
I also find it useful to have multiple instances of certain applications
sometimes:

    
    
      open -na safari

------
silentbicycle
Read man pages and "The Unix Programming Environment" by Kernighan and Pike.
It's old, but conveys the design principles behind Unix, particularly how to
write little tools that work well with all the others. "The Art of Unix
Programming" is also good, despite Eric S. Raymond's getting on a soapbox all
the time.

The 4.4 BSD supplementary books (PSD, USD, SMM, etc.) are full of major
documentation, with chapters on vi, awk, cc, yacc, gprof, etc. They're very
old-school. (OpenBSD installs them with its default documentation, I think
FreeBSD does as well.) Some parts will be obsolete, but the big ideas are the
same. For example, it covers lex and yacc better in about ten pages each than
the O'Reilly book could in twenty times that. Same with make, and several
other major parts in the Unix toolchain.

For starters, learn to use: apropos, cron, the basics of sed and awk, the
shebang, a _real_ editor, screen or tmux, tar, gzip, and find. Read man pages,
see where they lead you. Try to automate stuff, and go spelunking in the
infrastructure for whatever Unix distro you use.

I also found that I learned a LOT by getting away from stuff that assumed
Linux. Soooo much Linux newbie stuff is written by people who only know Linux
(and may use GNOME for most things, really). You don't really learn Unix that
way. Also, watch out for people who assume that bash is the standard shell. (A
pet peeve of mine.)

------
subbu
cd - (Go back to previous working directory)

sudo !! (Run the last command as root)

:w !sudo tee % (Save a readonly file in VIM)

> tmp.file (Empty a file)

cmd !$ (Run cmd with previous commands arguments)

They are just some of my favorites.

~~~
philh
That should be cmd !*. !$ would only be the final argument to the previous
command, which you can also insert with alt-. (and pressing alt-. again will
cycle through final arguments to earlier commands).

Adding :p to the end of a history expansion (like !$:p) will cause it not to
be executed, but the expanded command line will be echoed and put in your
history so you can check edit it.

~~~
decklin
M-. also takes a numeric argument (M-number or ESC number) to yank the nth
word instead of the last.

------
aseeon
<http://www.commandlinefu.com>

~~~
gtani
[http://www.makeuseof.com/tag/4-websites-to-learn-cool-
linux-...](http://www.makeuseof.com/tag/4-websites-to-learn-cool-linux-
command-line-tricks/)

<http://www.hiraeth.com/alan/tutorials/courses/Unix-Tools.pdf>

and Powers / Peek' oreilly book "Unix Power tools" on google books

================================

IBM had a good series:

[http://www.ibm.com/developerworks/linux/library/l-11sysadtip...](http://www.ibm.com/developerworks/linux/library/l-11sysadtips/index.html)

[https://www.ibm.com/developerworks/linux/library/l-10sysadti...](https://www.ibm.com/developerworks/linux/library/l-10sysadtips/)

<http://www.linuxformat.co.uk/wiki/index.php/58_Cool_Hacks>

[http://www.ibm.com/developerworks/aix/library/au-
productivit...](http://www.ibm.com/developerworks/aix/library/au-
productivitytips.html)

------
jrockway
My tip: pushd without an arg pushes the current directory onto the stack, and
then switches to what popd would have. Hence, repeated invocations of pushd
let you cycle between two directories without thinking.

$CDPATH is also amusing:

    
    
      $ mkdir /foo/bar/baz
      $ cd baz
      bash: cd: baz: no such file or directory
      $ export CDPATH="/foo/bar"
      $ cd baz
      $ pwd
      /foo/bar/baz

~~~
etherealG
could you give an example of the pushd thing you mentioned?

~~~
jrockway

        $ pwd
        /home/jrockway
        $ pushd foo
        $ pwd
        /home/jrockway/foo
        $ popd
        $ pwd
        /home/jrockway
        $ popd
        error: directory stack empty
        $ pushd
        error: directory stack empty
        $ pushd foo
        $ pwd
        /home/jrockway/foo
        $ pushd
        /home/jrockway
        $ pushd 
        /home/jrockway/foo
        $ pushd
        /home/jrockway
        ...

------
gcv
1\. Check out z: <http://github.com/rupa/z>. It's a pretty nice productivity
booster for flipping between different directories. It's quite easy to port to
zsh, also.

2\. Learn to use the editor modes on your shell. If you use Emacs mode in zsh,
for example, things like transient-mark mode work with all the standard Emacs
keybindings, and you can easily copy and paste pieces of your command line. vi
mode also supports similar bells and whistles.

3\. In zsh, if you are typing a command and then realize you need to look up
its man page, move your cursor back to the command and hit Esc-h (or Meta-h or
Alt-h, depending on your terminal). Once you quit the man viewer, your command
line will be restored.

4\. In zsh, if your command line gets too unwieldy to edit in zle, just hit
Esc-e (Meta-e). It'll let you edit the command line in your editor (be sure
your EDITOR environment variable is set).

5\. zsh has an awesome buffer stack feature. Imagine you're typing a long
command (call it command A), then realize that you need to run something else
first (call it command B). Don't Ctrl-u or delete anything; leave your current
command alone, and hit Esc-q. You'll get a fresh prompt. Run B. Once B
executes, zsh will restore the command line from A. You can do this as many
times as you want.

Anyway --- go and read "From Bash to Z Shell, Conquering the Command Line" by
Kiddle, Peek, and Stephenson. I've never seen a better book on using shells
with maximum efficiency (and I'm still only learning). Just learning to use
and extend the autocompletion features of zsh was invaluable to me.

------
aarongough
One of the little things that it took me a while to find out about:

    
    
      echo '2^16' | bc
      65536
    

'bc' is a complete calculator language interpreter that also functions quite
nicely as a simple calculator. If you need to do any heavy lifting check out:
<http://en.wikipedia.org/wiki/Bc_programming_language>

~~~
aarongough
As a follow-up to that: It might seem basic but understanding pipes (the '|'
symbol) is very useful! The pipe symbol 'sends' the output of one command to
another. Examples:

    
    
      tail -n 500 log/production.log | less (display the last 500 lines of your production logs one page at a time)
    
      cat config/routes.rb | wc (count the number of words and lines in a config file)

------
arghnoname
A lot of the tricks here are using the shell builtins. Pretty much all the
nifty stuff you can do with that are in your shell man page. If you are using
bash, try typing man bash. Read it. There is also stuff in there about
scripting bash. This is worth knowing, because short one liners are very nice
for moving a bunch of files around or whatever else. I use one liners every
day.

Make sure you read about job control in your shell. It helps understanding the
why's and whats of some of the other tricks.

Learn to use pipes. On the cli, I use Bash, sed, awk, and grep a lot, and with
pipes you can do amazing things relative to the windows world. It's a very
useful skill.

Learn a scripting language. I tend to use perl because I can pipe into it or
break into a move involved script, whichever I prefer. I personally find
Python less suited for this, but I've heard that Ruby is a good sysadmin
language too. You don't have to learn how to be a grand wizard, but any
serious unix person should know one. It helps greatly with productivity.

Edit:

Forgot to add, learn find (man find). This, in combo with xargs, is very, very
useful.

------
sant0sk1
I wrote a bash function yesterday which allows you to

    
    
        cd ...
    

to move up two directories instead of having to

    
    
        cd ../..
    

Want to move up 4 directories?

    
    
        cd .....
    

Pretty useful, imo. Check it out

<http://blog.jerodsanto.net/2009/09/cd-up-up-up/>

~~~
gcv
In zsh, you (1) don't need to write a function, and (2) you don't need to type
"cd" to switch directories.

    
    
      setopt auto_cd
    
      alias -g ...='../..'
      alias -g ....='../../..'
      alias -g .....='../../../..'
      alias -g ......='../../../../..'
    

Then you can just type "..." and go up two directory levels, and so on.

~~~
sant0sk1
That's pretty cool. I've shied away from zsh for some time as I'm very
comfortable with bash, but a lot of evidence has been coming to me lately that
makes me want to give it a go.

------
apu
readline is amazing, for bringing consistent shell-like behavior in a variety
of programs, including the shell, python, mysql, etc.

You configure it using ~/.inputrc...here are the commands I use:

    
    
      set completion-ignore-case on
    

Tab-completion will ignore case if it's not ambiguous...

    
    
      set show-all-if-ambiguous on
    

...but if it is ambiguous, this will give you a list if you hit tab again.

    
    
      Space: magic-space
    

Magic space is key. When you press space, it will expand out any ! commands
you have on the line. If you have none, it's just a normal space. This lets
you be much more bold with your use of history commands, as you can see what
it expands out to before you run it. Note that this can interfere a little
within other environments, such as python.

    
    
      "\e[A": history-search-backward
      "\e[B": history-search-forward
    

These two set incremental history search. You can type part of the command you
know is in your history, and then pressing up or down will cycle through them.
E.g., if you're looking for that particular find command you did a few days
ago, you can type 'find' and then hit up arrow until you find it.

    
    
      set editing-mode vi
    

Finally, not for everyone, but for us vi users, this is awesome. You get a lot
of the standard vi functionality on the command line (and anywhere else you
use readline, include mysql and python). This means esc + j/k to scroll up or
down through history (far faster than arrow keys), using f/F/t/T commands,
d/c/a/i ways to enter edit mode, even . and , work as expected, etc.

~~~
alavrik

      set editing-mode vi

BTW, for those who want to use it occasionally in bash, vi editing mode can be
enabled interactively by running

    
    
      set -o vi

And this command will revert it back to default ("emacs") editing mode:

    
    
      set -o emacs

------
dryicerx
Using the .ssh configuration file to save host specific options
(~/.ssh/config). Never again long ssh commands to connect to your frequently
connecting list.

    
    
        Host home home.example.com
          HostName home.example.com
          Port 12345
          User dryicerx
          IdentityFile ~/.ssh/somekey.pem
          LocalForward 59001 localhost:5901 
          
        Host *amazonaws*
          User root
          IdentityFile ~/.ssh/yourawskey
    
    

now on your command line, and the settings will automatically be applied.

    
    
        ssh home  
        ssh 1-2-3-4.blah.amazonaws.something.com
         
    

I specially love the last one if you are working with EC2, as it catches any
EC2 domain name and applies the key with the root user.

------
lallysingh
Named bookmarks and a jump-to function:

    
    
      function bookmark () {
        echo -n "Name of bookmark (no spaces): "
        read BKNAME
        if [ -e ~/.bkmarks/$BKNAME ]
        then
          echo -n "Overwrite? "
          read CONFIRM
          if [ "x$CONFIRM" != "xyes" ] && [ "x$CONFIRM" != "xy" ]
          then
              return 0;
          fi
        fi
        echo -n "Description: "
        read DESC
        ln -s `pwd` ~/.bkmarks/$BKNAME
        echo "`pwd` : $BKNAME : $DESC : `date`" >> ~/.bookmarks
      }
    
      function go () {
        if [ -x ~/.bkmarks/$1 ]
          then pushd ~/.bkmarks/$1; cd `pwd -P`; 
          else echo "~/.bkmarks/$1 not found";
        fi
      }

------
vijaydev
To copy/move/rename files with suffixes quickly:

    
    
      cp /home/foo/abc.cpp{,-old}
    

equivalent to

    
    
      cp /home/foo/abc.cpp /home/foo/abc.cpp-old

~~~
apu
Braces in general are great for all sorts of tasks. The general principle is
that it duplicates arguments, with changes. Thus,

    
    
      foo/bar/{a,b,c}
    

expands out to

    
    
      foo/bar/a foo/bar/b foo/bar/c
    

and you can also use {1..9} to expand to all the numbers between 1 and 9
(inclusive).

Here are some ways I commonly use it:

\- Create lots of subdirectories at once:

    
    
      mkdir -p test/{a,b,c}/{1..4}
    

\- Rename/copy files (as in parent comment)

\- Removing lots of randomly named files, with common elements:

    
    
      rm {a,d,f,gr}*
    

(this will remove all files starting with those prefixes...note that it will
not delete other files starting with 'g', only those with 'gr')

------
bohr99
ctrl + r

Starts a search through your history file.

~~~
uggedal
Map <up> and <down> to do the same in .zsh:

    
    
      # search history on <up> key instead of paging previous commands:
      bindkey "\e[A" history-search-backward
      bindkey "\e[B" history-search-forward

~~~
apu
Or in bash, using .inputrc (readline):

    
    
      "\e[A": history-search-backward
      "\e[B": history-search-forward

------
decklin
C-x C-e to open up the current command in $EDITOR. For previous commands, fc N
M where N and M are numbers or search strings (opens the range of commands in
your editor).

------
tfh

      set -o vi # vi like behaviour of the command line.
    

Use "/" to search through the history, "y" to copy, "p" to paste etc..

    
    
      cat > somefile.txt <<EOF # quickly creates a text file
    

Type EOF in a new line to finish.

~~~
jrockway
Or just omit the <<EOF and press Control-D to finish (assuming you have not
done anything strange with stty).

~~~
rimantas
touch somefile.txt

~~~
jrockway
His "quickly create a file" also adds text to it.

------
fogus
I use this to synch a directory with another:

    
    
        tar cf - * | (cd <dest-dir> ; tar xf - )

~~~
makecheck
Also look at "rsync".

~~~
fogus
And I do use rsync when I want to do a differential synch.

However, the tar solution works better for large file sets, and unlike using a
simple `cp -rfp`, hardlink inodes are preserved.

------
JoelMcCracken
screen is infinitely useful.

~~~
mcxx
Yes, it is. I recommend remapping the standard C-a to C-z, as C-a acts as
"jump to the beginning of line" in couple of shells, Emacs and Emacs clones.

~~~
spudlyo
I'm not sure that C-z is a great choice as it's often used for job control. I
use C-o myself. I suppose you could always use C-z C-z if you wanted to
suspend your current foreground process.

~~~
mcxx
I settled with C-z as it is easy and fast to type with one hand. However I am
occasionally too fast and hit C-z z instead of C-z C-z. 'fg' solves this.

------
philh
nc lets you talk directly to a remote host, and strace allows you to eavesdrop
on an arbitrary process' communication with a remote host (or with anything,
really).

Actually, I rather hope there's something better than strace for the latter.
But it works.

~~~
kree10
Also consider socat ( <http://www.dest-unreach.org/socat/> ) aka "netcat++".

------
papaf
This is more a general hint than a single tip. Years ago I tried to improve my
Unix skills by doing everything at home from the command line -- burning CDs,
playing music, copying trees etc. I learnt a lot but this exercise was much
less convenient and more mentally demanding than using a gui. Recently I have
become much more lazy.

~~~
jacquesm
Funny, we're pretty much opposite.

I see my 'window manager' as a neccesary evil to run browser and email
clients, everything else runs in terminal windows.

Of course you could use 'pine' and 'lynx' but that gets old pretty quickly.

~~~
kree10
s/pine/mutt/ && s/lynx/w3m/ : these get old much less quickly, though YMMV.

------
rp2

        xargs -d '\n'
    

as in

    
    
        find * -type f | egrep -i '\.(gif|jpg|png)$' | xargs -d '\n' identify
    

I used to use find -print0 | xargs -0 but sometimes you want to insert filters
in between or the file names aren't from find.

------
jimmybot
It's not a bash tip, but if you want something that generally makes command
line navigation easier without the need for heavy amounts of customization or
memorization, you might consider using the Friendly interactive shell (Fish
shell).

Smart history-based autocompletion, advanced tab completion, optional cd,
simplified syntax, among other features.

Not as hackerish as some of the other tips mentioned here, but it's a much
lower barrier to entry than the other commonly used shells out there.

Also submitted it here: <http://news.ycombinator.com/item?id=811113>

------
apu
Add this to your .bashrc to have much more intelligent tab-completion:

    
    
      case $- in
        *i*) [[ -f /etc/bash_completion ]] && . /etc/bash_completion ;;
      esac
    

In addition to the usual filename completions, this will do command-dependent
completions. A small sampling:

ssh: tabs complete hostnames in your ~/.ssh/config file

/etc/init.d/*: tabs complete possible options for any script in your init.d
directory (at least on debian/ubuntu)

most standard linux commands: type the command and then - and then tabs will
show you the list of possible options for that command.

kill/renice/pkill: tabs show you list of process ids/names

------
uggedal
My most useful zsh functions:

    
    
      # a visual recursive list of all files and directories
      function tree()
      {
        find . | sed -e 's/[^\/]*\//|--/g' -e 's/-- |/    |/g' | $PAGER
      }
    
      # recursive text search (ack is also nifty)
      function f()
      {
        find . -name '*' | xargs grep -l $1
      }
    
      # recursive search and replace (ignoring hidden files and dirs), example: replace foo bar
      function replace()
      {
        find . \( ! -regex '.*/\..*' \) -type f | xargs perl -pi -e "s/$1/$2/g"
      }

~~~
nibrahim
Perhaps Python specific but I use this a lot

    
    
      epy () { 
      	cmd="import $1 as a ; print   a.__file__.endswith('.pyc') and a.__file__[:-1] or a.__file__" 
      	file=$(/usr/bin/env python -c $cmd) 
      	echo $file
      	emacsclient --no-wait $file
      }

------
nudded

      !!:s/sl/ls
    

executes last command but replaces first occurence of sl with ls. (can also be
written as ^sl^ls)

    
    
      !!:gs/sl/ls
    

same as above, but now global replace

~~~
mtrimpe
I have that one aliased as sl2ls ...

~~~
apgwoz
Why wouldn't you just alias sl to ls? It'd be much less typing.

------
ianbishop
Pipes, grep and such come in handy in many applications.

For example:

ps -e | grep processname

will grab the pid of any process so that you can subsequently kill it.

sudo !! (run last command as sudo) is great too when you forget to type sudo
on a regular basis like me.

~~~
jncraton
_will grab the pid of any process so that you can subsequently kill it_

I like to use:

killall processname

~~~
spudlyo
And that works just dandy, until that one day you're on a Solaris machine...

------
pbhjpbhj
altgr-sysreq-r, then -e -i -s -u -b (spells busier backwards). This will do a
safe shut down and reboot including sync, even sometimes when ctrl-alt-del
won't work. Also you can use altgr-sysrq-k like ctrl-alt-bksp to kill the
window manager. Try altgr-sysrq-h from a console to see a terse help message.

Note altgr is the right hand alt key, the left one should work too but I find
the combos harder to do then, and sysrq is "system request" the same key as
"print screen".

IIRC Ubuntu now defaults to disabling the "magic sysrq" key combinations so
YMMV.

------
jdoliner
grep -l textregex fileregex

print the names of all the files satisfying fileregex with text satisfying
textregex. Nice if you need to find out where a function is defined.

------
jefffoster
GNU Global integrates well with the shell and allows you to do such neat stuff
as "less -t func" to see the definition of func.

------
compling
Process Substitution in bash

comm <(sort file1) <(sort file2) (comm file1 and file2 without temp files)

------
spahl
I love the '-exec' flag for find

~~~
makecheck
...except, that you should rarely use it. :)

It is much, much faster to pipe commands to "xargs". For instance:

    
    
        find . -name core -print0 | xargs -0 rm -f
    

Since "find" doesn't know when your command is multi-file, it has to run the
command once per file; whereas, "xargs" can run it as few times as necessary.
It works well for any command that accepts "a list of files".

In the example above, the -print0/-0 are a safety net to null-delimit
filenames so that it doesn't matter if any of the paths contains a space.

~~~
gaius
Also the -n and -P options to xargs. So you would do something like

    
    
        $ find . -name 'blah*' |wc -l
        2000
        $ find . -name 'blah*' |xargs -n 250 -P 8 ./mangle
    

That'll spawn 8 copies of mangle, each with 250 blah files on its command
line.

------
mattiss
Not really a trick, but something not everyone knows

appending & to run as a background process

./some_app &

------
mfukar
man man

------
skwiddor

        import kremvax /net /net
    
    

Plan9 doesn't need nat

------
juvenn
keep hacking on what not work, and diving into detail. I began my Linux life
exactly one year ago, when I bought myself the 1st laptop. It's rewarding and
joyfull

------
ZeroGravitas

        > make me a sandwich
        what? make it yourself
        > sudo !!
        okay

