
Interesting commands for the Linux shell - victorlf
https://www.lopezferrando.com/30-interesting-shell-commands/
======
sethrin
The ones I get a lot of use out of are:

curly brace substitution:

    
    
      $ mkdir -p new_project/{img,js,css}
      mkdir -p new_project/img new_project/js new_project/css
      $ mv some_file.txt{,.old}
      mv some_file.txt some_file.txt.old
    

Caret substitution:

    
    
      # systemctl status mysql.service
      -- snip output --
      # ^status^restart
      systemctl restart mysql.service
    

Global substitution (and history shortcut):

    
    
      $ echo "We're all mad here. I'm mad. You're mad."
      we're all mad here. I'm mad. You're mad.
      $ !!:gs/mad/HN/
      we're all HN here. I'm HN. You're HN.
    

I have a (WIP) ebook with more such tricks on GitHub if anyone is interested:
[https://tenebrousedge.github.io/shell_guide/](https://tenebrousedge.github.io/shell_guide/)

~~~
godelski
Curly brace expansion is one of the most useful and overlooked topic in these
kind of posts. And I'm always surprised it isn't mentioned with compiling.

g++ -o foo{,.cpp}

~~~
tych0
Worth noting that for the particular case you've cited, you can just use make,
even without a makefile:

    
    
      ~ cat test.cpp 
      #include <iostream>
      
      using namespace std;
      
      int main()
      {
          cout << "hi\n";
          return 0;
      }
      ~ stat Makefile
      stat: cannot stat 'Makefile': No such file or directory
      ~ 1 make test
      g++     test.cpp   -o test
      ~ ./test
      hi

~~~
godelski
Well you learn something every day. That's pretty awesome! Is there an easy
way to add options? Like -O2

------
disconnected
Disappointingly, this list treats "yes" like a toy that just prints things
over and over, and doesn't mention actual useful uses for "yes", like
accepting all defaults without having to press enter.

Practical example: when you are doing "make oldconfig" on the kernel, and you
don't care about all those questions:

yes "" | make oldconfig

Or, if you prefer answering no instead:

yes "n" | yourcommand

Also, the author refers to watch as a "supervisor" ("supervise command" \- his
words). That is bad terminology. Process supervision has well defined meaning
in this context, and watch isn't even close to doing it.

Examples of actual supervisors are supervisord, runit, monit, s6, and of
course systemd (which also does service management and, er, too much other
stuff, honestly).

~~~
skocznymroczny
Unfortunately it doesn't work when trying to connect via ssh and warning about
unknown key pops up.

~~~
hnlmorg
That is intentional behavior on the part of `ssh`. You can still disable that
prompt within `ssh` (though it's not recommended):

    
    
        ssh -oStrictHostKeyChecking=no 192.168.0.100
    

...or by adding rules in /etc/ssh/ssh_config or ~/.ssh/config

    
    
        Host 192.168.0.*
            StrictHostKeyChecking no

------
hnlmorg
Some handy tips there but I would recommend changing some of the `find`
examples:

    
    
        find . -exec echo {} \;      # One file by line
    

You don't need to execute echo to do that as find will output by default
anyway. There is also a `-print` flag if you wish to force `find` to output.

    
    
        find . -exec echo {} \+      # All in the same line
    

This is think is a dangerous example because any files with spaces will look
like two files instead of one delimited by space.

Lastly, in both the above examples you're returning files _and directories_
rather than just files. If you wanted to exclude directories then use the
`-type f` flag to specify files:

    
    
        find . -type f ...
    

(equally you could specify only directories with `-type d`)

Other `find` tips I've found useful that might be worth adding:

* You don't need to specify the source directory in GNU find (you do on FreeBSD et al) so if you're lazy then `find` will default to your working directory:
    
    
        find -type f -name "*.txt"
    

* You can do case insensitive named matches with `-iname` (this is also GNU specific):
    
    
        find -type f -iname "invoice*"

~~~
pixdamix
The only forbidden characters in a Unix filenames are '/' and '\0'

Want to mess with such a script?

$ touch "$(echo -e -n 'lol\nposix')"

~~~
majewsky
One of the cruelest things you can do is a filename that consists only of a
combining diacritic (without a glyph that it could combine with). Will break
outputs of various programs (starting with ls) in sometimes hilarious ways.

If you're trying it out now and cannot figure out how to delete it: "ls -li"
to find the file's inode number, then `find -inum $INODE_NUMBER -delete`.

~~~
anowlcalledjosh
Wow, that's really horrible. I have a file sitting around with a couple of
newlines in the name just so I can see how many programs don't cope with it,
but I hadn't thought of using a lone combining diacritic.

If anyone wants a command to make one, try

    
    
        touch $'\U035F'
    

(using U+035F COMBINING DOUBLE MACRON BELOW for no particular reason, see [1]
for more)

[1]:
[https://en.wikipedia.org/wiki/Combining_Diacritical_Marks](https://en.wikipedia.org/wiki/Combining_Diacritical_Marks)

------
joosters

       grep -P "\t"
    

Not criticising the author here, as grep -P is good, but you might not also
know that you can enter tabs and other codes in bash by pressing ctrl-v. So
you could also type:

    
    
       grep "[ctrl-v]TAB"

~~~
l0b0
The first alternative is still preferable, since it's actually readable.

You can also do $'\t' in at least Bash (and probably Zsh).

~~~
joosters
Oh, agreed. But the ctrl-v trick is useful in general anywhere where you'd
like to put a special character in an input or command.

------
stiangrindvoll
Doesn't matter how long you've been using a GNU/Linux shell, you'll always
learn something new every day. Thanks for this.

~~~
hnlmorg
Funny enough I was saying this to a work colleague yesterday when I was
looking at a shell script and scratching me head wondering how it was working
when a variable named $RANDOM wasn't assigned a value. After a little
investigation it turned out Bash (and possibly other $SHELL's?) returns a
random number with each call of the $RANDOM variable. Handy little trick.

~~~
vacri
There is also the inbuilt variable $SECONDS, which seconds since the instance
of bash was started. Makes it really easy to do 'time elapsed' at the end of a
bash script.

    
    
        $ echo $SECONDS
        83783
        $ echo $SECONDS
        83784
        $ echo $SECONDS
        83787

~~~
monsieurbanana
That's nifty.

    
    
        $ echo "$SECONDS / 60 / 60" | bc
        362.38500000000000000000

~~~
vram22
echo "scale=2; $SECONDS / 60 / 60" | bc

# will give 2 digits after the decimal point, instead of bc -l which gives
more digits of precision after the decimal point, than we may usually want for
this particular calculation.

~~~
lozf
Recently I needed results that included nothing after the decimal point, so
resorted to awk to do the math:

    
    
       awk "BEGIN { print int($SECONDS /60 /60)}"

~~~
vram22
Interesting. I suppose you had to type Ctrl-D or Ctrl-Z to signal no input to
awk, since it expects some in the above command? Never tried running awk
without either a filename as input or stdin coming from a file or a pipe. Will
check it and see what happens.

Modifying my example from above:

echo "scale=0; $SECONDS / 60 / 60" | bc

might also work, need to check.

------
Tepix
> 6\. List contents of tar.gz and extract only one file

> tar -ztvf file.tgz

> tar -zxvf file.tgz filename

Tar no longer requires the modifier for the compression when extracting an
archive. So no matter if it's a .tar.gz, .tar.Z, .tar.bz2, etc you can just
use "tar xvf"

~~~
kaushalmodi
And the same for compression..

    
    
        tar caf foo.tar.xz foo/
    

The extension of the file after the `f` switch tells tar what compression to
use.

c - compress

a - auto

------
abhirag
As long as we are on the topic of Linux shell commands I would like to share a
tip which has helped me, whenever you are using wildcards with rm you can
first test them with ls and see the files that will be deleted and then
replace the ls with rm. This along with -i and -I flags makes rm just a tad
bit less scary for me. Kinda basic but hopefully somebody finds it helpful :)

~~~
vram22
As mbrock says, you can also use echo instead of ls for that. And "echo *"
serves, in a pinch, as a rudimentary ls, when the ls command has been deleted
and you are trying to do some recovery of a Unix system. Similarly dd can be
used to cat a file if cat is deleted: dd < file

And in fact whenever you want to do some command like:

cmd some_flags_and_args_and_metacharacters ...

you can just replace cmd with echo first to see what that full command will
expand to, before you actually run it, so you know you will get the result you
want (or not). This works because the $foo and its many variants and other
metacharacters are all expanded by the shell, not by the individual commands.
However watch out for > file and >> file which will overwrite or append to the
given file with the output of the echo command.

~~~
dotancohen
Knowing how to cope with a deleted `ls` and `cat` sounds like a horrible
battle scar to carry. How on Earth did you ever get into that situation?

~~~
vram22
Just saw your comment now ...

Was not really an issue for me personally, since it was not on my machine.
Used to encounter such situations with some regularity when I was a Unix (and
general) system engineer in the field, for a large hardware + Unix vendor,
early in my career. The job involved solving all sorts of software problems
(sometimes involving many levels of the stack) for customers of my employer.
Sometimes, less technically savvy customers, like data entry operators or end
users of the Unix systems sold by that vendor to them, would end up doing
things like that - deleting critical files, losing backups or not taking
backups and then the hard disk or OS crashed, corrupting the data on disk,
etc. Got to handle many and diverse interesting problems in this area, and
learned a lot from it; some of those skills have served me in good stead later
in my career too - troubleshooting, Unix fundamentals and skills, interaction
of apps with the OS, etc.

As sethrin says, you don't have to acquire the scars personally. And thanks,
sethrin, for those links - should make for some good reading, I'm interested
in this area though I do not work on it from some time. It can be good fun and
mental exercises in problem solving.

------
aembleton
MMV is the latest interesting one I've found recently:
[https://ss64.com/bash/mmv.html](https://ss64.com/bash/mmv.html)

For example, I can do the following:

    
    
       mmv "flight.*" "flight-new.#1"
    

and this will rename all of my files that start with flight. to flight-new.
preserving the file extension. So useful, when you've got a bunch of different
files with the same name but with different extensions such as html, txt and
sms.

~~~
pmoriarty
Check out qmv from renameutils.[1] Using it you can use your $EDITOR to batch-
rename files. It does sanity checking like making sure you don't rename two
files to the same name and lets you fix such mistakes before trying again.

[1] - [http://www.nongnu.org/renameutils/](http://www.nongnu.org/renameutils/)

------
leephillips
I probably should be embarrassed to admit I didn't know about much of the
stuff on this page about parameter expansion:

[http://wiki.bash-hackers.org/syntax/pe](http://wiki.bash-
hackers.org/syntax/pe)

Bash is insane.

~~~
chubot
It is a horrible language design, which was impressed upon me after
implementing it for my shell Oil. Here are some pathological cases I pointed
out:

${####}:
[http://www.oilshell.org/blog/2016/10/28.html](http://www.oilshell.org/blog/2016/10/28.html)

Each # means a different thing.

${foo//z//}:
[http://www.oilshell.org/blog/2016/10/29.html](http://www.oilshell.org/blog/2016/10/29.html)

There are three different meanings of / there.

"${a[@]}" \--
[http://www.oilshell.org/blog/2016/11/06.html](http://www.oilshell.org/blog/2016/11/06.html)

You need 8 punctuation chars in addition to the array name to correctly
interpolate it. Every other way gives you word splitting issues.

------
michaelcampbell
I guess I'm just overly sensitive (and maybe English is not the poster's first
language), but I cringe at "the Linux shell"... I have used ksh, bash, (a
little) csh, and zsh over the years, and love the architecture that makes "the
shell" just another non-special binary that's not unique to the OS.

And yes, a lot of the things mentioned work in a lot of shells, but some
don't, or act differently.

~~~
testestx
I would find it a little if it were called the GNU or the GNU/Linux default
shell

------
Exuma
Here are some that I use in Pet:

Description: Remove executable recursively for all files

    
    
        Command: chmod -x $(find . -type f)
    

Description: List files with permissions number

    
    
        Command: ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) *2^(8-i));if(k)printf("%0o ",k);print}'
    

Description: Output primary terminal colors

    
    
        Command: for i in {0..16}; do echo -e "\e[38;05;${i}m\\\e[38;05;${i}m"; done | column -c 80 -s '  '; echo -e "\e[m"
    

Description: NPM list with top-level only

    
    
        Command: npm list --depth=0 2>/dev/null
    

Description: Show active connections

    
    
        Command: netstat -tn 2>/dev/null | grep :80 | awk '{print $5}' | sed -e 's/::ffff://' |cut -d: -f1 | sort | uniq -c | sort -rn | head

~~~
xelxebar
While we're here sharing random cli-fu, you might like `find`'s native ability
to spawn processes. Check out the `-exec` flag:

    
    
        find . -type f -exec chmod -x {} \+

~~~
schoen
This is more correct than the previous poster's approach, because it will work
with files that contain space characters, while the previous poster's version
breaks in this situation.

Another approach that works is

    
    
      find . type f -print0 | xargs -0 chmod -x
    

although find's built-in -exec can be easier to use than xargs for
constructing some kinds of command lines.

~~~
majewsky
You probably want a -r with that xargs. -r makes it exit without executing the
command line if stdin is empty.

~~~
schoen
I totally forgot about the case where find gives no output.

------
a3_nm
Nice! Point 23 is a bit misleading though: comm only works on _sorted_ input
files. Also, "disown -h" in Point 21 works on bash but not on zsh. Also, in
Point 22, "timeout" only works if the command does not trap SIGTERM (or you
have to use -k).

------
ptero
> If a program eats too much memory, the swap can get filled with the rest of
> the memory and when you go back to normal, everything is slow. Just restart
> the swap partition to fix it: sudo swapoff -a; sudo swapon -a

Is this true? Not impossible, but I am surprised. If true, what is this
fixing? In my naive view (never studied swap management), if at the current
time a page is swapped out (and by now we have more memory -- we can kill swap
completely and do fine), it should get swapped in when needed next time. As
there is more memory now it should not, in general, be swapped out again.

If true we are exchanging a number of short delays later for a longer delay
now, which to me hardly looks like a win.

~~~
0xfeba
I haven't used a swap in the past 10 years and have not noticed any problems.
Is it relevant anymore?

~~~
executesorder66
Yes. You need swap to hibernate a laptop.

~~~
majewsky
But is hibernate relevant? I've found suspend to be much more reliable, and
good enough in terms of energy consumption (thanks to modern low-power states
in CPUs).

~~~
executesorder66
I find it relevant. I use it to store the state my work laptop is in at the
end of the day, and then restore that state when I get back to work the next
work day. That way I don't need to keep it switched on when I'm not using
it/transporting it.

Is there a better solution to this other than hibernating?

------
xearl
nice collection! upvoted for "15\. `namei -l`" alone, which is far too little
known.

a better "20\. Randomize lines in file":

    
    
      shuf file.txt
    

instead of

    
    
      cat file.txt | sort -R
    

(sort -R sorts by hash, which is not really randomisation.)

~~~
trapperkeeper74
And prefer redirection to cat.

    
    
        whatever < file.txt 
    

not

    
    
        cat file.txt | whatever
    
    

Also, there is no need for - or z on GNU tar in t or x modes:

    
    
        tar tf whatever.tgz
        tar tf whatever.tar.bz2
        tar tf whatever.txz
        ...
    
        tar xf whatever.tar.gz
        tar xf whatever.tar.Z
        ...
    

all work just fine

~~~
TorKlingberg
I know whatever < file.txt is slightly more efficient, but there is value is
keeping things going left to right with pipes in between. It makes it easy to
insert a grep or sort, or swap out a part.

~~~
veddan

        <file.txt grep foo | sort

~~~
BenjiWiebe
Ok, if that actually works, that's amazing. Wow.

~~~
vram22
It works because of the general principle that I mentioned here:

[https://news.ycombinator.com/item?id=15249370](https://news.ycombinator.com/item?id=15249370)

The shell (not the command) is the one expanding those metacharacters, so
(within limits), in:

cmd < file

or

< file cmd

where you put that piece (< file) on the line does not matter, because the
actual command cmd never sees the redirection at all - it is done [1] by the
time the command starts. The cmd command will only see the command line
arguments (other than redirections) and options (arguments that start with a
dash). Even expansion of $ and other sigils (unless quoted) happens before the
command starts.

[1] I.e. the shell and kernel set up the standard input of the command to be
connected to the file opened for reading (in the case of < file).

------
kleff
Never knew about _readlink_ for files, but I use

    
    
      pwd -P 
    

all the time to get the real full path (no symlinks) of the current directory.
Really easy to remember as well, _P_ rint _W_ orking _D_ irectory.

~~~
kazgul
That is not the same use case.

You could use this to get the current location of the script being run (not
the location where it runs).

    
    
       readlink -f $0

------
yodsanklai
I really like the "ag" command. Very convenient to grep in a bunch of files
filtered by type. Example:

    
    
        ag --ocaml to_string
    

Very fast and simple syntax.

~~~
kaushalmodi
Now try rg (ripgrep). It has a much more robust file ignore handling than ag
in my experience.

When I last tried, ag couldn't handle this in .ignore file:

    
    
        !foo.1
        foo.*
    

That would ignore all foo.* files except foo.1 in rg.

Also rg is a bit faster than ag for my use cases.

~~~
VeejayRampay
Is it possible to register new file formats in ripgrep though (say for example
slim files so that I could do rg -tslim PATTERN)? Last time I checked it was
somehow possible but way too complicated for so common a task.

~~~
kaushalmodi
From its README:

    
    
        rg --type-add 'foo:*.{foo,foobar}' -tfoo bar
    

\---

You simply make the below into an alias:

    
    
        rg --type-add 'foo:*.{foo,foobar}'
    

Or, if the type you need is generic that other users can also use, submit a
PR. The developer was kind to except mine..
[https://github.com/BurntSushi/ripgrep/pull/107/files](https://github.com/BurntSushi/ripgrep/pull/107/files)
.. it's a trivial 1-line PR.

------
hathym
cached version:
[https://webcache.googleusercontent.com/search?q=cache:JQ7qBa...](https://webcache.googleusercontent.com/search?q=cache:JQ7qBa09bekJ:https://www.lopezferrando.com/30-interesting-
shell-commands/+&cd=1&hl=en&ct=clnk&gl=fr)

------
yellowapple
You'll definitely want to check `free -m` before calling `swapoff` to make
sure you really have enough memory for everything in swap, lest you want to
invoke the wrath of the OOM killer.

------
Pete_D
Not a command as such, but I recommend skimming the readline man page[0] and
trying some of the bindings out to see if you're missing out on anything. I
went _years_ without knowing about ctrl+R (search command history).

[0] [http://man7.org/linux/man-
pages/man3/readline.3.html#EDITING...](http://man7.org/linux/man-
pages/man3/readline.3.html#EDITING_COMMANDS)

------
vram22
Not really a single command, it's a one-liner - a pipeline, but might be
interesting, not only for the one-liner but for the comments about Unix
processes that ensued on my blog:

UNIX one-liner to kill a hanging Firefox process:

[https://jugad2.blogspot.in/2008/09/unix-one-liner-to-kill-
ha...](https://jugad2.blogspot.in/2008/09/unix-one-liner-to-kill-hanging-
firefox.html)

------
sillysaurus3
Throw this in your ~/bin as a script named math:

    
    
      #!/bin/sh
      scale=4 # results will print to the 4th decimal
      echo "scale=$scale; $@" | bc -l
    

Now you can do math.

    
    
      $ math '1+1'
      2
      $ math '2/3'
      .6666
    

This is especially useful in shell scripts with interpolated variables:

    
    
      x=10
      x=`math $x - 1`

~~~
xelxebar
Alternatively, you could use an alias:

    
    
        # .bashrc
        alias bc='bc --mathlib'
    

and a .bcrc file:

    
    
        # .bcrc
        scale = 4
    

Actually, this is what my .bcrc looks like:

    
    
        scale = 39
    
        k_c = 299792458                   # Speed of Light
        k_g = 6.67384 * 10^-11            # Gravitation
        k_atm = 100325                    # Atmospheric pressure
        k_h = 6.62606957 * 10^-34         # Planck's constant
        k_hbar = 1.054571726 * 10^-34     # H Bar
        k_mu = 1.256637061 * 10^-6        # Vacuum permeability
        k_ep = 8.854187817 * 10^-12       # Vacuum permittivity
        k_epsilon = 8.854187817 * 10^-12  # Vacuum permittivity
        k_e = 1.602176565 * 10^-19        # Elementary charge
        k_coulomb = 8.987551787 * 10^9    # Coulomb's constant
        k_me = 9.10938294 * 10^-31        # Rest mass of an electron
        k_mp = 1.672621777 * 10^-27       # Rest mass of a proton
        k_n = 6.02214129 * 10^23          # Avogadro's number
        k_b = 1.3806488 * 10^-23          # Boltzmann's constant
        k_r = 8.3144621                   # Ideal gas constant
        k_si = 5.670373 * 10^-8           # Stefan-Boltzmann constant
        k_sigma = 5.670373 * 10^-8        # Stefan-Boltzmann constant
        k_mt = 5.97219^24                 # Mass of Earth (Tierra)
        k_rt = 6.371 * 10^6               # Mean radius of Earth (Tierra)
    
        pi = 3.1415926535897932384626433832795028841968
    
        # requires --mathlib
        define t(x) { return s(x)/c(x); }
        define as(x) { return 2*a(x/(1+sqrt(1-x^2))); }
        define ac(x) { return 2*a(sqrt(1-x^2)/(1+x)); }
        define at(x) { return a(x); }
        define csc(x) { return 1/s(x); }
        define sec(x) { return 1/c(x); }
        define cot(x) { return c(x)/s(x); }

~~~
schoen
It's not nearly as flexible as bc overall, but GNU units has lots more
constants built-in and also trig functions.

I checked and it has all of the ones that you mentioned, sometimes under
slightly different names. I was surprised that e is defined the elementary
charge rather than Euler's constant!

~~~
majewsky
units(1) is my go-to calculator for everything. Really nice tool. I recommend

    
    
      alias units="units --verbose"
    

because the verbose output is much less ambiguous.

~~~
schoen
Nice, thanks for the suggestion.

------
ghotli
ctrl-r for fuzzy searching your history is all you really need to know.

------
husamia
Related Resource:
[http://www.commandlinefu.com/commands/browse](http://www.commandlinefu.com/commands/browse)

------
cutler
CentOS 7 specifies `man rename` : rename [options] expression replacement
file...

The rename example fails with "rename: not enough arguments".

~~~
jwilk
Unfortunately, there are two incompatible rename programs in the wild. :(

Debian ships this: [https://metacpan.org/pod/distribution/File-
Rename/rename.PL](https://metacpan.org/pod/distribution/File-Rename/rename.PL)

Most(?) other distros ship the one from util-linux:
[http://man7.org/linux/man-
pages/man1/rename.1.html](http://man7.org/linux/man-pages/man1/rename.1.html)

------
sirwitti
Thanks, every once in a while these lists are useful!

------
jheriko
.... or just use a modern os where all this stuff is intuitive and easy :P

