
The Art of Command Line (2015) - axiomdata316
https://github.com/jlevy/the-art-of-command-line
======
smith-kyle
> Learn basic Bash. Actually, type man bash and at least skim the whole thing;
> it's pretty easy to follow and not that long

Skimming _The Grapes of Wrath_ would be shorter. Thank you, but no thank you.

~~~
heinrichhartman
How large is it? Like 40p when you put it on a printer?

Let me help you with this:

    
    
        man -t bash > man.ps && lpr man.ps
    

Then, grab a coffee, sit in the sun and spent some quality time with your
tools. :)

~~~
acqq
The man pages are the worst form of information ever for anybody trying to
really solve any real-life problem which is not specified as "learn all
unnecessary switches and once-useful but used-by-nobody-today options of a
command X."

It happened to me often enough that even when I knew what I wanted to achieve
and which command X could do that I would first try to read darned man page
and fail to find the solution, but then just googling gave enough of material
to

a) have something to try

b) have a discussion of the cases where that "something" doesn't fulfill the
use most use cases of those who asked (the "natural need") including, of
course, the use case I've had.

I have a completely opposite attitude. Bash is horrible on many levels. Bash
man page is only a part of horribleness.

The main problem for all man pages is that they are mostly as if written for
those who already know about every unnecessary detail of both the tool in
question and all related tools or environmental issues, but who searches for
some even more obscure detail among all obscure details that they already
know.

Not to mention the descriptions that aren't: e.g. the complete explanation of
the option --frob-blonk FILE would be something like "this option uses the
FILE as the frob blonk" oh, how helpful.

I use bash every day, but I've at least managed to reduce its use to the
minimum that keeps me still sane.

~~~
mFixman
Skimming the bash manpage every once in a while is important to stay out of
information bubbles.

There are too many developers who rely only on Google/Stack Overflow to find
and write answers to questions, and it's common to repeat bad, incomplete, or
outdated information. I can think of a lot of times when the most accepted SO
answer for many questions was "copy and paste this gigantic block of code"
instead of using an existing command.

~~~
acqq
Most of information there is not worth having in my "brain cache" at all, and
especially not worth refreshing it regularly. Life is too short to even trying
to remember the things like what's the difference between .bash_profile
.profile .bashrc .inputrc and .bash_login (bonus point for remembering what is
redirected to what in the files themselves in which distribution) or what the
hell is a "login shell" and what is not. And there are many such details that
are effectively useless for 99.99% of the users but they just have to survive
the suffering to fix the darn thing when they have some issue.

I'm sure even most of the readers here can't give a one line answer what is
the "right" and "general" way to preset a darn variable for a shell. By design
of all these artifacts we work with, it necessarily leads to a page-length
discussion of which shells exist and what their differences are and whatnot.
Such a waste of everybody's mental energy on a global level.

~~~
0xADEADBEE
I think it's a bit unfair you're being downvoted because you raise some good
points. A lot of people agree POSIX is a mess and you've highlighted a
fraction of reasons related to that. Would any of the downvoters care to
elucidate? Maybe I'm missing something obvious.

~~~
__del__
Not a downvoter, but I assume the reason is that many people do understand
these mechanisms without evicting too much from their brain cache.

Files like .bash_profile, .bash_login, .profile, .zprofile, /etc/profile,
/etc/profile.d/whatever.sh all serve the same purpose, the primary aspect of
which is to apply things like environment variables at login. Display managers
and [login] shells source these things in a predictable precedence order so
that environment variables are correctly inherited by children of GUI
environment processes and login shells.

Login shells? Well yeah, the point of a login shell is that it invokes login-
related tasks like setting environment variables from a profile. That's
intentionally separate from the behavior of interactive shells because the
environments of interactive shells may have been modified intentionally and
shouldn't be "reset" to the values applied at login.

edit: also .inputrc is the readline configuration file for things like
shortcut keys in all programs which use readline (not just bash) and .bashrc
is for customizations specific to interactive shells

~~~
acqq
You see I’ve read this kind of information in exactly the words you use more
than once and I still can’t summarize in one sentence where should one put his
variables for things to “just work.” Locally for one user. Provided I just
don’t know and don’t even care what the differences between the login and
interactive shells are and that I just run the whatever terminal it is default
from the GUI menu (whichever GUI it is).

And your explanation what the differences are is for me still circular,
exactly like in the man pages: login shell is the one which invokes login-
related tasks, and I should care about the difference from the interactive
shells. I still have no idea what that even means, it is what exactly happens
when?

Anybody knows some sane explanation of these? I admit I never tried to find
it, as I tried to read the man pages, didn't find anything reasonable, but
knew that it doesn't matter to me. I just want the darn variables to exists
whenever I need them. I don't care about the differences. One single place is
enough.

But now that you say that you "do understand these mechanisms without evicting
too much from" your "brain cache" I'd really like to learn _everything_ about
the differences. Specifically, when is one invoked and when another for the
user "acqq", let's say, on a default Ubuntu system, and on a default Red Hat
system?

Are really more different user shells invoked in different forms for a single
user, on a single machine, between the booting of the machine, logging in (in
GUI) and then starting a few times the terminal with shell inside? When is
invoked and which is invoked when? And why shouldn't be a single place for the
variables I want to define? No idea, but I would be grateful to read about it
now. Thanks in advance.

------
molteanu
_Many people also use the classic Emacs, particularly for larger editing
tasks. (Of course, any modern software developer working on an extensive
project is unlikely to use only a pure text-based editor and should also be
familiar with modern graphical IDEs and tools.)_

Huh?! Unlikely?!

~~~
y4mi
yes, unlikely. most people use IDE's, very few limit themselves to simple
editors.

they're very rare but do exist.

~~~
sureaboutthis
So you think emacs is a simple editor, eh?

~~~
marcosdumay
I'm reading that "simple" word on the GP as something that doesn't impose a
workflow on you. (What's ironically best described by "versatile", but
somebody not used to the tool can not use most of the functions.)

By any other meaning, emacs and vim wouldn't qualify.

------
ImaCake
Slight tangent, but forcing biology undergrads to work their way through this
would probably help them a lot when they suddenly get saddled with some
sequencing output and have literally never heard of Bash let alone used it. My
lab of 40+ people has maybe 2 or 3 people who know how to use the command
line.

~~~
yitchelle
Genuine curious of the use case of command line by biology students. Can you
elaborate?

~~~
joycian
When you have DNA sequencing data, you often have to use bash / command line
to glue together the various tooling.

As an example:
[https://bedtools.readthedocs.io/en/latest/](https://bedtools.readthedocs.io/en/latest/)

~~~
cure
Yes, this is very common.

But there are also many people who use Common Workflow Language
([https://www.commonwl.org/](https://www.commonwl.org/)) for this purpose.

CWL allows for workflow portability between execution engines. The open source
engine my team creates (Arvados - [https://arvados.org](https://arvados.org))
is ideal for large-scale processing work.

We store, process and manage petabytes of genome data with it, and execute
workflows that span hundreds of simultaneous compute nodes.

~~~
joycian
This looks very cool! Thanks for sharing it. I'm definitely going to mention
this in the next technical meeting at the research group.

------
mychael
I feel grateful for learning these tools before repos like this existed.
Discovering new CLI tools back then was like learning about new bands by word
of mouth – it made it more special.

~~~
hazz99
I remember finding out about ctrl+r, I thought it was incredible. The shell
has so many helpful features and I'm continuously discovering more.

~~~
Insanity
I was so happy when I found out ctrl+r. I can't believe I went so many years
without using it, and now I use it constantly.

~~~
legends2k
Now couple that with infinite bash command history and you never have to type
out any command more than once.

Search for eternal bash history and set it up once. I can point out its
usefulness more.

------
follower
If pictures or a hand-drawn look is more your thing, Julia Evan's work is very
accessible. She sells zines like "Bite Size Command Line!"
[https://jvns.ca/blog/2018/08/05/new-zine--bite-size-
command-...](https://jvns.ca/blog/2018/08/05/new-zine--bite-size-command-
line/) but also posts a lot of the content on Twitter too: e.g.
[https://twitter.com/i/moments/1026078161115729920](https://twitter.com/i/moments/1026078161115729920)

(I actually made a proof-of-concept tool called 0rk which displays the images
in the terminal
[https://github.com/follower/0rk](https://github.com/follower/0rk) so you
don't even have to leave your terminal to look something up.)

There's also the TLDR pages project which aims to create "Simplified and
community-driven man pages" with practical examples which looks like a helpful
approach: [https://tldr.sh/](https://tldr.sh/)

Not sure if any of the TLDR clients already do this but it just occurred to me
it'd be cool to be able to view an example and then edit it directly...

------
sjy
If you're an experienced command line user, this is a great reminder of how
far you've come, and check list to figure out where your weak spots lie. I
wonder how long these skills will remain useful; skeptics often seem to
(wrongly) think that the CLI is 'obsolete.' I learned some of the things on
this list more than 10 years ago and I am sure for some people it has been
much longer than that. If anything, the increased prevalence of embedded Linux
devices has made these skills feel more useful over time.

------
g_sch
While this has some helpful pointers, I found it amusing that some points were
relatively little work, while others contain enough to keep you busy for weeks
- for example, comparing output redirection to network management.

------
merricksb
A big discussion 4 years ago:

[https://news.ycombinator.com/item?id=9720813](https://news.ycombinator.com/item?id=9720813)

(Provided for interest purposes, not suggesting it's a dupe -
[https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html))

------
aljarry
Some time ago I stumbled upon a really good introduction to text manipulation
in command line:

[https://developer.ibm.com/articles/au-
unixtext/](https://developer.ibm.com/articles/au-unixtext/)

After years of using bits of bash I've found quite a few new things for me.

~~~
JdeBP
Have you ever read _Unix Text Processing_?

------
augustk
My suggestion would be: Learn basic POSIX Shell (sh). It's smaller than Bash
and more portable.

[https://pubs.opengroup.org/onlinepubs/007904875/utilities/xc...](https://pubs.opengroup.org/onlinepubs/007904875/utilities/xcu_chap02.html)

~~~
klmr
It’s also really limited. I do a lot of shell programming and I lean heavily
on Bashisms to get any work done. Even in Bash I’m hitting the wall
constantly. Sh-only is utterly impractical: Reading code from big projects
that try to stay compatible (e.g. automake) really show-cases these
limitations. For instance, performing proper argument quoting of variables
without arrays is next to impossible.

~~~
yjftsjthsd-h
I intentionally use it as a sanity check; if the task exceeds POSIX sh, it's
time to pull out Python (for me; substitute your "real" language of choice).

~~~
klmr
This is a good heuristic for advanced logic, but not when the main purpose of
the script is to encode a workflow of coding other tools. I’m using other
languages for that too, but shell scripting languages are consistently
superior to all of them (including Perl and Ruby, IMHO; workflow languages
might help but pull in tons of dependencies).

------
mvindahl
Skimmed most of it but still learned some useful stuff. Will keep it in my
bookmarks.

Also, would suggest mentioning gitbash in the Windows section. You get it for
free with git so it's easy to come by, even in corporate environments where
obtaining permission to install software is hard. If combined with something
like ConEmu, it's actually pretty nice.

------
dhimes
Holy crap! I never knew

    
    
        curl cheat.sh/<command>
    

was a thing. All these years....

~~~
teddyh

      $ whois cheat.sh | grep 'Creation Date'
      Creation Date: 2017-04-16T13:42:06Z
    

Not that many years.

~~~
dhimes
Thank you. That helps (but then, that means I missed it...)

------
flavious
I've written many thousand lines of code in bash (along with way more in 15
years of programming in other languages) and I must tell you:

Never again!

Use xonch at least.

------
dvfjsdhgfv
> Obscure but useful

I find the choice of the word "obscure" a bit amusing, given that this section
lists many commands used ona daily basis by thousands of syadmins.

------
casper345
> Know > overwrites the output file and >> appends. Learn about stdout and
> stderr.

Yep, learned that the hard way

------
mattxxx
I was ready to knock it, but it's actually a decent reference.

If you're just picking it up, check it out.

------
mlang23
Gosh! How is it possible I missed C-x C-e for over 20 years?

------
kensai
Is it worth it to master the (newly reloaded) Windows Terminal?

~~~
Riverheart
If you work with Windows at all it's worthwhile to have some Powershell
scripting chops for system automation. And while it's kind of like Bash, it's
syntax is more common and everything is an object which is nice. No more
string parsing hoops to get basic info.

------
lprd
This looks great! Is there an equivalent for zsh users?

~~~
enriquto
> This looks great! Is there an equivalent for zsh users?

Most of it works just the same.

------
Zardoz84
It's missing nmon and bmon

~~~
Cthulhu_
Do I really have to point out that it's an open source project and you're free
to write it and open up a pull request instead of merely pointing out things
in a comment thread? At least make the effort and open up an issue in the
project itself - provide your motivation, link to resources, etc.

~~~
Zardoz84
I thought about doing it, but I can't do it from a phone on a train.

