> Most people today don’t know what the command line is, much less why they would want to bother with it.
This is true today, and it was true as well "in the 1980s", to use the same time frame as TFA. The difference is that today there are morepeoplethanever who know what the command line is, and who can use it. At least an order of magnitude more people; maybe two. We can certainly say that we live in the CLI golden age!
I’ve been writing command lines for parts of our app to help split up the monolith. Global state and dependencies thwart reasoning and thus debugging and performance optimization. Splitting out 500, 1000, 5,000, 10,000 or sometimes 50k lines of code to work separately can clarify a lot of things.
If done right it can also encourage Functional Core, Imperative Shell, because a sensible Unix-philosophy command line needs lots of actions without side effects and a few with. You can write a little command that generates and dumps out the system state just before a (bad) decision is made, and do so against production systems with virtual impunity. And that means you can hand these tools to someone who you need to become part of a bus number, even if they are otherwise hesitant to do so.
If you substitute "people" with "computer users" (which is obviously implied, despite your pedantry) then the author's jist is correct. What a villager in a remote undiscovered Amazonian civilisation thinks of the terminal is not relevant.
If you want to judge the shift in quality in a thing that grew in quantity, then you should look at percentages — otherwise you end up creating statements that are true, but don't say anything meaningful.
E.g. there are probably more absolute listeners of Jazz music today than back at the height of the cultural bloom of the genre. But that isn't because Jazz is more popular today than it once was, but because there are more absolute listeners of any kind of music. Would you say that Jazz in the US is now more important, influential etc. than it was at its peak?
No, but there's so much great jazz these days that wouldn't have existed without those absolute numbers. Quantity (aggregate supply) is a quality (increases specialization of labor) all its own!
Yeah, sure. But I hope you are aware your argument is dishonest? If you want to compare two times, looking at absolute numbers isn't useful unless those numbers contributed to the rise or fall of the phenomenon we are observing.
Whether from that quantity new qualities emerged is a different question. E.g. the new quantity of diverse jazz listeners probably has lead to an explosion of new sub-genres, that might have never emerged otherwise.
Still, just because there is more different Jazz now does not mean Jazz has become more relevant overall. This was the factor we discussed btw.
Please also consider a --dry-run option that gives a preview of what actions would be taken without actually making any changes. This is really helpful for learning a tool and making sure you got complex options and fileglobs correct before committing to potentially irreversible changes.
I'll go further. PowerShell covers a lot of the concerns in the OP out of the box. It is extremely well thought out and is one of my favorite cli models to buy into.
Powershell is just plain a great shell. Blows all the *sh variants out of the water. I'd love for it to gain traction in Linux (so that, for example I could use it as my shell on my desktop) but I don't really see that happening.
I use MacOS on my personal machine and Linux for various shellboxes and I switched to Powershell years ago and haven't looked back. Occasionally I invoke bash as a language runtime for checking shell script stuff, the way I would any other language's REPL, but for a shell? Powershell is strictly better.
The one actual problem with Powershell in this area is quoting for external commands. It's solvable in scripts by replacing certain things with double-quoted equivalents but not really interactively, and it is an occasional pain (though I still think it's overall less of a problem than quoting in general in bash and its ilk).
WhatIf in Powershell also remains completely broken; not that it doesn't work, but it is not properly passed down the call stack like it's supposed to from cmdlets or functions written in PowerShell (as opposed to those loaded from .net assemblies).
I have seen it, I think it's because it's not easy to see whether the cmdlets or functions your calling support and support it right. It does work, but I have seen it not always work and that results in having to check or test every command. I think the GP is overstating the incovenience, though, and Powershell is still much, much better than any Unix shell in this respect (which simply has no mechanism and no standard way to discover one).
Even safer: your program does never perform any actual deed. It just prints commands that you can run afterwards, using an external program, ideally a shell. This has the advantage of allowing the user to edit all the actions one by one.
Instead of
myprog # see what would happen
myprog --commit # alright, do it
You do
myprog # see what would happen
myprog | sh # alright, do it
But if you want to change something:
myprog > x
vi x
cat x | sh
And if you just want to run everything in parallel:
apt-get. It simply tells you what it's going to remove/install.
Calling an API to retrieve data is not really the type of program that requires a dry-run flag. It's mainly useful for commands that change the state of something in ways that could potentially be destructive, unwanted and/or hard to revert.
I think there are ways to detect if stdout is a pipe and operate differently (e.g. by using different defaults for coloring output), but I'm not sure if there are ways to detect what the other side of the pipe is, much less what `dry-run` would actually be expected to be in this case
Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful)
But at that point you could just handle —dry-run directly
> Maybe I’m misunderstanding but sounds like you’re proposing the modifying the command on the left to detect whether it’s piping it’s output (curious how you do that btw sounds pretty cool / useful)
You can do that. Possibly not from all languages but for anything that can call functions in the c standard library, that’s what isatty() is for (among other uses). It takes a file descriptor and returns whether it is a terminal or not. If you do this with stdout, this tells you whether it goes to a terminal or whether it is redirected in a way.
As the parent suspects, though, this won’t tell you anything about what is on the other side of the redirection.
It also doesn’t tell you what is the terminus of a pipeline. Because often with isatty what you really want to know is if the pipeline ends with a tty. `ls` is screen formatted but `ls | less` is not without extra work.
I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command to not do anything because stdout was a pipe.
> I often pipe output to tee and would be pretty annoyed if that changed the behavior of the original command
The output sent to tee is usually not the same as the output from the command to the terminal, so you are getting something different than most human users expect from original command... the reason is that terminal escape codes and other formatting for humans may need to be omitted from output to a pipe. You do this by asking the OS, "is this thing a terminal?".
Python example
"terminal" if sys.stdout.isatty() else "something else"
Sure; the output format changes, but the functionality doesn't.
Even ls outputs a tabular format by default when it's on a terminal and a list one file/dir per line when it's not on a terminal (if it's piped to cat for example or why ls | wc -l correctly counts the entries).
But the (essential) behavior of the command remains the same. ls still lists files/dirs... scp still copies files, etc.
> But the (essential) behavior of the command remains the same.
Of course. A command needs to do it's defined function.
You'll find some programs that are quite a bit different when invoked from outside the terminal vs inside the terminal. Developers need to take into account both situations, which is really the point the original post.
You can see this pretty easily with `ls` as well by using different options for the color. If you run `ls --color=auto`, you'll get colored output when running directly but black and white output if you pipe to `less`. However, if you pass `--color=always`, you'll get colored output when running directly and a bunch of garbage around some of the entries when piping to `less` because it doesn't interpret ANSI escape codes for color by default (although depending on what the output you're piping into `less` is, there are some workarounds like https://www.gnu.org/software/src-highlite/)
If it operated differently based on what it was outputting to, that kinda defeats the point of 'dry run', which is to see exactly what's going to happen based on what's on the other side of the pipe when I run it for real. "Did this blow up because it's a bad command, or because there's a bug in the 'only kinda real dry run' code?".
What if there wasa 'deep-pipe' '||' which would be based on a set env/docker/blah - which would launch an env and execute your '||'d code in it, and output some log/metrics/whatever?
on the other hand you could have 'dry-run <command>' that via .so interposition tricks could intercept and list all destructive changes done by an arbitrary <command>, as a form of sandboxing.
Not generally because you can't e.g. know if a write to a socket is destructive or just a query for information needed to decide the next steps/output.
Here's a possibly more interesting take on this: instead of do-thing.py, imagine if there was thing.py, which processed all that complex CLI options, and as output produced a linear shell script, with each line invoking some atomic operation, and a comment explaining which options contributed to that particular line.
Yeah, that's a pattern I use for occasional sysadmin tools - the command itself generates the actual commands, I review them, then "<up arrow> | sh -xeu". (Yes, there's no guarantee that the output is the same, so I don't use the pattern when that's a risk; it's also rarely if ever used for things I expect other people to run, just bulk operations that are saving me repetition.)
"try" is not a prefix (like watch, nice, etc) uses an overlayfs in order to be able to see and accept or reject changes to your filesystem from a command
> If stdout is not an interactive terminal, don’t display any animations. This will stop progress bars turning into Christmas trees in CI log output
Never display animations in stdout! I quite liked TFA in general, but I was skimming around looking for where they were going to advise on the difference between stderr & stdout until I saw that and realised they weren't.
stderr should be all (not just 'errors') of your logging, informational type stuff, the bits that maybe you might animate (and some people will hate) if tty, etc.
stdout should be the useful output - which you may or may not have - regardless of whether tty or not, primarily because an inconsistency like that is just confusing.
e.g.
echo foo | mysed 's/oo/aa/' | cat
# mysed should:
# stdout: faa
# stderr: mysed version 1 here hello\nfound oo\nprinting aa (or whatever)
I don't want to have to fight your tool with grep to get the 'actual' output lines. And I don't want to struggle to debug it because if I remove `| cat` above (as a silly example) it behaves differently than with it.
To add a small tweak to this rule... "stdout should be what you ask for".
If I ask for --help, that should go to stdout. If I ask for logs, they should go to stdout. If I don't ask for it, stderr.
If we could go back in time and make it stdin, stdout, and stdext (because UNIX so six letters, but standard_extra, or standard_extended), we might have a prayer of getting people to follow that convention.
But it's called stderr, so devs think, quite reasonably, that it should be used for error reporting, and conversely, if it isn't an error, it goes in stdout.
But I agree with you that this is the better way to structure a program. You might confuse more people with it, but they'll be able to do more useful things with its output, so that's a win.
I think if I could go back in time, my fix would be to make stderr the default. i.e. everything written 'out' goes to stderr until you explicitly write it as an output to stdout (or another descriptor/file), the inverse of the current situation.
Unix doesn't really have a limit that I know of, certainly not a low one, stdin/out/err are just names for the (very) common descriptors numbered 0/1/2 respectively; you can use another. (In fact it's sometimes helpful in scripting for dealing with exactly this issue of potentially badly behaved subprocesses.)
> Use symbols and emoji where it makes things clearer.
for the love of god please don't. the yubikey-agent example provided exemplifies everything i dislike about github READMEs and whimsical user interfaces.
on the technical side, symbols and emojis can render inconsistently among terminals, leading to potentially confusing messaging. on the artistic side, personal tolerances towards whimsy and playfulness vary wildly and should only be used very sparingly and ONLY if you know what you're doing (if you have to ask, you probably don't)
I like it. I like colored terminal output, and emoji are colorful, which helps me rapidly form a gestalt of what's going on. I like syntax highlighting too, and find code quite a bit more difficult to read without it.
Not everyone is like that, and that's ok. I don't expect my whims to be catered to, and you shouldn't either.
Strongly resonates. The first time I saw the CLI Guidelines I even stopped reading them upon reaching that section. This time I read through the remainder and found it quite OK.
I get that some CLIs are absolutely huge and require nesting (like aws), but it really drives me nuts traversing nested CLIs. I'd rather most apps spit out all their options in the help and let me use less to find what I need instead of going into each level and running help.
I've spoken about this previously, I write a huge number of TUI apps. I write every single app the have a nice TUI frontend for people who are less technical (like QA) that is purely the UI/UX IF you want to use it. What that does, in all of my apps, is pass the settings chosen in the TUI to some sort of separate generator file or function that can always be called by it's own on the terminal with all options/help that are used in the TUI.
So it's scriptable and useful by multiple levels of skill.
I find it depends on the number of subcommands and if those subcommands are clear. It's rather useful to have the options whittled down to just the ones you need if you know which subcommand you need but otherwise can be painful.
Grepping through a large list of options is also painful, seems ther needs to be a balance here.
But we are comparing the pain of searching vs the pain of having to re-run the help command for each possible subcommand, and parsing through all of that. Much easier to just have one command invocation and search through it.
There is a web analogy for this in how people organize FAQs. Some have a list of section links, and you have to click on a section to get the FAQs for that topic. Others just put everything on one giant page.
Here's the problem scenario with splitting things up into section pages: You think you see the appropriate section, but then you don't see your concern answered. There are two possibilities: either the organization was counter-intuitive and your concern was answered in one of the other sections, or your concern wasn't answered anywhere. And what's the only way to be sure? Visit every single section page and search through all of them.
Much, much less painful to just have it all on one page and search it.
> Traditionally, UNIX commands were written under the assumption they were going to be used primarily by other programs. They had more in common with functions in a programming language than with graphical applications.
Not quite. They were primarily intended for interactive use within a login shell. There are the programs which generate output on stdout (ls, cat, find, tty, who, date), and there are the "silent" text filters (tr, grep, cut, uniq, sort, wc). A one-liner would enable you to do basic computing tasks in that era. Any complex program would be written in C. After the appearance of DSLs like sed and AWK, certain string-heavy programs were offloaded to the shell.
The shell is not a sane programming environment and was never intended as such.
> There are 10kloc C programs that could be 10 lines of shell
Only when the shell calls other external C programs. Ten lines of calling ffmpeg or curl is not shell programming.
> there are 1kloc shell programs that could've been 100 lines of C
The 1kloc shell programs are fragile spaghetti that breaks in weird ways. Any invocation of an external program can fail for a variety of reasons, and the shell doesn't provide adequate mechanisms for dealing with it, apart from exit codes and filtering error text output.
> Only when the shell calls other external C programs. Ten lines of calling ffmpeg or curl is not shell programming.
100% wrong. this is what the shell was designed to do and where it is at it's best.
often shell "scripts" are used like "macros" or power-tools, shortcuts to save off a complex invocation or workflow. error handling isn't as important in a one-off and "adequate" is whatever gets the job done for the user, which it does.
it's rare that 1kloc shell script is the best engineering choice vs. (in the ancient days) Perl or (today) Python. e.g. "real" programming languages. you mostly should not write large programs in shell. and you really should not glue together pipelines of external programs using, for example, Python or C, which is onerous.
ah, the HN crowd: where everything is either black or white, great or terrible. how about "each to his own" and "use the right tool for the job"
It's not rare at all. In every corporate environment I've worked in, my choices were PowerShell or 3 months of red tape, meetings, and security audits.
I get that in a dev shop that's not the case, but most businesses employ 0 devs. So people end up being forced into shell scripting because it's the only approved option.
Depending on what you're doing, C# might also be an option - since the compiler is a part of the .NET Runtime (not just SDK), it's available on any Windows install with PowerShell these days. It's too bad that the system one is always an old version that doesn't do .csx, but still.
it's not rare, it's rare that it's the best engineering choice.
if your management can't allow even a python script, but it can allow bash? then it's not an engineering problem. as you even allude - your business does not have a software engineering culture.
a good engineering culture does not depend on job titles or budget or degree either. but it can't exist without healthy management.
It's much easier to control allowed functionality in PowerShell via security policies. Python is comparatively a free for all. It makes sense in a non-dev environment.
> Ten lines of calling ffmpeg or curl is not shell programming
Ten lines of calling ffmpeg or curl is shell programming in precisely the same sense that 100 lines of C that `#include <sys/socket.h>` are C programming.
If your code isn't standing on the shoulders of giants, you're probably wasting everyone's time.
> Any invocation of an external program can fail for a variety of reasons, and the shell doesn't provide adequate mechanisms for dealing with it, apart from exit codes and filtering error text output.
True, handling multiple sub-processes is difficult in most Unix shells. This is a function where Powershell could have done so much better, but didn't. It is somewhat better than bash&co, but could have been much more so. Python does it much better than Powershell does.
Sane or not, shells are programming languages, and in early Unix this was quite a bit more prominent and obvious, the fanout into sh, bash, ksh, and csh being exemplary.
To me it makes total sense to think of the standard POSIX toolkit as the standard library of the various shell languages, that seems basically correct in fact.
I think your example self-explains why it's broken by design. It's a good example.
> a command line interface must be human readable and machine readable at the same time. There is no canonical way to solve this problem.
And you know, there could be one. Apple has Human Interface Guidelines to reify the meaning of the visual abstractions in its desktop UI. The problem is that the command line didn't come from people who think like Apple designers; it came from people who think "How can I express what I want using the least code possible, because laziness, impatience, and hubris are virtues?" And they weren't wrong for the time (especially because every byte matters), but the design decisions they made got baked into tooling that can't now be moved.
I think at this point we'd have to punt the POSIX toolchain to get something better; it's hard for me to imagine how we'd build discoverable, conceptually-consistent UX atop what we currently have.
I think graphical user interfaces are no high standard. They do not compose. How would you express a loop or a recursion in a graphical user interface?
Great for discoverability though, and that doesn't require graphics, just context.
There should be a button I can push in my shell that lets me ask "what does the token at cursor mean," and a button that lets me type a plain language search string that wires down to a contextual search (i.e. I'm in the middle of typing out "grep" I should be able to ask "how do I search folders?").
We didn't have the tools to build this when grep was invented; we have them now.
I'm genuinely unsure how to translate your question into, hmm. Anything actually.
The snarky answer is "By writing it in source code, in a GUI text editor", a thing I do frequently. But the problem is that I have no idea what you're getting at in the first place, so that's just an attempt to recover some meaning from what you wrote.
Shells allow one to pipe around the data flowing from called process to called process in a very smooth and transparent fashion.
By and large, GUIs do not. There is no such thing as a universal shell for GUIs, and attempts to layer automation atop the GUI abstraction are generally spotty and unreliable (certainly when compared to CLI and shells). This is both for technical reasons (i.e. it's much easier to clearly delineate the two ends of a pipe than to clearly delineate "I want to click on the red square inside the 'diagram' window inside the drawing app") and for ecosystem reasons (since GUIs aren't thought of as automatable, GUI designers are free to move pieces around version-to-version of software, making it extremely challenging to describe a GUI structurally).
I've seen some neat attempts at GUI automation (Sikuli is my favorite) but it's never been a core feature like it is in the CLIs-glued-together-by-shells world.
CLIs are in the modern world a subset of GUI programs which run inside a text-oriented window, and plenty of more sophisticated GUI programs can be directed textually, with macros or short scripts, and/or have a command-line component for doing that sort of pipe or batch-oriented work.
Yeah, sometimes there's functionality stuck behind a button or menu select which I want exposed in a more textual way, in macOS that's when you break out Automator, Alfred, or Hammerspoon (the only one I've ever used fwiw), Linux and Windows have their own equivalents.
I don't think the distinction you're pointing to is nearly so stark or clear-cut as it's often made out to be. Ecosystems converge towards the tasks which are amenable to batching and pipelining being equipped to do so.
But in general, batching and pipelining are done via a text-based interface. Extremely rare is the teachable GUI interaface where there's a point-and-click way to describe the concept "See these five items I clicked? Do that twenty-eight more times with this radio button family swizzled via a linear sweep."
My point being, I don't see why there ever should be. A text window is a valid and frequently-included part of GUI programs (I happen to be using one this instant, in fact), so there isn't a lot of advantage in replacing something like Lua scripting with a bunch of buttons and whizbangs. It's been tried for normal programming, and no one likes it.
The specific kind of task you described is frequently exposed as macros in programs complex enough to deserve it.
> so there isn't a lot of advantage in replacing something like Lua scripting with a bunch of buttons and whizbangs
I mean... "no-code development" is an entire category of product offerings. People are positively thirsty for being able to do that sort of thing without having to learn to love staring at giant blobs of text all day.
> I think at this point we'd have to punt the POSIX toolchain to get something better; it's hard for me to imagine how we'd build discoverable, conceptually-consistent UX atop what we currently have.
Posix could potentially do this. It already has a bit about conventions and could be expanded. Problem is getting things to adhere to it, plus I doubt the posix authors could be convinced to add a lot more to it.
I don't really understand the question. Nobody got to fly until we figured out how to build airplanes, people before that lived on the ground. Likewise, we'll live with iterations on the current command line, until AI is fully integrated with it.
Never ever. Humans are idiots. If you build machines like humans you will get idiotic machines. Machines need to do things exactly right and not almost right or mostly right. Current AI trends will not create better machines, just more idiotic machines. Those idiotic machines will be sufficient to impress idiots, but they will not help to do things exactly right. Automated theorem proving is the only way to build better machines.
> Automated theorem proving is the only way to build better machines.
That is only true for a very narrow notion of "better machines". Automated theorem proving is still light years away from being applicable in the majority of software projects. Don't get me wrong I am aware of the progress made in the last decades. Yet, it will be some time until someone writing the next iOS app will reach routinely for an automated theorem prover to lower the defect rate.
And even then, the questions remains whether fulfilling a formal specification is at all correlated with "better machines" or "better software". There are domains where this is conceivable: os kernels (L4), certifying compilers (CompCert), and others. But how does a theorem prover, automated or not, help with improving the next generations of video codecs? That is an intrinsically subjective problem -- the quality axis, less so the performance axis. How does theorem proving help with neuronal networks? How does it help with capturing the right business process to actually improve business outcomes and not just introducing new bureaucracy?
Automated theorem proving will only ever be an arrow in the quiver, because even in theory, the space of useful computer programs is a vast superset of those which may have all their properties formally verified. In practice, it's a much larger superset than theory allows. Growing the space of formally verifiable subprograms is a worthy endeavor, sure, but so is good old-fashioned engineering.
> Automated theorem proving is the only way to build better machines.
Sure, but that will just be one aspect of an integrated and wholistic AI, which can configure the parameters input to the automated theorem proving component, and act on the results.
> The problem: a command line interface must be human readable and machine readable at the same time. There is no canonical way to solve this problem.
It's not really the "same time". Usually, human and machine use the same command, but at different times. And there are many ways to enable different outputs, even at the same time. But this all depends on having some standard which everyone follows. And that's where it becomes complicated.
Your example not only doesn't prove that it's broken by design (only broken most of the time ?), it even seems to give a (relatively ?) easy way to fix it ?
> Since very few implementations of ls allow you to terminate filenames with NUL characters instead of newlines
In fact the solution to this issue seems to be so obvious that I might be missing something ? (Rejection by shell interfaces for some reason ??)
Credential files are a good, simple, portable option. Files have permissions already. They don't depend on an external service or a proprietary API.
And, if your program accepts a credential file, it will be compatible with systemd credentials. systemd credentials offer more security than an unencrypted credential file. They are encrypted and can be TPM-bound, but they don't require the software using the credential to have native TPM support.
I like it when programs have a way to specify a command to retrieve secrets.
mbsync (https://isync.sourceforge.io/mbsync.html) e.g. has afaik 3 options to provide a password for IMAP authentication:
If you don't configure a password, you'll be prompted on execution.
You can also put the plain text password in the configuration (impractical if you want to share your configuration).
But there is also a configuration option to provide a command to retrieve the password.
That way you can delegate the password handling to another program, e.g. a password manager like pass(1) (https://www.passwordstore.org/) or some interactive graphical prompt.
A secrets management service would be most convenient. Documentation makes them easy to set up without having to build anything extra yourself. A secrets manager like Doppler (https://Doppler.com) or AWS Secrets Manager (https://aws.amazon.com/secrets-manager/) has the advantage of protecting your secrets in a secure place and the advantage of minimizing exposure of those secrets - even to your own developers. That way, you don't end up with a data breach that could have easily been avoided. These types of leaks can cost companies everything and are becoming way more common.
What I find super irritating is that some terminals will automatically execute a command if you paste it from the clipboard and there is a newline char at the end. IMO command line interfaces should not do this.
Fish shell by default does the expected behaviour of inserting in and allowing you to edit the command (multiline if needed) before executing it with [enter]. I've found Fish shell to have a lot of sane defaults and have yet to find a thing I would like to customize except for the prompt.
If i know there will be at most one newline, i start by typing "# ", then paste. If the newline gets interpreted, then the whole line is just a comment anyway. It's then easy to edit the # off the start of the line before running it for real.
You find it irritating that terminals execute commands when they receive a newline, which is indistinguishable from pressing the enter key? The program is doing exactly as it should. Consider not copying newline characters, and you will solve the problem.
I thought much the same thing when I read CLIG. I'm a big fan of the 17 Unix rules he puts down, though, and I think a lot of really good equilibria exist on the spectrum between the two.
Whart's this based on? If anything, this is among the more concise set of interface guidelines I've come across. Design guidelines typically run into hundreds of pages long when PDF'ed -- this page is just about thirty on default "Save as PDF" settings...
I know it goes without saying for most of us here, but actually being a heavy terminal user yourself is one of the most important things to understand how to design CLIs. It helps a ton to understand the ecosystem you live in, not just your own organism.
Example: Something I did a few months back ago for a tiny personal project @ https://github.com/hiAndrewQuinn/finstem was implement `--format CSV`, `TSV` and `JSON` flags. I haven't had need for any of these myself, but they exist so any future people who want to use `csvkit`, `awk` and `jq` respectively to wrap around my program have easy ways to do so. That's not stuff I would have had the instincts to do if I wasn't myself a user of all 3 of those programs.
>being a heavy terminal user . . . is one of the most important things
I wish people wouldn't conflate CLIs with terminals. I run (shell) command lines all day, but try hard to avoid terminals / apps that emulate terminals.
(To run command lines, I use Emacs, which I never run inside a terminal.)
We recently chose cobra[1] to create a cli application. It comes with so many best practices already packaged like autocompletions, help texts etc. etc.
tar isn't a POSIX command, and retains its argument format from before the POSIX CLI guidelines were standardized (as does the POSIX ar command). pax, the POSIX-specified (but rarely implemented/used) equivalent of tar does follow the POSIX CLI guidelines.
Of all the challenges with using CLIs, the one that bites me consistently is capitalization of flags. Really wish we'd standardized on being capital-agnostic (or even demanding lowercase only).
... but we didn't, so now you have to memorize whether recursion is capital or lowercase R, restart is capital or lowercase R, poweroff is capital or lowercase P, etc.
Most utilities I use daily have --long-flags for all of these. Using completion (prferrably https://github.com/unixorn/fzf-zsh-plugin) you can complete them pretty quickly and without knowing the precise name.
"Don’t allow arbitrary abbreviations of subcommands. [...] you allowed them to type any non-ambiguous prefix, like mycmd ins, or even just mycmd i, and have it be an alias for mycmd install. Now you’re stuck: you can’t add any more commands beginning with i, because there are scripts out there that assume i means install."
Please avoid the use of short arguments in scripts. It makes the least sense there. The short arguments (along with aliases, abbreviations, and whatnot) are a convenience for human usage, to reduce the amount of manual typing. In scripts you can be explicit with minimal cost (and you also should, considering the ratio of writes vs. reads).
whatever happened to "Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new 'features'"?
now we got lots of mega-cli programs, each one with its own distinct option language.
Examples: kubectl, docker, openssl, git - (git got two command line languages: plumbing and porcelain)
> It is assumed that command-line interfaces are the opposite of this—that you have to remember how to do everything. The original Macintosh Human Interface Guidelines, published in 1987, recommend “See-and-point (instead of remember-and-type),” as if you could only choose one or the other.
I find that guis have worse discoverability and cli better. It's pretty hard to search for a gui affordance. Plus they are essentially unscriptable.
That said.
I think CLI programs are user friendly for professionals. Because they support input-process-output. Where input is STDOUT read by human, process is thinking by human and output is keyboard input to STDIN by human.
UIs for the general audience?
TUIs! TUIs are easily to parse, succinct in organization and fast input - all for humans. Humans can parse TUIs. And they can make up a mental modal.
GUIs fail often with a lack of organization, information overflow and distractions by weird metaphors.
The Windows 95 desktop metaphor is an example. It doesn’t make sense. Same for Windows 11 and its file-browser which makes it hard to recognize the filesystem or even just the home-directory. Now open Nautilus on Linux, it opens by default your home-directory (in most cases the place to be).
I like the CLI but TUIs are my love. GUIs are okay if are like a TUI.
> UIs for the general audience? TUIs! TUIs are easily to parse, succinct in organization and fast input - all for humans. Humans can parse TUIs. And they can make up a mental modal.
TUIs have the same drawbacks as GUIs and then some. Their big advantage is to work seamlessly over SSH but that’s pretty much it. They are not more discoverable and they are not more efficient than GUIs. There is nothing preventing you from having a decent GUI along the same lines as Midnight Commander to have a file manager without the metaphors you dislike, for example (as a matter of fact, there are several).
Working over SSH is not the only TUI advantage: IMHO one of the greatest benefits of TUIs is the usage of characters to display the UI. This way, the font size is always equal (no unreadably small fonts).
When designing TUIs I found this to be limiting in a creative sense -- I have to really think about how I arrange the TUI elements and information because I cannot put as many elements as I can on a GUI in the same screen space.
Also, TUIs seem to be mostely unaffected by the trend to make every GUI element “touch-friendly” large which is an advantage for me as a Desktop user.
TUIs are ideal (sometimes) where a command needs to be interactive. Many commands lend themselves well to batch processing or require no interactivity at all. In many cases, a script piped into a text editor (which is a TUI) is all that is needed, sparing apps from having to embed a text editor and deal with all of the design choices. Other times GUI will work a lot better.
The worst part about terminal programs is that they can’t be deprecated. Or I don’t know of a way to do it. Because once you release something someone might immediately put it into a script somewhere instead of running it interactively. Now what do you do? Display a “hint” about the deprecation? Well no one’s gonna read that because it’s a script which is run non-interactively.
So you just have to design the UI perfectly on the first try. That’s possible for small tools but what about larger ones? Past a certain point it becomes a truism that do-it-once perfectly without iteration is impossible.
How is it similar? I get deprecation warnings if I update the library. I can pin the version of the library. A library is something I work with, unlike a terminal program which might be written and forgotten.
Then those terminal programs get upgraded on the next system update because hey, you’re supposed to the get latest version right?
Terminal programs can do the same (`-v1`) in principle. Few do.
Not quite that simple. You can't just pin the program itself, you also have to pin its dependencies. Which means you have to run ldd to get the libraries it loads (whether directly or transitively), and copy those too, and then you patch the program and its dependencies to set the rpath to $ORIGIN/my/own/directories, then you have to examine all the other more subtle application-level gotchas (e.g. path search order for config files, hard-coded absolute paths, etc.) and port those too. Once you're done, congratulations! You've done 50% of the work of a package manager.
There's something I wish command lines would not adopt that is should not be as a CLI flag, and that's --json or --csv. Instead this should be offered as an environment variable that runs through all the commands like IOFORMAT=JSON { cmd1|cmd2|cmd3 } if they support the environment variable, they can all read input in a safe way and behave appropriately, if they don't, then they just print as normal
(BTW sometimes ENV vars are bug removers and sometimes a terrible source of mysterious bugs as well as a possible injection vector.)
A good option package should support long options being both command line flags and environment variables. The really good ones support them in init files as well (I'm partial to the JSON-ish HOCON format, though .INI works too).
I always spec (later overrides previous settings):
- system default (/etc/foo.conf)
- user defaults (~/.foo.conf)
- local defaults (${CWD}/foo.conf)
- user-specified init file (overrides local default)
- environment vars (FOO_OUTPUT=JSON)
- command line var (--output=json)
I disagree of having programs use env vars, ENV propagates, now every single program has an env var for that single program that you barely use, a good example is LSCOLORS, or nnn, all programs now consume extra memory. Which isn't a lot. but it adds up if every single program decides to use ENV.
I wouldn't worry about memory. As I noted, the propagation value of environment vars can be very handy (make sure the programs are communicating via JSON, from an upstream example) and can also lead to confusing bugs (why is this generating JSON output???)
I don't see the problem, you're just making something explicit what used to be implicit, unix is implicitly tsv, well now its a variable that you can change.
These are great. Most frustrating to me is that the basic commands: rm, mv, cp, touch, mkdir do not follow some of the basic guidelines, such as “ask before destructive action”. I have been reimplemting them in Rust for myself to fix this (although a wrapper would have been perhaps better, it’s hard to write it and have the program work in any shell).
If your program detects -? and tells the user to use -h or --help or --long-help or --full-help you should be kicked in the crotch. Just show me the help.
I'd say -h and --help should return 0 & print help to stdout. Any unknown option should return EINVAL and output an error message and help text to stderr.
I often show help if an unknown option was given but in the cases I don't I am not going through a list of special cases that I don't even know of to try and determine that someone wants the help text. I've never even seen -? before that I can recall.
There is a lot of value on understanding the context and trying to parse what you meant in a mangled command. For the foreseeable future, that value will fall entirely on documentation and search procedures, and chat will keep a steady negative value for actual interaction.
The current shells aren't deterministic either. Everything depends on history, context, environment.
Early AI shells will ask for confirmation like "ansible --check" or "terraform plan". Running commands in a disposable virtual environment will inspire confidence. When they are trusted enough, direct execution.
Soon after you're able to say "Set up a k8s cluster for me!" you'll decide you don't need k8s anymore, you'll just introduce your AI model to your data-lake and be done. The days of thinking about how many nodes do I need, which immutable linux distro, t5g vs m4.xlarge are almost gone.
Read the lyrics to Graham Nash's "Teach Your Children", substituting "model" for "children" as necessary. :-)
I'm not saying that what you describe is impossible, but we're clearly far from the point where AI can be trusted to do something like that unsupervised (which is a necessity for any kind of scripting). So declaring that "command line is dead" is rather premature.
Improving the existing command line for humans is a waste of effort. It's an heirloom activity already, unless you forbid the use of LLM AI.
It's inefficient for even the simplest AI model to use a CLI, so API's are the future. But using API's to create a k8s cluster are pretty primitive too. The nearish future is things like unikernels running LLM's and rule-based logic engines and data-lakes. LLM's can help you create rule-based logic engines and data-lakes, so win-win.
Window desktops are dead. MacOS is a zombie with lipstick. Linux desktops are like playing an electronic harpsichord. Linux servers and Windows servers are dead too. Who cares? CEO's ask "How many widgets did we sell this quarter and will I get a bonus?" Regular people ask "When is my next dental appointment, how do I get wine stains out of my rug, what's good on TV? " [There are more important questions but people probably won't ask their device and expect a useful answer]
All of these things current LLM AI can answer with 90% accurancy, which is better than human accuracy. General AI is not necessary for pretty significant changes in the status quo. The IT sector's only future purpose is to help train models, IMHO. And future means now, not in 10/20/50/ years.
LLMs are great until they start to hallucinate on subjects where you don't know enough to catch them. We're nowhere near solving that problem yet, and you're painting a hopelessly optimistic picture. Here, now, CLI is still in heavy use, and it's not going away wihin the next few years.
I was going to comment in protest - "but the CLI has always been there, and it'll always be..."
But then I realized that you're probably right. I'm kinda looking forward to looking back with bewilderment on those decades of doing CLI stuff with remembering all those commands and shell intricacies and whatnot.
In my vision of the future I see one of my descendants 100 years from now cussing furiously that his AI avatar is borking up his request, and one of the local neckbeards walks up, does an obscure incantation that pops up a command window, types a few lines of text and fixes the problem right up. "Newbs...geeze" he says as he walks away.
Maybe the proles will one day be permanently denied the CLI, but even if only as a lowest common denominator administration tool of last resort, I predict there will be a CLI somewhere.
If you have -r / --recursive, I would rather just have -recursive work as the others. POSIX be damned, the double hyphen is just an opportunity to make more mistakes, and the combined -rptgo single-hyphen flags are more opportunities to make mistakes.
Let some legacy programs like ls and sync keep their single-hyphen combinations. Most new programs should just accept separate flags, and allow either single or double hyphens.
Edit: Yeah, this opinion collects downvotes, doesn’t it? Y’all love it when you accidentally type -recursive instead of --recursive, and it turns out that it means the same thing as -r -e -c -u -s -i -v? That never made any sense to me, for the vast majority of tools out there. It sucks, to be honest.
I don't like combined single-hyphens for a different reason: sometimes they accept arguments, and it seems strange to me that `-a0` can variously mean `-a 0` and `-a -0`. (I think some insist that argument-having options are long, and short options can only be flags, which sort of solves it, except for the existence of the others which makes it still confusing.) That said, in a script, which is the main time I'd have to read the way someone else has written it anyway, I favour everything being --long --anyway --to-make-it=really-clear. (Though there are a few that are so common I won't insist on, `cut -dX -fY` say.)
>I don't like combined single-hyphens for a different reason: sometimes they accept arguments, and it seems strange to me that `-a0` can variously mean `-a 0` and `-a -0`.
I think the answer to that is just to stop designing argument parsers that way—if you want -a 0 then maybe -a 0 is acceptable, maybe -a=0 is acceptable, but -a0 should not be. I don’t see how this part is controversial.
For the same reason, -all should not be parsed as -a -l -l.
>I favour everything being --long --anyway --to-make-it=really-clear.
Yeah—I think it would be equally clear to write it like -long -anyway -to-make-it=really-clear, if we decided to make more parsers that worked that way. Some parsers do work that way.
>You must like `find`.
I have to assume that this is just sarcasm. The `find` command gives you a DSL for writing queries, and for some reason, the tokens in that DSL are option flags starting with -. Bizarre. I don’t think anybody wants to design something like that, and I don’t think there’s really anything to learn from find except maybe “sometimes, for historical reasons, the command-line arguments for standardized tools just plain suck.”
This is true today, and it was true as well "in the 1980s", to use the same time frame as TFA. The difference is that today there are more people than ever who know what the command line is, and who can use it. At least an order of magnitude more people; maybe two. We can certainly say that we live in the CLI golden age!