
Things to Know About GNU Readline - matt_d
https://twobithistory.org/2019/08/22/readline.html
======
gumby
> Ramey told me via email that Readline, far from being an original idea, was
> created to implement functionality prescribed by the POSIX specification,
> which in the late 1980s had just been created.

This actually isn't true. The POSIX shell spec (1003 IIRC) was published in
1992. The headline library was in the first bash release; Brian might have
written it before starting on bash. The inspiration was the TOPS-20 input JSYS
(system call) mixed in with Emacs.

How do I know? Well you can ask Brian; as he reminded me it came from design
discussions for bash I had with him on this very topic at the time,
specifically that interactive text input should be a library.

~~~
brianjhanfox
And I trust Gumby's memory almost to a fault :-) I had already written a
version of Emacs for the Apple ][ and Apple //e called Amacs in 1983 or so. I
loved Emacs and Zmacs, and wanted to build software that had that clarity of
architecture. So I knew I wanted readline to have as much of the feature set
of a genuine editor as made sense.

I think Gumby was instrumental in talking to me about making a reasonable API
for the completion part of readline. But I wrote a lot of code in the late
80s. And I wanted to use command line editing and completion in GDB, because
we didn't have it. And I thought that was idiotic.

~~~
inflatableDodo
Hi, any thoughts on what would be a good place to start if someone was looking
to write a terminal shell variant these days? Also, many thanks for all the
useful things. :)

~~~
gumby
As bash is GPLed you could start with it.

If you wanted to be more extreme perhaps you could think of a way to make lua
the scripting language, or embed full shell controls into python.

------
choeger
Maintained by a single person.

Let that sink in. How many developers and admins rely on readline for their
daily work? How many critical installations world wide depend on it?

As a contrast: How much revenue is generated with, and how many people work
on, something as mundane as a chat application for mobile phones?

Sometimes it is astonishing that our software eco system actually works.

~~~
heyoni
Hmm...maybe it should be repackaged into a Saas product with a monthly, per
user license!

~~~
rhizome
And a cookie notification.

------
westurner
I map <up> to history-search-backward in my .inputrc; so I can type 'sudo '
and press <up> to cycle through everything starting with sudo:

    
    
        #  <up>      -- history search backward (match current input)
        "\e[A": history-search-backward
        #  <down>    -- history search forward (match current input)
        "\e[B": history-search-forward
    

[https://github.com/westurner/dotfiles/blob/develop/etc/.inpu...](https://github.com/westurner/dotfiles/blob/develop/etc/.inputrc)

~~~
deeg
I do the exact same thing. This will probably be considered sacrilege by some
but I also remap tab completion to cycle through all matches instead of
stopping at the first ambiguous character:

# Set up tab and Shift-tab to cycle through completion options

"\e[Z": "\e-1\t"

TAB: menu-complete

# Alt-S is normal style TAB complete

"\es": complete

~~~
colejohnson66
> I also remap tab completion to cycle through all matches instead of stopping
> at the first ambiguous character

Windows PowerShell does this and I love it. I don’t understand why people
_wouldn’t_ want this

~~~
wruza
Because instead of helping me enter a long filename by its distinctive parts,
it forces flat mental read-decide-skip loop based on first few letters, which
is slow and cannot be narrowed down without typing out additional letters by
hand. It is like selecting from a popup menu through a 1-line window.

If it presented a real popup for file selection, combining both worlds (tab to
longest match, untab to undo, up-down to navigate), that would be great. But
it doesn’t.

~~~
nathanwh
For anyone who isn’t already aware, fzf[0] does this exactly (and more!)

[https://github.com/junegunn/fzf](https://github.com/junegunn/fzf)

~~~
Fnoord
Rust implementation [1]

[1] [https://github.com/lotabout/skim](https://github.com/lotabout/skim)

------
msarnoff
And if you ever find yourself using a command-line utility that doesn't have
readline support (`tclsh` and `smlnj` come to mind), you can use `rlwrap`,
available via your package manager of choice:

    
    
      rlwrap tclsh
    

and voila, no more `^[[C^[[D` spew.

~~~
davidw
Sigh... it always frustrated me that the Tcl folks never got something like
readline integrated. IDK about the legal details, but Ruby and Python somehow
managed.

tkcon is an alternative, graphical shell for Tcl that's a good alternative to
using tclsh on the command line, BTW.

~~~
kevin_thibedeau
Tclsh is only supposed to be a demo for embedding the interpreter. It consists
of only 150 SLOC. Anyone could add readline if they really needed it.

[https://github.com/tcltk/tcl/blob/master/unix/tclAppInit.c](https://github.com/tcltk/tcl/blob/master/unix/tclAppInit.c)

~~~
davidw
This is like software marketing 101:

You're going to have much happier users if you do that thing that "anyone
could do" rather than ask that each person who might want to use readline read
up on Tcl's C API and hack together something that links to readline.

------
unmole
See also, linenoise[0], a small self-contained alternative to readline and
libedit written by antirez who also wrote something called Redis.

0:
[https://github.com/antirez/linenoise](https://github.com/antirez/linenoise)

------
aidenn0
Also, according to RMS, CLISP is a derivative work of readline because of the
optional dependency on it. (CLISP is now GNU CLISP, but was not originally
part of the gnu project).

~~~
mikekchar
Imagine you are a photographer. You take a photo of something. The photo is
covered by copyright. I can't use it without your permission.

Now imagine that I write a book and I include your photo in it. This is an
infringement of copyright. I can't distribute my book, containing your photo,
without infringing copyright. It doesn't matter how much your photo
contributes to my book -- including it in my book and distributing it is
infringement. You might even be able to get an injunction to stop me from
distributing my book without removing your photo.

Now imagine that you put your photo under a CC "share alike" license. What it
says is that, while I don't generally have the right to use your photo in my
book, _if_ I decide to license my book using a CC "share alike" license, then
you will give me permission to use your photo. To make things simple, you give
blanket permission to _anyone_ to use the photo as long as whatever is
including the photo is licensed as CC "share alike" \-- in that case it is not
necessary to get specific permission to use it. Finally, let's say that you
make the offer that if I infringe your copyright by including the photo in my
project that you will _forgive_ that infringement if I agree to re-license the
project under CC "share alike".

Exactly same thing.

~~~
mikekchar
I was on my walk and I realised that I completely missed the interesting part
of this discussion (which was hinted at by the OP). So I'll do my best now
(with the caveat that this is a very complex legal issue and I am not a
lawyer).

One of the most crucial things about the GPL is that it is _only_ triggered on
distribution. You are specifically granted permission to use the software for
any purpose (freedom 0). This _includes_ embedding it in another piece of
software. As long as you don't distribute the result, then you are 100% OK
("distribution" has a specific definition which is stated in the license, so
refer to that if you have any questions about how that works).

The "anti-GPL hack" that some people use is that they say, "Well, if I don't
distribute a library that I use, then I am not infringing on the license --
because the license only kicks in on distribution. If the _user_ links the
software, then it's 100% OK because they are specifically given permission to
use the software for any purpose in the license. Whoo hoo!"

This is where the argument of "derived work" comes in. Imagine that you write
a poem. I then write a critique of that poem. I don't include the poem in my
critique because I don't have the right to distribute it. Just referencing
your poem is not an infringement of copyright. Just because I describe your
poem doesn't mean it's a derived work.

However, imagine that I decide to distribute my critique in a web page. Many
of the things I say depend on the actual text of the poem, so I'd really like
to add it to my text. But I can't distribute the poem. What to do? You are
distributing your poem on the internet too, but you have a license that
doesn't allow me to use it directly. Maybe I can be a bit clever here.

What I'll do is write a web page that _deep-links_ to the poem. I'll write
some javascript that allows the user to press a button and it will copy the
text of the poem into my document. That way I can refer to the poem without
actually distributing it. My document won't make any sense if the user doesn't
press the button to link to the original text, but the user will be doing
that, not me.

Is my website a derived work of the poem? This is a good question. I think it
probably is. My document does not "work" without the original poem. It depends
on the actual structure of the poem in non-trivial ways. However, I am sure
there are also good arguments for the opposite. Given the outcome of Oracle vs
Google where the judge decided that APIs are copyrightable, I feel fairly
comfortable in my position, but as far as I know it has never been decided in
court. Your guess is as good as mine (and better if you are a lawyer ;-) ).

But just to reiterate -- it does not matter how much of the program infringes.
If any of the program infringes than it all infringes. You can't legally
distribute it. So while it might seem weird that a very small part of your
code might be deemed to be a derived work of some other code and therefore you
can't distribute it -- that's how it works (and, to be honest, I can't see a
reasonable way of proceeding if it didn't work that way).

~~~
sokoloff
IIRC, clisp _worked_ just fine without GNU readline.
[https://www.gnu.org/software/clisp/impnotes/readline-
mod.htm...](https://www.gnu.org/software/clisp/impnotes/readline-mod.html)

If Lispworks writes a compatibility layer that allows SLIME to work with
Lispworks, they are not causing Lispworks to infringe on the Emacs software
license.

(Side note: for your hypothetical website critiquing another copyrighted work,
there is a somewhat unclear [in the sense that it's facts and circumstances
dependent] fair use doctrine in the US that almost surely allows genuine
critique of that work to include relevant snippets [while disallowing copying
the work in its entirety under guise of critiquing it].
[https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors](https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors)
)

~~~
mikekchar
It doesn't really matter if the code as a whole worked without readline. What
matters is if the code that linked to readline worked. It's that tiny bit of
code that may or may not be a derived work. If it is, then you can't
distribute it. No matter how tiny it is, if you can't distribute it, then you
can't distribute it.

Keep in mind that agreeing to license your work under the GPL is an _offer_
for remedying infringement. You don't have to accept. If you don't accept,
then you have to deal with the consequences just like any other infringement
case.

For the side not: It is true :-) However, just to be a bit playful, I should
point out that fair use is _not_ a defense in US copyright law. If your use is
fair use it is _still_ infringing. Fair use kicks in when determining damages.
So your infringing use will suffer no damages because it is fair use. However,
you should always be careful when relying on fair use because it is _not a
defense_ \-- your infringement case can go to court and you still have to pay
your lawyers!

~~~
sokoloff
From what I've read/studied (though I am not a lawyer), fair use is an
affirmative defense against copyright infringement in the US.

I'll have to think more about the readline interface bit you raise. My gut
instinct suggests that's not quite right, but my rational brain isn't able to
label/ID/argue _why_ , so your point has some merit. I'll need to think about
it some (so thanks for that [seriously]!)

Maybe there's _another_ line input library with a compatible binary interface
and MIT licensed and clisp is written to interface to _that_ library. That
would seem like it might fly.

~~~
cat199
libedit was written for just this purpose

[https://github.com/NetBSD/src/tree/trunk/lib/libedit](https://github.com/NetBSD/src/tree/trunk/lib/libedit)
(cvsweb appears down)

[https://www.thrysoee.dk/editline/](https://www.thrysoee.dk/editline/)

~~~
brianjhanfox
Interestingly enough, neither of these libraries are as objectively as good as
readline. For example, they do not implement a true kill ring. They aren't
macro programmable. I find it almost amusing that the authors were too lazy to
actually reimplement all of the extremely well documented behavior in
libreadline and libhistory.

~~~
sokoloff
If the purpose is "plausible deniability" to allow GNU readline to be linked
by users, that seems perfectly reasonable. Why work harder than needed?

------
gray_-_wolf
> Also, without some sort of indicator, Vim’s modal design is awkward
> here—it’s very easy to forget which mode you’re in.

Luckily for us, there is option for the indicator :)

    
    
        set editing-mode vi
        set show-mode-in-prompt on
        set vi-ins-mode-string +
        set vi-cmd-mode-string :

~~~
laktak
I prefer to set the cursor shape ;)

[https://stackoverflow.com/a/42107711/52817](https://stackoverflow.com/a/42107711/52817)

------
stock_toaster
Another thing you may not have known, libedit[1] is mostly readline
compatible, and BSD licensed.

[1]: [https://thrysoee.dk/editline/](https://thrysoee.dk/editline/)

------
jolmg
> The Readline API is much more extensive than that single function of course,
> and anyone using it can tweak all sorts of things about the library’s
> behavior. Library users can even add new functions that end users can
> configure via ~/.inputrc, meaning that Readline is very easy to extend. But,
> as far as I can tell, even Bash ultimately calls the simple readline()
> function to get input just as in the example above, though there is a lot of
> configuration beforehand.

I feel that was designed backwards. Instead of having facilities so
applications can extend readline by adding their own line editing functions
specific to them, it should have facilities so users can add their own
extensions and make them applicable to all applications that use readline.
That would be cool.

In particular, I would like to be able to extend it to add better multi-line
editing support like what zsh's zle has.

In zle, I can use `o` and `O` to create lines below or above the cursor, `j`
and `k` to move to the next or previous line, `>>` to further indent the
current line, etc. It's basically a mini-version of a vi buffer, right in the
command line.

In readline, you can insert a newline in the current prompt with Ctrl-V
Ctrl-J, but to move between the lines, I have to move left or right beyond the
beginning or end of the current line.

Even worse, readline's history format doesn't support having a single entry
for multi-line commands, like zsh does. :(

~~~
zests
It might help to know that control+x+e will open the current command being
typed in $EDITOR.

~~~
cat199
or esc-esc-v if in vi mode..

------
aasasd
Wonders of Readline become painfully apparent when you have to use a program
that doesn't employ it despite having a line-by-line command input interface.
But! By some stroke of luck and computer magic, Readline can be injected right
into the program, at least in Unixes: there's an util called `with-readline`
for that.

Some programs have the editing features but lack command history for some
reason, or don't store it between sessions―this is also solved by the addition
of Readline.

~~~
unmole
> Readline can be injected right into the program, at least in Unixes: there's
> an util called `with-readline` for that.

Damn! I knew this 6 months ago. I ended up writing a Python wrapper to add
readline capabilities for a tool @work.

------
xvilka
We (radare2 project) recently added more readline-like features in our own
implementation called dietline. We cannot use the readline itself due to
portability issues but tried to implement the most useful features of it. See
the basic documentation in radare2 book[1]. And of course, any contribution[2]
to improve our implementation is welcome.

[1]
[https://radare.gitbooks.io/radare2book/content/basic_command...](https://radare.gitbooks.io/radare2book/content/basic_commands/dietline.html)

[2]
[https://github.com/radare/radare2/issues?utf8=%E2%9C%93&q=is...](https://github.com/radare/radare2/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+dietline)

~~~
saagarjha
What portability issues did you have?

~~~
xvilka
Radare2 is supposed to work on the target platform as a debugger and RE tool,
and platforms can be custom and quite limited. This is why it even provides a
basic UNIX commands in the shell.

------
egwynn
Oh man, it kinda hurts to see someone remapping `C-o` like this… That’s one of
my favorite readline bindings!

EDIT: From the bash man page:

> operate-and-get-next (C-o)

> Accept the current line for execution and fetch the next line relative to
> the current line from the history for editing. Any argument is ignored.

------
CJefferson
I have a love-hate relationship with Readline. While it is good once learnt,
it's really hard to learn.

For beginners it's not obvious there is anything to learn, and even if they do
want to know, you have to basically be told to Google for 'Readline'. It would
be great if it could somehow be made more discoverable.

~~~
gumby
Back when it was written Emacs was pretty popular (well, it had been the
default around the lab for about a decade). So users would use it without
noticing. The same reason the standard text widgets of Mac OS support Emacs
commands.

~~~
macintux
I recall my joy when I realized that the Netscape browser supported Emacs
shortcuts when working in a textarea.

------
westurner
Is this macro from the article dangerous because it doesn't quote the
argument?

    
    
      Control-j: "\C-a$(\C-e)"
    

I can never remember how expansion and variable substitution work in shells.

~~~
dan-robertson
The macro means:

    
    
      \C-a: beginning of line
      $: self-insert
      (: self-insert
      \C-e: end of line
      ): self-insert
    

So if your prompt (with | for cursor) looks like this:

    
    
      grep 'blah blah' |some_file
    

And you execute that macro, you get:

    
    
      $(grep 'blah blah' some_file)|
    

Which is correctly quoted and I think always correctly quotes whatever is at
the prompt (unless you e.g. get half way through a string and press enter so
the beginning of the prompt is halfway through a string, or maybe if you have
multiple lines)

~~~
westurner
Yeah, but this and this do different things:

    
    
      # prints a newline
      echo $(echo "-e a\nb")
    
      # prints "-e a\nb"
      echo "$(echo "-e a\nb")"

------
soyiuz
~/.inputrc

    
    
        set show-mode-in-prompt on
        set vi-cmd-mode-string "cmd:"
        set vi-ins-mode-string "ins:"

------
nineears
Ctrl-U doesn't delete everything before the cursor. It moves everything before
the cursor into the cut buffer.

~~~
brianjhanfox
"cut buffer" is actually "kill ring"

C-u only does that by default so that readline wouldn't confuse *nix users.

------
derekp7
Got a question, what is the standard workaround to the fact that Readline's
forward search (Crtl-S) is also XOFF (effectively pauses the terminal)? Other
than turning off software flow control via stty -- I'd like to keep that on.

~~~
Asooka
Ctrl-S is XON. Ctrl-Q is XOFF.

------
saagarjha
> He said that millions of people probably use Bash without realizing it
> (because every Apple device runs Bash)

iOS devices do not ship with bash.

~~~
chetramey
At least the last time I looked at a jailbroken iOS device, it opened a
terminal running bash-3.2 (the same version as on macOS). It's there, you just
can't get to it.

~~~
saagarjha
Most "packaged" jailbreaks drop a shell binary into /bin, or outright install
packages for things such as Bash and the GNU Core Utilities; iOS does not
actually ship with these. (If it did, I would assume that the GPL would
require that Apple provided source code for it here:
[https://opensource.apple.com/release/ios-1141.html](https://opensource.apple.com/release/ios-1141.html)
.)

~~~
chetramey
I don't buy it. First, why would a Linux-based jailbreak install a binary for
bash-3.2, when that version is going on 13 years old? Second, there's nothing
that requires Apple to make the GPL sources available in _both_ MacOS and iOS.
Doing it once is sufficient. I'm sticking with what I've seen and been told:
the guts of MacOS are there, just hidden.

~~~
saagarjha
> First, why would a Linux-based jailbreak install a binary for bash-3.2, when
> that version is going on 13 years old?

Because nobody updated coreutils on iOS for years, AFAIK. FWIW, the unc0ver
jailbreak installs bash 5.0.3.

> Second, there's nothing that requires Apple to make the GPL sources
> available in both MacOS and iOS. Doing it once is sufficient.

Why does the GPL not require that they release their modifications if they
distribute them?

> I'm sticking with what I've seen and been told: the guts of MacOS are there,
> just hidden.
    
    
      $ find -E /Volumes/PeaceG16G102.D321OS -regex "(ba)?sh"
      $ ls /Volumes/PeaceG16G102.D321OS/bin
      df ps
    

(That's the iOS 12.4.1 restore image for iPhone XS, if you're wondering.)

------
Koshkin
We used to have a simple line editor so we could edit a text file. Then GNU
came along and gave us a wondrously complex editor so we could edit a single
line...

From the man page:

    
    
      Bugs
      
      It's too big and too slow.
    

GNU software at its finest.

