
How Unix erases things when you type a backspace while entering text - zdw
https://utcc.utoronto.ca/~cks/space/blog/unix/HowUnixBackspaces
======
ChuckMcM
What is perhaps more interesting is the evolution and thinking about how it is
done so easily on modern systems.

I think I've rewritten the basic 'cooked' mode probably a few times the latest
being for a simulation of it on embedded systems[1]. The basic code is easy
and it fails on the interesting edge cases (the most common one being if you
auto CR/LF at the end of the screen few readline functions back up to the
previous line.)

These days however if you have "lots" of memory and rendering a screen takes
less time than rendering a single character did back in the day. You can just
delete the character out of the buffer and re-render the entire screen. No
need to count, your screen can do funky things like set your prompt different
colors have tabs at an arbitrary number of spaces etc. Basically you can start
at the top and render the entire 40 - 50 lines of text, poof in less an few
mS. That was unthinkable when even at 9600 baud it took a couple of seconds to
re-render the enter screen.

[1]
[https://github.com/ChuckM/stm32f469i/blob/master/demos/util/...](https://github.com/ChuckM/stm32f469i/blob/master/demos/util/console.c#L158)

~~~
abecedarius
There was a Byte article around 1982 presenting a text editor in Z80 assembler
that worked just that way: rerender the view on every keystroke. I was
impressed -- "wow, it's that simple?" Of course, this was with memory-mapped
textmode display, not a serial terminal, but it felt audacious to me to
'waste' that much computing for simplicity.

The article included the source code of the whole editor. That much assembly
code in a magazine was unusual even then.

~~~
userbinator
A lot of early text editors, particularly the tiny single-segment DOS ones,
worked on the same principle. They didn't exhibit any flicker (most of the
time --- anyone remember "CGA snow"?) because writing the same values to VRAM
doesn't affect the screen display.

~~~
abecedarius
This one might've used a vertical-blank interrupt to time the refreshes --
it's what I think I remember, but I could easily be mixing it up with some
other article around then, like Chris Crawford's series on Atari 800 graphics.

------
mkl
This is interesting, because it reveals the reason Windows Services for Linux
gets some terminal things wrong - it's just emulating a Linux kernel, and they
haven't done this bit the same. Running

    
    
      cat > /dev/null
    

then pressing tab then backspace works properly on Linux (including SSH from a
terminal on Windows), but doesn't work on WSL (or Cygwin).

~~~
0x0
In related news, Windows (NT/2000/XP/2003) had a BSOD crasher in terminal
tab+backspace handling for decades.
[https://www.owasp.org/index.php/The_CSRSS_Backspace_Bug_stil...](https://www.owasp.org/index.php/The_CSRSS_Backspace_Bug_still_works_in_windows_2003_sp1)

~~~
JdeBP
The original article still exists, at [http://jdebp.eu./FGA/csrss-backspace-
bug.html](http://jdebp.eu./FGA/csrss-backspace-bug.html)

------
Jaruzel
You know when you are getting _old_ when you read:

> _When you hit backspace, the kernel tty line discipline rubs out your
> previous character by printing (in the simple case) Ctrl-H, a space, and
> then another Ctrl-H._

And you think 'Yup, I knew that.' ...

(In my case, I just about recall re-implementing it when mucking about with
serial consoles in my youth - On low baud connections, you could actually see
it do it - backspace - space - backspace )

~~~
kps

      > printing (in the simple case) Ctrl-H, a space, and then another Ctrl-H.
    

For the _really_ simple case, run 'stty -echoe'.

(And perhaps run something like 'cat', since most shells run in raw mode and
do their own editing by default.)

------
userbinator
The whole TTY layer actually contains a surprising amount of functionality,
handling line editing being only one of them; this is one of the best articles
I've seen about the rest of that:
[http://www.linusakesson.net/programming/tty/](http://www.linusakesson.net/programming/tty/)
(It's also been mentioned multiple times on HN.)

------
basicplus2
When I first started programming a pdpd11 using basicplus2 I wrote a program
that had a built in password which consisted of the password which included
backspaces and *'s which obliterated the first half of the printed password so
when it was printed out you could not read the password.

Primitive but effective.

Even when viewing it on the screen it was quick enough you could not read the
password

~~~
corysama
Way back on Solaris I once managed to accidentally name a file backspace.
Somehow I managed to finger-flub and hit "mv filename <crtl-h> <enter>" or
something like that. Had to use a GUI interface to delete that file.

~~~
vram22
Not at a Unix machine right, now, but this might work:

rm \BKSP where the BKSP represents the backspace key. If it worked, it did
because the backslash escapes the usual meaning of the backspace key, which is
to erase the preceding character.

Also, related (though may not work for above case), to delete other files with
funny characters in the name (like a leading dash):

rm -- -filename

where the -- means (end of the options), so even a leading dash after -- is
not treated it as an option.

Best to use the -i option too, with such cases, for interactive confirmation
of the remove.

~~~
aap_
You don't even need a special option, just do rm ./-filename

~~~
vram22
Good one.

------
amelius
This seems complicated and error prone. Although I can see the advantages of
this approach, it really makes me wonder whether the designers chose the right
abstractions.

Also, the fact that I can type "cat binary-file" and mess up my terminal
settings tells me that they didn't clearly separated concerns.

~~~
throwawayish
This is legacy built on legacy built to work with minimal CPU cycles on
existing hardware designed to be compatible with hardware from the 40s.

Yes, concerns aren't separated ;)

A much more recent and far superior design can be found in Windows NT, where
things like coloring are done entirely through an ioctl-equivalent (whereas
Unix has amounts of in-band signaling, while some later extension use
tty_ioctl).

~~~
kps

      > Unix has amounts of in-band signaling
    

The in-band signalling is not a Unix thing; it's a widely-supported
international standard ISO 6429 aka ANSI X3.64 aka ECMA-48.

------
rasjani
I didn't read the article and all the comments but this is the sort of thing
once again reminds that there's really nothing new in most of the new
technologies :) Lets take Dom updates for example. Do you update the whole
tree and rerender the whole screen or only the changed parts. Problem is
essentially the "same" albeit tech and corner cases have their own set of
differences. Next, consider that maybe your browser is just more fancier
terminal and webserver is just evolution of old mainframes.

Well, that's just my way of thinking but because way of thinking and having
started my career writing unix applications, I still can maintain my interest
in latest 'trends' and at least feel that I have something to contribute when
trend seems to gear towards hiring younger and younger devs.

------
chillaxtian
i'm confused as to why the article implies the kernel is serving the shell.

isn't the shell usually a program running in userspace? that interprets
commands, and prints characters on the screen, etc, in userspace? i didn't
think the kernel was involved at all, except to receive exec syscalls.

~~~
DSMan195276
You're conflating the shell and the terminal. The shell is a running process,
the terminal (or tty) is essentially an input-output device connected to the
kernel. If you want to see what I mean, open a terminal and run the `tty`
utility. It will print the device file for your current terminal (Probably a
`/dev/pts/ _` entry). If you write any data to that 'file', it will appear in
your terminal, and if you read from that file you'll get completely unfiltered
input from the terminal (Unfiltered as in you simply see the keys as they are
pressed. If you backspace, you'll see a backspace character - no line
editing).

The kernel acts as the gatekeeper between the input/output of the processes
running on a tty, and the input/output of the tty device itself. In 'raw'
mode, the kernel simply takes the input from the tty and sends it as input to
the running process, and then takes the output from the running processes and
sends it as output to the tty device. The kernel still isn't completely
pointless in this instance though, as it does provide some other facilities.

But it doesn't _have* to do that. In 'cooked' mode (The default mode), the
kernel will allow line-editing. It does this by buffering the inputs from the
tty device (And writing them back to the tty device, so they appear on the
screen) without actually sending them to the running processes until you press
return.

The result is the programs like `cat`[1] which do nothing more then call
`read()`, and send the buffer to `write()` still allow line editing features
like backspace - even though the program itself has no concept of line editing
(or even lines at all).

It should be clarified that modern `bash` does operate in raw mode, and
handles all the line editing itself (The `readline` library also does this - I
can't say for certain if `bash` uses `readline` internally or not, I haven't
checked). Unix utilities like `cat` though don't change tty settings and thus
are generally operating in `cooked` mode (You can change these settings via
`stty`, but it's generally not necessary).

[1]
[https://github.com/DSMan195276/protura/blob/master/disk/util...](https://github.com/DSMan195276/protura/blob/master/disk/utils/coreutils/cat/cat.c#L89)

~~~
WildUtah
But try remapping C-w in bash. The tty remaps it on the very next line. You'll
have to stty werase unmap it to be able to change it in your read line
.inputrc.

So the tty cooking still has charge of some things.

~~~
DSMan195276
I think that's actually just bash messing with you. Bash implements it's own
erase-word and binds it to whatever is set to werase in the terminal, so if
you don't clear the werase entry bash will just keep rebinding it every new
prompt. It's definitely not the kernel's doing - for example vim, which also
uses raw mode, can use C-w just fine, and the functionality bash gives to C-w
is different from what the kernel provides.

------
rocky1138
But what if I want a string that includes the backspace character?

~~~
macns
Honestly, why would you want that?

~~~
rocky1138
I don't :) I was just asking a stupid question.

