
Live coding a vi for CP/M from scratch - ingve
http://cowlark.com/2019-06-28-cpm-vi/
======
david-given
Hello, author here.

Eeeaggh. Yes, this took a while. I started after lunch and finished sometime
after midnight (look at the clock in the bottom right hand corner of the
screen). I'm still kicking myself at some of the dead ends I went through.

The result (even after I cleaned it up after the stream) is still a bit slow.
The biggest issue is that it redraws the entire line every time you press a
key in insert mode; fixing this is moderately straightforward, as I have all
the information to know where the first modified character is, but it'll
require some refactoring.

The biggest think I want next is a yank buffer, but the only realistic option
is to store it on disk, which would be painfully slow. Being able to do
repeated inserts would be nice, too.

I actually know Bram Moolenaar; maybe I should ask how he did it...

~~~
kragen
Bravo! You should be super proud! On a 9600 baud terminal, a whole line is 77
milliseconds, so it's going to feel a bit slow even if you only redraw half
the line, and at 2400 baud it would be painful. Insert-character escape
sequences are going to make a big difference.

I’m no expert, but I think the Kaypro doesn't have a real serial port between
the terminal and the computer the way a lot of CP/M machines did (Altair and
IMSAI usually, H-8 and H-88 and H-89 always, while the Osborne used a memory-
mapped screen like the Apple). I don't know how fast it can repaint the
screen.

I don't think firing the loud solenoids on the floppy drive and waiting a
second or two every time you delete something (or (p)ut it) is going to be a
good user experience; the yank buffer probably needs to be in RAM to be
usable. You could maybe prompt the user whether to discard the deletion if the
text being deleted is over a threshold of size.

~~~
rbanffy
Altairs and IMSAIs didn't have standard terminals. You could add an S-100
video board with video output and keyboard input (usually something that would
output ASCII) or you could attach a real terminal over a serial line on an
S-100 board (which is kind of nice if you want to be far from the computer but
not that useful for a desktop). Many CP/M desktops used the "bus attached"
terminal (because it saves components). A common arrangement (and not only for
CP/M machines) would be a CRT controller managing its own memory where the
program/OS would address the video memory through the microprocessor's IO
channels. This was slow compared to having video driven from main memory (like
the Apple II) but CRTCs did a lot the hardware designer wouldn't need to worry
about, it was many orders of magnitude faster than a 9600 bps serial terminal,
and allowed moving terminal emulation to software/BIOS.

The Apple II had a couple "terminal" boards more or less like what I described
that allowed it to display 80 and 132-column video. Their memory and registers
would be manipulated through memory access as the 6502 didn't have dedicated
IO functions. One very common was the Videx Videoterm, that used a 6845 just
like the IBM CGA and MDA boards (but with a much nicer VT-100-like font). The
Microsoft CP/M SoftCard emulates a SOROQ IQ-120 terminal in software.

~~~
ncmncm
Thank you, I worked on the Videx gadget. We used to put the font ROM in VT52
terminals because it looked so much better than what they had. (These were
also sold as Zenith and even Heathkit brands.)

~~~
kragen
What Heathkit model was a rebadged VT52? The H9 was crudely similar to a VT50
but in many ways less capable (it didn't even have a character like the
AMD-3A’s ^K to move the cursor up, as far as I can tell), and though its
successor the H19 supported VT52 sequences, it also supported extensions like
ESC E and also an ANSI mode. Or do you mean Heathkit made a Videx-based
terminal (presumably not the H9)?

~~~
ncmncm
It's been a long time, but H19 rings a bell. It feels like it was the same as
the Zenith terminal I used with an LSI-11 running BSD 2.8 Unix (which was Vax
Unix implausibly backported; imagine virtual memory with address space for
only two swappable 8k pages!), and only emulated the VT-52.

Anyway the bit layout in the 2708 EPROM it used was identical to the Videx
card's. I think the fonts we used were a late development, and that for most
of its life the Videx font ROM was as bad as the original terminals'. We came
to realize that the verticals really needed to be two pixels wide.

Probably if we had the good sense to use an emitter follower output stage, we
could have got good sharp verticals.

I need to make clear that the only connections between the terminal and card
were that both used a 6845 display driver and a 2708 character ROM.

------
kristianp
This guy has some incredible hacks, such as the mandelbrot for BBC Micro (in
6502 assembly) that uses a lookup table for squares of numbers, and takes
advantage of the connectedness of the colours:

"It turns out that Mandelbrot and Julia sets have the property that any area
which is completely bounded by a colour contains only that colour. (Because
it’s simply connected.) (No, I don’t understand a word of that paper.) So the
program recursively traces out the bounds of boxes, and if all sides of the
box are the same colour it fills it and doesn’t recurse in."

[http://cowlark.com/2018-05-26-bogomandel/index.html](http://cowlark.com/2018-05-26-bogomandel/index.html)

Also a bootloader for the GPU of the rasperry Pi:
[http://cowlark.com/piface/](http://cowlark.com/piface/)

~~~
david-given
Thanks very much! But someone else worked out the maths and the base kernel
--- I just did the implementation (and loads of other people chimed in on
microoptimisations). I _do_ think it may be the fastest 8-bit Mandelbrot in
the world, although some of the stuff the C64 people have done is completely
insane, so I'd want to check there first.

This is all mine, though: [http://cowlark.com/2013-10-19-insane-
make](http://cowlark.com/2013-10-19-insane-make)

------
userbinator
The source is very straightforward and readable. I suspect part of the reason
why is because live-coding forces you to eschew additional
complexity/abstraction in favour of the simplest solution.

That said, I'm not sure if the additional complexity of a gap buffer is worth
it --- a lot of editors in the early days of DOS used a single buffer,
effectively memmove()'ing the tail part when you inserted characters, and
there was no real noticeable lag even when entering characters at the
beginning of a "large" (i.e. >100KB) file. On a CP/M system with 64K of total
address space at most, it just doesn't take all that long to go through all of
memory. As an advantage, cursor movement code is greatly simplified and
doesn't involve modifying the buffers.

~~~
kragen
I don't know the 8080 instruction set, but the memcpy in
[https://en.m.wikipedia.org/wiki/Intel_8080#Example_code](https://en.m.wikipedia.org/wiki/Intel_8080#Example_code)
has 7 instructions in its inner loop and copies 2 bytes per iteration, which
is probably about 20 microseconds at 2MHz and thus 640 milliseconds to copy
64K. That's a really long time to handle a keystroke, since they may be coming
in every 100 ms or less. You can probably unroll the loop productively to some
extent but you probably can't speed it up by the necessary order of magnitude.

Gap buffers, by contrast, should keep everything under a millisecond except
search and moving the gap a long distance.

Many gap-buffer editors don't move the gap whenever the cursor moves; they
wait until you modify something. But that does seem like it would be more
complicated. I'm not sure how much simpler it can get than the gap-buffer
case.

~~~
tom_
Even a Z80 would struggle to keep up: you're looking at something like 17-18
T-states/byte for a useful memcpy, which is ~215 KBytes/sec at 4 MHz.

Suppose your document is 32 KBytes, and you're inserting chars at the start.
It's going to take 0.2 seconds per keypress! You'd notice.

(0.2 seconds per keypress = 300 keypresses per second. Assume 6 keypresses per
word (5 letters and a space), and that's 50 wpm. You probably type faster than
that.)

~~~
Someone
_”Suppose your document is 32 KBytes, and you 're inserting chars at the
start.”_

Who has such enormous documents, and who inserts chars at the start :-) ?

Given the 64kB limit for (program+data), you’ll soon need some way to keep
things on disk (most likely by having a master document that lists the names
of the files storing each ‘chapter’ in the complete document, and requiring
the user to manually switch from file to file)

Therefore, I would implement that, search/replace and a yank buffer before I
started worrying about such relatively edgy performance issues.

For comparison, the original MacWrite on a 128k Macintosh handled only 2 to 3
pages of text, and also soon got slower than that 0.2s per key press. It was
smart enough to skip updates if typing was relatively fast, though. _that_ is
a feature you might want to add (e.g. by peeking into the keyboard buffer
before refreshing the screen), as the best way to speed up code is by doing
less.

~~~
kragen
These are good points, but a thing to keep in mind is that we were discussing
how to update the internal buffer data structure, not refreshing the screen.
However, you could imagine peeking into the keyboard buffer so as to do a
batch-insert if several keystrokes happened while you were doing the
memmove(), so that the next memmove() would insert two or three bytes instead
of just one — then at least it wouldn't progressively fall further behind your
keystrokes.

All else being equal, though, I think a source-code editor that can
comfortably handle a 48-kibibyte source file is considerably superior to one
that can only handle, say, 10 or 20 kilobytes before it starts getting
sluggish. Searching works better, refactoring is easier, you need less header
files in C, and you can split and join source files a lot less often. Note
that qe.c itself is (barely) over 20 kilobytes, and that's using TAB
characters for compression.

------
ohazi
Nine straight hours. Holy crap. I'm sure we've all had marathon sessions from
time to time, but maintaining enough composure to keep the livestream going
for that long (without breaks?) sounds extremely challenging.

~~~
kazinator
> _Nine straight hours. Holy crap._

Day in the life of a full-time software dev.

~~~
closeparen
Six of those hours (on a good day) would be spent in meetings, answering
emails, helping interns, giving code reviews for colleagues, writing
documents, reviewing documents, getting lunch/snacks/coffee, waiting in line
for the bathroom or elevator, and attempting to recover focus when you return
to your desk after any of the above.

~~~
kazinator
Then there are days when you really do code.

~~~
david-given
Those are _good_ days.

------
dancek
I don't like watching videos, but the list of milestones and timestamps is
great. Text editors are one of those things that would be nice to know how to
implement, but that never was high enough on my priority list. Looking at
this, I feel like I've significantly overestimated how hard it is to write the
UI layer. Granted, I'm going to be maybe much slower, but still this seems
like a doable project even if I don't have that much time.

Why don't we have a list of nice hacker projects somewhere? With an estimate
of skill and effort required, relative to other projects?

------
snazz
Recording yourself for YouTube has to help, but I’m really interested in what
the author used to stay focused and productive for an entire nine hours. I can
barely make it a half hour before getting distracted by something.

~~~
telotortium
He did take a dinner and a tea break, so he wasn't working completely straight

~~~
david-given
There are a couple of short (a few seconds only) edits where I just zoned out
thinking about something, but the big breaks are clearly labelled. Also, check
out the clock at the bottom right corner!

I started this at about 14:00 and finished at about 01:30, so about eleven to
twelve hours wall clock time. And yes, if I hadn't been doing the recording
I'd have given up and gone to bed. Also I, er, had to get up at 07:00 the next
morning to go hiking.

------
ladybro
Hey, I wrapped + annotated the video to make your timestamps clickable:
[https://use.mindstamp.io/video/QUqevzIz?notes=1](https://use.mindstamp.io/video/QUqevzIz?notes=1)

------
UncleSlacky
Probably a noob question, but why not just compile vi on a CP/M C compiler?
After a quick search, I found this being discussed on comp.os.cpm just last
year:
[https://groups.google.com/forum/#!topic/comp.os.cpm/lfuEr0hR...](https://groups.google.com/forum/#!topic/comp.os.cpm/lfuEr0hRcrM)

It's hosted here: [https://github.com/udo-munk/s](https://github.com/udo-
munk/s)

~~~
david-given
There are plenty of CP/M C compilers (I'm planning on including BDS C into
cpmish at some point:
[https://en.wikipedia.org/wiki/BDS_C](https://en.wikipedia.org/wiki/BDS_C)
It's written in hand-tooled assembly!) but with the exception of Hitech C
they're all Small C or K&R C. Hitech C works fine, but it's extremely slow and
somewhat quirky. CP/M's really a bit small for an ANSI C compiler. (Also
Hitech C only targets Z80, not 8080.)

The main reasons for doing this cross-compiled were: that's what I had; life
was too short; I wanted something for the cpmish cross-compilation environment
_anyway_.

Making cpmish self-hosted would be awesome, though.

Hitech C used to be a free-as-in-beer download from htsoft's website, but it
vanished about twelve years ago. I wonder if they'd be open to open sourcing
it...

~~~
Zardoz84
HiTech not was bought by Microchip ?

~~~
david-given
So they were. That explains why the only compilers on their web site are
Microchip ones... oh well, worth an email to them asking what happened; maybe
they'd be willing to open source the CP/M compiler!

------
cschep
This type of content is why I come to Hacker News, brilliant!

------
slater
Speaking of live coding, does anyone remember a certain longhaired Russian guy
(I think he works/worked for Yandex?) who used to do live-coding streams?

~~~
protoloc
Maybe you are referring to this guy?
[https://www.youtube.com/channel/UCJAVLOqT6Mgn_YD5lAxxkUA](https://www.youtube.com/channel/UCJAVLOqT6Mgn_YD5lAxxkUA)

~~~
slater
Yep, that was the guy! Seems to have stopped, alas.

------
thanatropism
Hi,

So — why not Wordgrinder?

------
fsiefken
But Why... I know the answer and the site gives a good explanation. But still,
practically speaking? Who cares about CP/M, perhaps because it's the only
useful OS running on 8 bit processors? Does CP/M have something DOS doesn't
have? Or is it impossible to run DOS on the Z80?

~~~
soneil
I think that is a large part of it, yes. There's been a good few kit computers
in the last few years - the rc2014 being my favourite (not because it's
particularly amazing, but because it's designed to be understandable and
digestable), and cp/m is the most approachable (roughly dos-like) OS available
for them. (There's also Fuzix, which seems like a monster accomplishment in
itself - but I haven't tried it myself. I rather suspect that running a unix-
like on an 8bit just ends up feeling like a very slow unix, rather than
anything special)

(and no, msdos doesn't run on most 8bits. The Intel 8080 was 8bit. The Z80
expanded on that with additional instructions, and cp/m lives in this branch
of the family tree. The 8086 gained 16bit instructions, and branches off in an
entirely different direction birthing the x86 family tree, which is where
you'll find msdos & most variants)

~~~
david-given
Fuzix is amazing. I did a port to the MSP430, a Texas Instruments single-board
computer with 66kB of RAM; it feels just like a Unix, despite having no MMU
and no multitasking...
[http://cowlark.com/2015-10-27-fuzix](http://cowlark.com/2015-10-27-fuzix)

I find CP/M interesting because it's provably the minimum viable product for
an operating system. It was the personal computer desktop operating system
that really worked. It came with full documentation, a full development
systems (an assembler and a debugger), and a full set of tooling to allow you
to port CP/M to new systems. It was portable, too, and you could run the same
program unmodified on nearly any hardware which supported CP/M. So it was this
great open-ended system, cheap enough to get into the hands of ordinary
people, simple enough that you could understand it all, and yet sophisticated
enough that, with enough time, you could achieve _anything_ on it...

The CP/M kernel is 3.5kB of 8080 machine code. I think each frame of that
video consumed more space than that.

