
A look back at memory models in 16-bit MS-DOS - pjmlp
https://devblogs.microsoft.com/oldnewthing/20200728-00/?p=104012
======
WalterBright
> MS-DOS had an additional memory model known as Tiny, in which the code and
> data were all combined into a single 64KB segment. This was the memory model
> required by programs that ended with the .COM extension, and it existed for
> backward compatibility with CP/M.

The usual "tiny" memory model had all the segment registers ES DS SS CS set to
the same value. It was loaded by simply copying the program file into memory
and jumping to it.

At Zortech we realized that it wasn't necessary for all the segment registers
to be the same - it's just that the program file needed to be less than 64Kb.
This meant the code plus the statically initialized data had to be less than
64Kb. SS and DS, though, could be changed to the start of the initialized
data, and had access to 64Kb beyond that, enabling significantly larger
programs while still using near pointers.

~~~
russellbeattie
Heh, I was about to point to that same paragraph because of how disingenuous
the term "backwards compatibility" is. I'm sure the author wasn't
intentionally trying to revise history and probably didn't think about the
term much, but to set the record straight, DOS wasn't a _successor_ to CP/M,
but a "quick and dirty" clone, then later an actual competitor. COM files were
included in DOS to make sure programming for it was similar to programing for
CP/M.[1]

The story (which I'm sure many of you know) was that in 1979, Seattle Computer
was selling an 8086 board, but CP/M didn't run on that CPU yet. So Tim
Paterson solved the problem by whipping up QDOS to help sell more boards,
cloning the architecture and APIs of CP/M directly from the manuals (without
access to the actual source code).[2]

The simplest sort of executable code is a COM file (doesn't get much simpler,
actually), so if you're copying the way an OS works, that's pretty much where
you'll start. It was about being a familiar paradigm for programmers, not some
sort of cross-platform binary compatibility to an out of date OS as insinuated
by the term "backwards compatible".

1\.
[https://en.m.wikipedia.org/wiki/COM_file](https://en.m.wikipedia.org/wiki/COM_file)

2\.
[http://bitsavers.org/pdf/imsai/dos-a/05_CPM_Interface_Guide_...](http://bitsavers.org/pdf/imsai/dos-a/05_CPM_Interface_Guide_1976.pdf)

~~~
bawolff
Its totally reasonable to use back compat to mean "compatible with a legacy
system made by someone else that our system replaced"

~~~
toast0
Sure, but it wasn't that, it was "compatable with a contemporary system that
did not survive as long as our system"

~~~
boomlinde
Sure, but an earlier system nonetheless. Similarly, I'd describe Windows 95 as
backwards compatible with Windows 3, even if one might describe Windows 3.1 as
a contemporary system at the time of 95's release.

~~~
toast0
Was OS/2 backwards compatible with Windows?

------
WalterBright
At one point I added the "v" memory model for Zortech C++, which was a virtual
memory system for code. Functions would be loaded from disk on demand, and
discarded from memory when not needed. The loader would do all the fixups
necessary. I was pretty proud of it, it was far better than the usual
hierarchical overlay system.

But by then people were using 286 DOS extenders which were better.

~~~
bluedino
Is there a good article on how DOS overlays worked?

Back when I learned (?) C, I never learned about the x86/DOS memory models. I
just knew I was always hitting the 64k limit, and fortunately DJGPP existed at
the time so I just switched from Quick C/Turbo C to that.

~~~
WalterBright
Overlays were not part of the DOS operating system. But there were at least
three different methods:

1\. the traditional one (used by the PDP-11), where overlays were loaded into
fixed addresses. This was used by Microsoft tools.

2\. Borland's "Zoom" overlays, I forgot how that worked.

3\. Zortech's "virtual" system where overlays could be loaded into whatever
memory was available

~~~
skissane
> Overlays were not part of the DOS operating system.

Microsoft introduced overlay support in MS-DOS 2.x, via the "load overlay"
function – INT 21h,AH=4Bh,AL=03h. You can see the references in the MS-DOS
source code here: [https://github.com/microsoft/MS-
DOS/blob/master/v2.0/source/...](https://github.com/microsoft/MS-
DOS/blob/master/v2.0/source/EXEC.ASM#L16)

This was already known long before Microsoft released the MS-DOS 2.x source
code; Ralf Brown's Interrupt List documents it:

[http://www.delorie.com/djgpp/doc/rbinter/id/51/29.html](http://www.delorie.com/djgpp/doc/rbinter/id/51/29.html)

[http://www.delorie.com/djgpp/doc/rbinter/it/91/15.html](http://www.delorie.com/djgpp/doc/rbinter/it/91/15.html)

------
jchw
The near and far pointers bit is also the reason why in the Windows flavor of
Systems Hungarian notation, you have a lot of “LP” prefixes; I believe it
means Long Pointer and would give you a far pointer. It’s typical to have one
pointer typedef for Long Pointer, like LPBYTE, and sometimes one near, like
PIMAGE_DOS_HEADER. Of course now it’s all ignored so if you see someone
writing LP in modern code chances are it’s just tradition.

Also, today the segment-jumping far call still exists on modern AMD64
machines, and is how you cross “heaven's gate” to go between 32 bit and 64 bit
execution.

~~~
jcranmer
> Also, today the segment-jumpimg far call still exists on modern AMD64
> machines, and is how you cross “heaven's gate” to go between 32 bit and 64
> bit execution.

Addendum: this model only exists on Windows. On Unixes, programs are entirely
64-bit or entirely 32-bit, so there's no LDT setup to have a 32-bit segment in
64-bit code. And there's no syscall on x86-64 to let userspace muck with the
LDT as there is on x86-32.

~~~
jchw
Hm? I don’t think this is true. It’s CS 0x23 and 0x33 just like on Windows,
no? This is, afaik, how Wine WoW64 works. I cannot find a definitive source,
but for ex: [https://stackoverflow.com/questions/24113729/switch-
from-32b...](https://stackoverflow.com/questions/24113729/switch-from-32bit-
mode-to-64-bit-long-mode-on-64bit-linux)

I also think on Linux the x86 syscalls using software interrupts will still
work in a 64 bit process, too.

------
ajross
It's worth pointing out that, outside the IT industry, very little
"professional" software was actually written to use far data pointers as a
default type (a long function call, on the other hand, was only a tiny bit
slower than a near one -- this was pervasive). The overhead involved in doing
the segment dance for every pointer indirection was simply too high to be
practical.

Almost all real mode software made use of some home-grown framework for
managing large data across a segmented space.

------
userbinator
As someone who wrote 16-bit x86 Asm for a long time, if you are writing Asm by
hand, the "tiny" model is not as tiny as it may sound at first glance ---
65,536 bytes seems very far when you can write useful applications starting in
the dozens of bytes and going up to a few KB. A "Hello World" is around 21
bytes, half of which is the message itself. The majority of the executables
that came with MS-DOS 6.22 are under 64K as well, despite being written in C.

Then I look at the Win32 C equivalent with the default MSVC compiler options,
and it's over 64KB already...

~~~
jchw
FWIW: it’s trivial to make minimal binaries, just, nothing optimizes for it
for the most part. If you use OpenWatcom v2 with nodefaultlib you can get
win32 binaries that are around ~1500 bytes, which is not too shabby when you
consider its padded to 512 byte boundaries, and that the PE headers take at
least 300 bytes or so if you aren’t overlapping structures.

I have a project that implements a regular expression engine, JSON parser and
some game patching code into a DLL that is around 10 KiB (iirc) using
OpenWatcom. Of course it’s easy if you’re only targeting Windows and 32 bit
but you can do similar if you drop the standard library in MSVC.

~~~
wruza
Oh yes, "This program cannot be run in DOS mode". Decades have passed, but
Microsoft still has not learned how to make them run.

------
zackmorris
Weird I was just thinking about this last night, in terms of how a multicore
computer might work.

Imagine a chip with, say, 256 cores, arranged in an 8x8 grid. Each core would
have 64k locally, but could call out to other memory regions and retrieve them
automagically in less than 2*sqrt(N CPUs) memory clock cycles.

I'd like it to work like DOS, where the unsigned 16 bit pointers would access
the local memory of each chip. But also be able to write a program in 32 bit
mode where all of the cores can access the global memory space like in a
regular computer. A high-memory program like that might reserve the bottom 64k
for system code and low memory variables.

Even better, I'd like progressively more powerful versions of this computer,
starting with 4 or 8 bit cores, on up through 16 bit, 32 bit and even 64 bit.

Does anyone know a starting point for a memory architecture like this? Does it
already exist?

~~~
voldacar
Some architectures you might like reading about:

[https://en.wikipedia.org/wiki/Transputer](https://en.wikipedia.org/wiki/Transputer)

[https://en.wikipedia.org/wiki/Connection_Machine#CM_designs](https://en.wikipedia.org/wiki/Connection_Machine#CM_designs)

[http://www.greenarraychips.com/home/documents/index.html#arc...](http://www.greenarraychips.com/home/documents/index.html#architecture)

------
golem14
That's one of the reason Atari/Amiga fans were so happy with their 68000 CPUs.
It was so much simpler to program on.

------
rrauenza
I'm having flashbacks now of programming in Windows 3.1 and the 64k limit for
contiguous allocation. We rolled our own Vector template in C++ in order to
get around it, but it still made things a pain, like sorting.

And then there was the dreaded GPF... General Protection Fault.

------
mark-r
I always considered myself very lucky that I didn't start programming Windows
until it went 32-bit. I knew about near and far pointers but never had to
worry about them because they weren't relevant to anything I did.

------
wim
Ah this brings back memories of x86's Unreal Mode
[https://en.wikipedia.org/wiki/Unreal_mode](https://en.wikipedia.org/wiki/Unreal_mode)!

------
orionblastar
Not complete without memory maps and what the interrupts do:
[https://github.com/generalram/normtech](https://github.com/generalram/normtech)

------
dboreham
68k programmer at the time: blissfully unaware of all this.

------
amelius
The non-uniformity of GPU programming models sometimes makes me think of the
days of segmented memory ... The agonizing pain ...

------
OliverJones
One word: burgermaster.

