
The 640 K Barrier - mwcampbell
http://www.filfre.net/2017/04/the-640-k-barrier/
======
korethr
This article is bringing back memories. Not just of fiddling with CONFIG.SYS
and AUTOEXEC.BAT, but fiddling with the sound card too, as was mentioned in
passing. As mentioned in the article, I did indeed feel a sense of triumph
when I finally got Wing Commander: Privateer to run.

As EMS was a kludge that sucked up high memory for its mappings, I eventually
had a fancy CONFIG.SYS with different menu options for different memory
setups, depending on which game I wanted to play.

Good times.

~~~
spapas82
Couldn't agree more! I was like 10 years old in the 90's ( _before_ Windows 95
on 1995) when all good games run on DOS. I had became an expert on how to edit
config.sys and autoexec.bat to enable EMS or XMS or save a few KB off the 640
kb limit - some games needed more than 590 kb from base memory free even
though they were also using extended memory!

And of course you needed to know exactly your sound card configuration (IRQ
number and COM port) and your video driver capabilities and memory size (I
started with a hercules and moved over to a trident before getting a 3dfx
voodoo).

Yes, running a game then was much more difficult than double clicking an icon
but it was worth it. The games of that era were great and you could aquire
skills (persistance, attention to detail, thinking on finding solutions)
useful for the rest of your life - if you were a kid at least!

~~~
xenadu02
Sound cards had to have IRQ #, IO port base address, and often a DMA channel.

The IRQs were handled by the PIC controllers, usually two with the secondary
piggybacking on the main controller's IRQ #2. This was how the hardware could
signal to the software that it wanted attention. The sound driver needed to
know what interrupt to listen for or it would never reply to the sound card's
requests. It was setup this way because these were physical pins! If you look
at the ISA slot pinout there are physical traces for IRQ 3-7. 16-bit ISA slots
added pins for IRQ 10-14. A bunch of IRQs were permanently assigned from the
old IBM PC days which is why turning off COM ports in the BIOS would free up
the IRQ for use by other peripherals. I seem to remember having a card at one
point that was 8-bit, but had a little extension off the end with a couple of
traces for 16-bit slot support only so it could support higher IRQs; The
manual told you not to use IRQs > 9 if plugged into an 8-bit slot.

The IO port base address is directly related to the article: It selected where
in main memory the device was memory-mapped. Writes to that address would
cause the device to latch onto the bus and start receiving data. Anything not
in the appropriate range was ignored even though all devices technically
received the signal electrically. If the driver had the wrong address it would
either write to unmapped memory (not that there was an MMU so the data words
just went by on the bus without any listeners). Or worse: the write went to a
different device that might ignore what it considered malformed data, but in
many cases would do random things, scribble on memory, or just blast the bus
and cause a lockup. IO base addresses were pre-assigned to peripherals like
COM and LPT ports too but they were not in such high demand compared to IRQ or
DMA channels; still in some BIOSes you could change the COM port's base
address which would make it show up as COM3 instead of COM1 for example.

The DMA channel was exactly what it sounds like: the system had a limited
number of direct channels to the memory controller. Often the protocol for a
sound driver was to write a "Read audio from address XYZ, length X" to the IO
port. The sound card would use the DMA channel to read the audio from that
address. Then it would signal an interrupt when it finished so the driver
could submit the next batch of samples.

You usually configured ISA cards by setting jumpers on the card. The available
jumper settings were usually much narrower than the hardware supported so
conflicts were more common.

PCI still used the physical IRQ system but had a protocol for devices to
negotiate available settings with BIOS on startup.

PCI express uses a higher-level protocol abstraction. An interrupt is just a
special message on the bus and doesn't have any dedicated pins.

~~~
digi_owl
Some later ISA cards also supported "negotiation", iirc. It was marketed as
plug and play, but often referred to as plug and pray (because sometimes the
result would be less than useful). Also made DOS drivers downright massive!

~~~
korethr
I remember those 'Plug and Play' ISA cards. The PnP bit never worked, I
eventually hard set the IRQ, IO port and DMA channel directly to get sound
working, and that was that. At least, until the Windows95 install was messed
up enough that it was time to reinstall again.

------
mikestew
If one ever wondered why the original PCs were viewed as toys by professionals
in the field, this article is a good explanation as to why. Imagine you go
from, say, an IBM 370 mainframe which has had virtual memory, VMs, no-execute
flags, and considerably more than 640K back in the _60s_ to...this morass just
to use the memory you paid for (and pay you did, back then). Granted, PCs
didn't cost a couple of million with a maintenance agreement, but that didn't
make them any more useful for running your nightly batch jobs.

~~~
digi_owl
As i learn of the mainframes of yesterday it feels like all the hoopla of _nix
on x86 these days is basically rediscovering all those things mainframes did
back then...

~~~
astrodust
Hypervisors are all the rage today but that's so 1970s in the mainframe world.
Welcome to the party, kids.

~~~
digi_owl
Indeed. What I find most "puzzling" is the lack of acknowledgement of this in
the business. Its like it is a whole new, almost magical, thing that the devs
dreamed up.

I wonder if it has to do with how the microcomputer world had to almost
bootstrap itself without any input from the mainframe people, because the
latter considered the micros little more than toys.

And this continue into the present, as a large segment of the business is self
taught. And by now even going to university will not expose you to mainframes,
as they have largely been abandoned in favor of clusters.

~~~
smacktoward
It's at least as much do with the industry's youth fetish; ideas that were
common on mainframes are going to seem new and groundbreaking to you if you
never work with anyone old enough to have used a mainframe.

The tech industry has no mechanisms for developing institutional memory, so we
are constantly reinventing old wheels instead of developing new ones.

~~~
digi_owl
Good point. It is hard to convince a 30-something to accept the "death
sprint", compared to 20-somethings brought up LAN parties and red bull
binging.

~~~
scrame
No, 30-somethings had those too, and moutain dew and what-not. The differences
is that people in their 30s might have already spent their 20s doing "death
springs" and are sick of it.

------
lordnacho
The 640K thing precipitated the first program I ever wrote. If you can call it
that.

I had a bunch of games I wanted to play in the early/mid 1990s, including Wing
Commander. Each one seemed to require it's own config.sys/autoexec.bat, so I
wrote a menu type thing to choose the game and do the appropriate setup.

The experience made me think about what else I could do by joining a few
commands together. I wish the internet had been available to me then, because
I would have made much faster progress if something like stackoverflow had
existed.

~~~
AceJohnny2
> _because I would have made much faster progress if something like
> stackoverflow had existed._

Kids these days have it easy! /s

------
jsnell
It feels kind of embarrassing to admit that despite years of tweaking this
stuff to get games to run as a kid, I never actually bothered to find out what
the difference between EMS and XMS was. Or even know the basics of how they
both worked. "It's just extra memory, right?".

It's bizarre that the computer magazines I read (and obsessively reread) would
not have run an article like this.

~~~
joakleaf
As I recall: you could only access XMS memory by copying. So you had to _copy_
a memory block from above the 1MB barrier to the lower 1MB addressable space
to read, and copy back up again to write.

EMS was pageable. You could had up to four 16 KB pages below 1MB that were
fully accessible, but you had to select which address above 1MB each of those
four pages had to point to.

So in short, XMS was inefficient, because you had to copy, and EMS was more
efficient because you simply decided which part of the extended memory you
wanted to map to lower memory.

I think, EMS was emulated in real mode on the 80386 chip by using its virtual
8086 mode. This was not possible on the 80286, so the 80286 could only use XMS
in real mode.

I briefly developed using both EMS and XMS, and as I recall, XMS was somehow
less painful to get up and running and generally work with, but the
performance conscience part of me obviously favored EMS's paging mechanism.

Fortunately, 32-bit memory came along quickly both in the form of Windows and
DOS protected mode.

~~~
cameldrv
EMS was originally for the 286. It wasn't emulated by the CPU like you could
on the 386. You bought an add-in board that contained the extra memory and
handled memory requests on the bus for the bank switched area and accepted
commands to change what areas the banked switched areas pointed to.

When the 386 came out, it was common to have more than 1MB on the motherboard,
but the motherboard (typically?) didn't natively emulate the EMS bank
switching scheme, probably because at about that time, everyone figured that
people were going to stop using DOS and switch to OS/2, and be running 32 bit
code that could natively address higher than 1MB.

When that didn't happen right away, Quarterdeck came out with QEMM/386, which
used the Virtual 8086 features of the 386 to emulate EMS in software.
Microsoft then shipped a not quite as good clone of QEMM as EMM386 with DOS
5.0.

------
13of40
I attended a talk by Bill Gates several years back, where he stated that when
IBM was developing the PC, he tried to convince them to base it on the
MC68000, and that their decision to go with the 8088 set personal computing
back a decade. As the article points out, though, there were a ton of 68000
machines that ultimately got left in the dust by the PC, so I wonder how it
would have really turned out.

~~~
valuearb
Intel turned out way better than Motorola (and everyone else) at increasing
transistor density.

~~~
pm90
Well, maybe that was because they had a lot more money to throw at R&D? Not
saying either way, but it looks like this was the opportunity that really
pushed Intel into the big league. To their credit, they used it wisely and
kept making better chips.

~~~
astrodust
It's more like Intel had a CEO that understood what it took to win in the CPU
business and Motorola ended up with a CEO that printed out his email and
banned Macs from the company when they were one of the few licensees making
Apple Mac clones and Apple was one of their biggest buyers of CPUs.

------
polpo
This article is a bit inaccurate about XMS. XMS works just fine on the 286;
see the original spec [1]. This is done purely in real mode on the 286 with
the the A20 line [2] which allows addressing a 64kB area above 1024MB, in
combination with the the undocumented LOADALL instruction, which allows
addressing even beyond that [3].

[1]
[http://www.phatcode.net/res/219/files/xms20.txt](http://www.phatcode.net/res/219/files/xms20.txt)

[2]
[https://en.wikipedia.org/wiki/A20_line](https://en.wikipedia.org/wiki/A20_line)

[3]
[https://retrocomputing.stackexchange.com/questions/1006/what...](https://retrocomputing.stackexchange.com/questions/1006/what-
is-the-infamous-instruction-loadall-on-the-80286-and-what-was-it-used-
for/1011)

------
raarts
The article mentions confusion. That's an understatement. Information was much
harder to come by at the time, you had to rely on articles in computer
magazines. Also many bugs existed in memory drivers, not to mention
interoperability problems, due to programmers not understanding the issues.

This brings back memories, but not particularly good ones.

------
guelo
The original mistake was made by the IBM engineers that came up with that
memory map. They reserved a few bytes at the beginning and then a bunch of
bytes at the end. That really makes no sense. They could have avoided all the
pain that followed by putting all the reserved space in a contiguous block at
the beginning. Maybe they thought it was easier for application programmers to
use smaller address numbers? Or maybe it was due to some hardware shortcut,
which wouldn't be surprising considering they had already taken the shortcut
of using the 8088 instead of 8086. This kind of story is repeated whenever
temporary hacks turn into massive successes, for example Javascript's
limitations that still haunt us today. If you dig down into these histories
you almost always find that it's not the engineers' fault but shortsighted
managers.

~~~
zwieback
I thought it was because it's easier to decode a few upper address lines on
the bus so each peripheral is cheaper and easier to build. If you put the
hardware addresses in the lower address space you'd have to decode the entire
address to see if the access is on your card.

I think the real problem was that people didn't want to let go of DOS. OS/2
1.x was a fine OS, especially from 1.2 onward and it ran great on a 286.

~~~
Someone
_" If you put the hardware addresses in the lower address space you'd have to
decode the entire address to see if the access is on your card."_

That is incorrect. If your card has a block of 2^n bytes of memory, you need
to check the (in this case) 20-n uppercut bits of an address to check whether
it is in 'your' address range, whatever block your card is assigned to.
Checking whether that is all zeroes isn't easier than checking whether it is,
say, 42.

And I don't think people were attached to DOS; they were attached to their
(expensive!) programs and those required DOS.

OS/2 tried to support those programs, but wasn't perfect, and even if it,did,
it didn't give those programs more memory.

~~~
astrodust
The real flaw was not where they put it, but that it was immobile, the whole
block was "reserved".

If they made a few tweaks that allowed peripherals to slot into different
spots using some kind of negotiation over the ISA bus this could have been
avoided, but that was also way too sophisticated for what was a quick
prototype.

------
Zardoz84
I rememeber that I was very effective tunning the config.sys and autoexec.bat
to grab the max conventional ram. If my memory don't fail, I think that I
managed to get ~636-634KiB on my first PC (an AMD 386@40 with 4 MiB of RAM).

I usually, launch memmaker, and then fine-tune the config.sys & autoexec.bat .
country.sys ? doskey.sys ? ansi.sys ? Remove it! I need moar free conventional
ram !

~~~
nvahalik
That was my first thought—hmmm... memmaker.exe ought to take care of this.

------
cardiffspaceman
I designed a game engine for a few games, implemented it on Windows 3.1, and
assisted with its DOS implementation. The code running on this engine built
equally conveniently for Windows or for DOS due to the design of the engine.
The publisher wanted the games to launch and run decently if "mem/c" indicated
500K. This was what was typically available in a 1Meg machine if it had things
like Netware installed. We achieved this in part with commercial source-
licensed libraries and in part with Borland's VROOM overlay technology.

------
orbitingpluto
Microsoft Mouse (software release 9.00) from 1993 was a coveted driver back in
it's day. If I remember correctly I think it only took 2K of conventional
memory and the rest shunted into upper memory. After discovering that on a BBS
I rarely needed an alternative config.sys/autoexec.bat.

------
ThomaszKrueger
It should be noted that some level of multitasking was achieved with so-called
TSRs (terminate and stay resident programs). My preferred utility of course
was Sidekick.

------
psyc
For the first decade or so that I was programming, the 640k barrier was as
normal as air.

Now I do back-of-envelope calculations that tell me a single data structure is
100MB, and I'm like, "Yeah that's fine."

------
thelazydogsback
Should have some props in there for Topview, Desqview, and especially QNX...

~~~
zwieback
Props from me. I loved Desqview, it worked great. I didn't really use high
memory other than for Desqview and SmartDrv.

Once the 386 came around it was so much easier, Watcom C++ was my favorite way
to take advantage of my luxurious 16MB I could afford at the time.

~~~
thelazydogsback
Compiling helloworld.c with one floppy and no hard drive on the original PC
required one disk for each of the following: (1) source code, (2) cpp.exe, (3)
cc1.exe, (4) cc2.exe, (5) lib manager (can't remember the name - something
like "marion"?), (5) and the linker -- I think it was 8 floppy swaps total. It
blows my mind that my first HD was 5 _MB_ and cost $800. Also missing was the
programming side things: LocalAlloc vs. GlobalAlloc, low-mem HWNDs, etc. - ug
- don't miss that...

~~~
13of40
I realized yesterday that for years I've been mentally measuring data by how
many Amiga 3.5 inch disks it would take to store. Noticed this when I was
patting myself on the back for getting the build output of the tool I'm
working on down below ~60 disks.

------
dalf
A lot of memories pop into my mind with this article including unrelated stuff
as the first try of Linux, kernel 1.2.8, and the "how to setup the CDROM
reader attached to soundcard ?" question.

To go back to the subject, I remember to have used a "driver" to free up some
additional memory, but I can't remember the name and how it worked. May be I
found it reading Imphobia ? The name may starts with a R.

There was the unreal mode / flat mode too :
[https://en.wikipedia.org/wiki/Unreal_mode](https://en.wikipedia.org/wiki/Unreal_mode)

I've remember to have code a memory allocator for Turbo Pascal. With pure DOS
(not Windows), it was easy to use after the setup.

------
thought_alarm
Great read.

Are there technical reasons why Microsoft didn't release a Protected Mode
version of MS-DOS in the 80s, or was it just a case of trying to push users
and developers to OS/2 and Windows?

It seems like such a no-brainer, especially since MS-DOS provided so few
services to applications anyway. You have MS-DOS 8 and MS-DOS 16, and you let
the user choose which version to boot into, or you automatically kick over to
MS-DOS 16 when a Protected Mode app is launched. It seems a lot saner than
forcing users to muck around in config.sys.

That people were forced to deal with the memory limitations of the 8088 PC
well into the 90s is just bananas.

~~~
stickfigure
It's not quite that simple. DOS is not like what we think of as an OS today;
more like some libraries that sit in RAM at known addresses plus a loader
barely smart enough to transfer data from disk to RAM and start execution. All
the intelligence is in the application program itself. And those programs were
still stuck with 20-bit pointers.

It wasn't just a question of rewriting the OS, it was also a question of
rewriting all the application software. Which eventually happened.

~~~
digi_owl
And why I half-jokingly consider unikernels "DOS in a can".

~~~
throwaway91111
Nothing bars unikernels from running their code in protected memory as the
sole non-ring-0 proc.

------
ajnin
Very interesting article. I hadn't realized that Microsoft strategy of
developing their market not by the merits of they product but by lock-in to
use business programs (and also games and such) went back so far.

Also it's great to finally learn what "UMB" stands for in "DOS=HIGH,UMB".
Never felt right to add that option without knowing its meaning :-).

------
supernintendo
> "640 K ought to be enough for anybody."

Bill Gates never actually said this [1]. There are those who might attribute
this to the Mandela Effect.

[1] [https://www.wired.com/1997/01/did-gates-really-
say-640k-is-e...](https://www.wired.com/1997/01/did-gates-really-say-640k-is-
enough-for-anyone/)

------
Aardwolf
Awesome article! Best read in a while. I wish it would continue to the state
of things the next years, with DOS4GW, and Windows 95 and 98 still running on
top of DOS. The article stops too early :)

------
zwieback
Shout out to Windows Real Mode, which had a built-in virtual memory manager
that swapped out unused memory to disk. All of that required apps to
participate, of course, but it could multi-task in the lower 640K.

~~~
LeoPanthera
Astonishingly, Real Mode still exists even on 64-bit Intel chips, and is still
the default mode. Booting a modern OS involves a whole stack of bootstraps to
jump from Real Mode to Protected Mode to Long Mode.

It's duct tape all the way down.

~~~
JdeBP
Actually, real mode hasn't been the default mode for a long while.

* [https://superuser.com/a/347115/38062](https://superuser.com/a/347115/38062)

* [https://superuser.com/a/695716/38062](https://superuser.com/a/695716/38062)

* [https://superuser.com/a/345333/38062](https://superuser.com/a/345333/38062)

~~~
LeoPanthera
I stand corrected, thanks.

------
magoon
So much time spent optimizing my system to provide as much "conventional
memory" so I could run a performant BBS in 640K, with all the drivers and
such. It's hard to imagine now.

------
avodonosov
Intel could have allowed more memory in the real mode, instead of providing
that only in protected mode incompatible with the existing software ecosystem.

------
yuhong
This reminds me of the OS/2 2.0 fiasco, which is so bad it is one of my
favorite topics.

------
floatboth
I'm really surprised that the DOS extenders weren't the first solution to
appear…

~~~
mwcampbell
If I understand the article correctly, DOS extenders weren't feasible until
the 386, because the 286 couldn't switch out of protected mode and back to
real mode except by doing a full reset.

~~~
WalterBright
There were effective 286 DOS Extenders. Zortech included the Rational Systems
286 DOS Extender with the Zortech C++ compiler.

~~~
bni
After reading the article, I am really curious how these could work.

~~~
cardiffspaceman
We're talking protected mode. The segment register would be an index into a
table of segments of memory and the segments would be implemented in your RAM,
and all of them would be accessible "at once" modulo the fact that there were
only a few segment registers. The DOS extender could hack DOS so that you
could in some fashion call DOS through "INT 21h" and it would do something
(emulation, call through glue code into real mode etc). The call might require
a transition into and out of real mode, which would not be cheap. Sometimes
the only way to call DOS was to use "low" memory that was directly accessible
from real mode, and you'd need to use special routines instead of malloc/free
to manage that memory. You'd still have to call DOS through a hack but the
memory arguments of the call would not need to be rearranged within the
extender.

------
orionblastar
I used to work as a technician getting DOS/Windows 3.X and OS/2 2.X systems
working. There was a program I used to identify expansion cards and get the ID
number off of them. It could detect memory size as well, but that doesn't work
in modern systems anymore.

[https://sourceforge.net/projects/gconf/](https://sourceforge.net/projects/gconf/)

Had a VP with a WANG PC that used Microchannel, he needed Netware, AS/400, and
the Wal-Mart client going at the same time and not cut into his 640K of DOS
memory.

There was a trick to it. First using card IDs I found the option floppies for
his expansion cards from CompUServe (As The Internet/WWW was in infancy and
our employer paid for CompUServe to research things) and I re-arranged the
expansion card memory to leave a 64K area that was not used that DOS could
load drivers into it to free up the main 640K memory. At the time Netware and
AS/400 used DOS drivers, while OS/2 had OS/2 drivers that could access the
memory beyond it. This was before Windows 95 (It was in beta test at the
time).

I had upgraded the VP to MS-DOS 6.22 used loadhigh and memmaker and other
things and managed to get all three networks working with most of the 640K
area free. No other tech had done things like that before.

I studied programming in college, but this gconf tool I wrote had helped me
out a lot. I wrote it in college and then modified it to be named whichnet so
that it could read Arcnet or Starnet network cards and use the correct Netware
driver for DOS on a boot floppy disk. Either by using the expansion card ID to
detect the right card or by reading ROM memory and look for patterns or a
signature. It had worked 99% of the time, and in the 1% it had failed and I
never learned why, but worked af\ter a reboot or shutdown and power up.

I wrote this HexStrip or HStrip to extract ASCII data out of Word Perfect or
other files that got corrupt and could no longer be read. It is also good for
scanning EXE and COM files in DOS to see if there is a virus that stores ASCII
text when it infects without running the EXE or COM. I used it in Tradewars
2002 with the trader.dat to pull out all of his advice for playing the game in
it.
[https://sourceforge.net/projects/hstrip/](https://sourceforge.net/projects/hstrip/)

I had bought QEMM which helped as well, until the Windows 9X and Windows NT OS
models put them out of business and Symantec bought out Quarterdeck and other
companies.

Also, I think B000:0000 to B700:0000 was the monochrome memory that could be
used to load things into it as well if you never used it. I used to write DOS
memory maps, until DOS came with MSD.exe that could show used and unused
memory maps. Intel 8086/8088 had 640K of RAM, and 384K of reserved memory,
that some of it can be used by EMS for paging to load things into it to free
up the 640K.

~~~
Zardoz84
memmaker (really himen.sys if my memory don't fail me) had an option to use
the monochrome memory

