
Learn to write a simple OS kernel with keyboard/screen support (2014) - razerblader7
https://github.com/arjun024/mkeykernel
======
anyfoo
Like many people, I have started toy OSes, and like most such projects, all of
them but my current one did not make it far past the booting stage.

The problem was that there was just too much to do: Booting, memory
management, interrupt/exception handling, video output, keyboard, disc access,
file systems, scheduling... most of those things are potentially interesting,
but not all at once, and yet they are all pretty much necessary to get your OS
to the "critical mass" where working on individual aspects starts being fun.

Until I had the idea to not just boot using plain old 16bit DOS, but too also
keep DOS running _in_ my OS, _as a task_. The thing about DOS is that it is
barely an "operating system" in the modern sense. Instead, it is more a crude
collection of 16bit x86 routines that do most of the aforementioned things:
Console input/output, file system access, crude memory management, a whole
shell actually (COMMAND.COM)... and you can even load drivers for networking,
sound, and almost everything else, provided for form of it already existed
about 15 years ago.

My current toy OS is a modern message passing microkernel, but for everything
that I did not feel like implementing yet, I call into the DOS task. The task
runs as a vm86 task, essentially within a hypervisor (called the "vm86
monitor"), but specifically for 16 bit real mode code. (64bit x86 got rid of
the now obsolete vm86 mode, so there you would have to actually implement
something much closer to a modern hypervisor.) This is done using using a shim
layer that receives messages using the message passing system of my OS and
translates them into the (usually) soft-interrupt driven DOS/BIOS function
calls.

This is possible with DOS, unlike modern OSes, because it's such a crude and
conceptually simple OS, centered around direct hardware access and with
essentially no own memory protection/address space separation, that it just
does not get in the way of implementing my own OS.

~~~
Rerarom
So basically you do something like Windows 3.x?

~~~
anyfoo
Or early OS/2, yap!

------
Santosh83
Just curious if someone has done OS dev in top down manner. That is, work
within a modern multi-tasking OS like Windows or Linux and develop a
collection of OS components (in the form of host OS modules?) that gradually
replace functionality of the host OS? I imagine this would be much easier to
do under Linux rather than Windows since the ABI/API for hooking into is open
and documented. This way one can start developing just those components that
interest you first, although I imagine key areas like memory management,
scheduling and interrupts may not be replaceable from within a running OS?

~~~
pjmlp
That is what rich language runtimes are all about.

When you do Common Lisp, Smalltalk, Java, .NET, Go, D, ... the underlying OS
kind of loses its meaning.

The standard library can bind to OS primitives, or you can implement your
hardware abstraction layer that allows for the language runtime to run bare
metal.

If you squint a bit, UNIX is C's runtime, and gets everywhere via POSIX.

~~~
jstimpfle
That's a misleading comment, and I can't find a valid point in there. You can
run C on a different platform than UNIX, or "run it on bare metal", just as
well. If you write C, you make reasonable abstractions, to achieve some amount
of OS independency, just as well. And typical usage of "rich language
runtimes" is on one of the mainstream OSes, just as well. You even often have
some OS specific code in there, because the OS is not entirely
interchangeable.

So, the OS does not lose more of its meaning if you move to rich language
runtime environments. Those language environments are mostly about garbage
collection. Most other things that make the OS "lose its meaning" are just
libraries (that you can write in any language).

~~~
pjmlp
Those language runtimes exist since the 60's, 10 years before C created the
urban myth of being the first systems language, and many of them did not have
anything to do with garbage collection.

Developing on Windows, and deploying across several UNIX flavours, mainframes
and embedded targets has been pretty much OS independent for me during the
last two decades, regardless how much OS specific code those runtime libraries
might have inside of them.

Even if the CI/CD pipeline had to do OS specific builds.

~~~
jstimpfle
> Those language runtimes exist since the 60's, 10 years before C created the
> urban myth of being the first systems language

I don't know what's your point here, or why you keep repeating this on HN
hundreds of times. I didn't ever hear anyone claiming it was the "first
systems language". Nor do I care.

(We all want a better language. I haven't tried _many_ , but any other systems
language I've seen I've found awkward, starting with Pascal with its mess of
1-based vs 0-based indexing, case insensitivity, weird mix of ref-counted /
manually managed, weird syntax choices, mess of basic string types,
requirement to create tons of typealiases... going to older languages that
have upper-case only identifiers, for example).

As to rest - it's just a matter of libraries. Not runtimes.

EDIT: it seems some definitions of runtime include standard libraries. So,
whatever.

------
d33
It's x86, isn't it? I'd love to see something as small as this one that
wouldn't rely as much on BIOS.

The way I understand it, the following code is based one of the assumptions
you can't make without x86 BIOS, can you?

    
    
        /* video memory begins at address 0xb8000 */
        char *vidptr = (char*)0xb8000;

~~~
jng
That doesn't depend on BIOS, it depends on the standard "frame buffer" address
for CGA/EGA/VGA graphics cards (I'm not 100% sure they all use the same
address, those three and Hercules were the standard back in the day, and it
seems my memory has started dropping nearly-useless information from 30 years
ago, not a bad idea).

I quote-unquote frame buffer because it's text mode, one byte for the ASCII
code (extended to 256 characters in configurable and often confusing and
incompatible ways, MS had "codepages" and ISO has the equivalent Latin-1
through Latin-14 or so), and one byte for the attribute.

Many modern cards used to keep the basic VGA compatibility upon boot, until
you switch them into more sophisticated modes. I'm not sure if they still do
these days. QEMU, which is the actual target for the minimal demo kernel
above, pretty sure supports it.

BIOS used INT 10h for video operations, from mode setting to printing
characters, code that calls INT 10h is actually relying on BIOS. IIRC next to
nobody used it except for mode setting, you could much more easily and
flexibly write the right bytes yourself (with a little bit more work for
compatibility). Plus you could use DOS INT 21h to actually print output
strings with a bit more of a concept of a terminal.

~~~
emily-c
You're still relying pretty heavily on the BIOS to write to this region that
you otherwise wouldn't be. Getting the legacy framebuffer region set up to be
decoded to the graphics card is using some hacks to get it to work over
PCI[e]. The memory controller/PCI root complex needs to be configured to
forward transactions to the the legacy A and B segments to the PCI bus using a
proprietary bit to enable that MMIO hole. To enable correct decoding of that,
all PCI bridges on the path to the graphics device need to be specially marked
in their config space bridge control as being "VGA compatible" so that legacy
framebuffer hole requests and VGA IO ports can ultimately be forwarded to the
graphics device. This needs to exist because the root ports/host bridges in
the system base and limit need to encompass the addresses of their children as
well as needing to exist in a hole that the memory controller/root complex
knows to forward requests down to PCI.

~~~
jng
Thanks for the very complete explanation. It makes sense that this would work
this way.

------
akkartik
Is there an 'Awesome' list of simple OS kernels somewhere?

~~~
eadan
It's far from comprehensive, but here are a handful of operating system
projects that have caught my eye:

[1] MezzanoOS, an OS written in Common Lisp
([https://github.com/froggey/Mezzano](https://github.com/froggey/Mezzano))

[2] SerenityOS, a Unix-like OS
([https://github.com/SerenityOS/serenity/](https://github.com/SerenityOS/serenity/)).
The primary author has a youtube channel
([https://www.youtube.com/c/AndreasKling/](https://www.youtube.com/c/AndreasKling/))

[3] RedoxOS, a microkernel OS written in Rust ([https://www.redox-
os.org/](https://www.redox-os.org/))

[4] Nebulet, a microkernel based on webassembly
([https://github.com/nebulet/nebulet](https://github.com/nebulet/nebulet))

[5] It was never released by Microsoft, but Midori OS had some fascinating
ideas like software isolated processes, asynchronous message passing and
object capabilities. Nice write-up about the project by Joe Duffy here
[http://joeduffyblog.com/2015/11/03/blogging-about-
midori/](http://joeduffyblog.com/2015/11/03/blogging-about-midori/)

~~~
pjmlp
As addendum Midori was used by Bing in production.

> While never reaching commercial release, at one time Midori powered all of
> Microsoft’s natural language search service for the West Coast and Asia.

[https://www.microsoft.com/en-
us/research/project/singularity...](https://www.microsoft.com/en-
us/research/project/singularity/)

And although not quite the same, at least Singularity's code is available.

The WinRT .NET 8.x compiler was based on Singularity's way of compiling[1][2]
and UWP .NET Native kind of had some learnings from Midori [3].

[1] - [https://channel9.msdn.com/Shows/Going+Deep/Mani-Ramaswamy-
an...](https://channel9.msdn.com/Shows/Going+Deep/Mani-Ramaswamy-and-Peter-
Sollich-Inside-Compiler-in-the-Cloud-and-MDIL)

[2] -
[https://channel9.msdn.com/Events/Build/2012/3-005](https://channel9.msdn.com/Events/Build/2012/3-005)

[3] - [https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-
Native](https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-Native)

