
Let’s write a simple Kernel (2014) - mmphosis
http://arjunsreedharan.org/post/82710718100/kernel-101-lets-write-a-kernel
======
exDM69
Bare metal projects are always a fun hobby. I've done some projects like this
in the past [0], let me share some things I learned the hard way.

If the author is reading, here are a few corrections.

> char * vidptr = (char * )0xb8000;

This needs "volatile" qualifier or all the writes to video memory will be
dropped when optimization is enabled.

Memory mapped I/O like this is one of the few use cases where you need
"volatile" in C.

> gcc -m32 -c kernel.c -o kc.o

You need to build a cross-compiler for bare metal projects. Using the system
gcc will not work in the long run. The compiler packaged with your operating
system is intended for building binaries for use with your operating system.
It may have downstream patches or configured to a target in a way that will
cause problems.

The worst part is that it may _seem_ to work for a while, until it doesn't. I
ran into some hard-to-debug issues with my past projects and learned this the
hard way. If I recall correctly, the issues were related to redzones and ABI
conventions.

You need to build GNU binutils and GCC for the "i686-pc-elf" target. I
documented this process for someone else's bare metal project here [1].

It really pays off to do this right from the start. Once you have a cross-
compiler and a build system that can produce debuggable elf images (you also
need to build gdb for the target), things get much easier. Using the built-in
debuggers in QEMU or Bochs can only get you so far. Having a proper debugger
with symbols and source view will make working much easier.

For a build system, plain old Makefiles work best, CMake and other high level
build systems are a pain in the ass with bare metal projects that require
linker scripts, etc.

[0]
[https://github.com/rikusalminen/danjeros](https://github.com/rikusalminen/danjeros)
[1]
[https://github.com/Overv/MineAssemble/blob/master/README.md](https://github.com/Overv/MineAssemble/blob/master/README.md)

~~~
bogomipz
>"You need to build a cross-compiler for bare metal projects."

Might you have any useful links or documentation on this that you could share?
Thanks.

~~~
mw6621
I found this a while back, it works great for compiling a cross compiler for
Linux and Windows.

[https://github.com/lordmilko/i686-elf-
tools](https://github.com/lordmilko/i686-elf-tools)

There's also a bit of information on OSDEV if you are wondering why you need a
cross compiler:

[http://wiki.osdev.org/GCC_Cross-Compiler](http://wiki.osdev.org/GCC_Cross-
Compiler)

~~~
carussell
FWIW, the need to bootstrap a cross-compiler from source is not inherent to
cross-compilers; it's possible to implement a compiler in such a way that
every version of the compiler is automatically a cross-compiler.

I remember a few years back when I was trying to play with MINIX. Tanenbaum
got several million EUR and hired some grad students to work on the thing.
They promptly replaced much of the system with NetBSD. ("Perhaps too much",
you can hear Tanenbaum say in one of his talks.) As a result of this the
system compiler ACK was switched out for LLVM/clang. I complained on the
mailing list about this because a full system build from source is something
that used to be doable in <10 minutes—something Tanenbaum used to boast
about—and it was now taking 3 hours if you decided to blow away your
source/build directory and do a from-scratch build. The worst part is that the
MINIX core, i.e., all the interesting parts, still only accounted for ~10
minutes of that build time, and virtually all the rest was spent compiling and
then recompiling LLVM. The response I got from one of the aforementioned grad
students was that this is "just how cross-compilers work". No, pal; that's how
the compiler that _you chose_ works.

Later, the Go folks fixed their compiler to be a cross-compiler by default.
See [https://dave.cheney.net/2015/03/03/cross-compilation-just-
go...](https://dave.cheney.net/2015/03/03/cross-compilation-just-got-a-whole-
lot-better-in-go-1-5)

IMO, it's unforgivable that any given mainstream toolset wouldn't make this a
baseline project goal.

~~~
mveety
The Plan 9 compilers work exactly like this, every compile is a cross-compile.
It's extremely convenient because you can do all of your builds on your really
fast machine for all of your other machines or if you're using a slow machine.
It's one of the reasons I can live happily with a raspberry pi as one of my
main workstations. All you need to do to build for a different target is
change $objtype.

------
d99kris
My favourite kernel development tutorial is bkerndev [0]. It provides easy-to-
read sources that can be used as a good base for simple projects.

Shameless plug: I used bkerndev verbatim for my bare metal project - Nope OS
[1] - a C64-like system that I built for my son when he was born, so that he
could get to know computers the same way I did. :)

[0]
[http://www.osdever.net/bkerndev/Docs/intro.htm](http://www.osdever.net/bkerndev/Docs/intro.htm)

[1] [https://github.com/d99kris/nopeos](https://github.com/d99kris/nopeos)

------
magnat
It's not much of a kernel. More like a self-contained bare-metal application.
Main goal of a kernel is to provide services to applications.

Although starting coding right away and getting tangible results is somewhat
rewarding, it would be nice to see core concepts of _designing_ a kernel
mentioned in a "Kernel 101" article:

* How to choose between monolithic/micro/nano architecture?

* What are primary goals and use cases?

* What hardware should it abstract away in HAL?

* How to design userland-kernel and drivers-kernel API/ABI?

* How much isolation is needed and what are ways to provide it?

------
bogomipz
I had a question about some of these passages, as they appear contradictory
and or incorrect.

>"Most registers of the x86 CPU have well defined values after power-on. The
Instruction Pointer (EIP) register holds the memory address for the
instruction being executed by the processor. EIP is hardcoded to the value
0xFFFFFFF0. Thus, the x86 CPU is hardwired to begin execution at the physical
address 0xFFFFFFF0. It is in fact, the last 16 bytes of the 32-bit address
space. This memory address is called reset vector.

This says that the EIP is doing a JMP to an address in RAM not a memory-mapped
IO address which points to ROM where the BIOS is stored.

>"Now, the chipset’s memory map makes sure that 0xFFFFFFF0 is mapped to a
certain part of the BIOS, not to the RAM. Meanwhile, the BIOS copies itself to
the RAM for faster access. This is called shadowing. The address 0xFFFFFFF0
will contain just a jump instruction to the address in memory where BIOS has
copied itself."

The second says that the CPU is doing a JMP to an IO mapped-memory address
which points to ROM.

So the first passage says that CPU is just doing a JMP to 0xFFFFFFF0 in RAM.
The second passage say that the CPU is doing a JMP to memory-mapped IO address
which points to ROM.

Are these not completely contradictory or am I reading this wrong?

It's always been my understanding that that reset vector always pointed to a
memory-mapped address which was located in ROM. Since the BIOS' POST routines
contain code to initialiZe and test memory(the BIOs will actually emit beep
codes if no memory is present or memory is faulty.), this would be a chicken
and egg problem.

Also the post mentions the chipset loads the BIOS into RAM as a process called
"Shadowing" which was done because ROM used to be slow. But since BIOS ROM
these days in generally NAND flash I don't believe this is the case any
longer.

Also these two statements appear to contradict each other:

>"All x86 processors begin in a simplistic 16-bit mode called real mode. The
GRUB bootloader makes the switch to 32-bit protected mode by setting the
lowest bit of CR0 register to 1. Thus the kernel loads in 32-bit protected
mode."

>"Do note that in case of linux kernel, GRUB detects linux boot protocol and
loads linux kernel in real mode. Linux kernel itself makes the switch to
protected mode."

The first states that the kernel loads in protected mode and then the
following states that kernel loads in real mode.

~~~
gmueckl
Grub itself is mostly written as a 32 bit program (almost it's own OS by now,
actually). The x86 Linux kernel does expect to perform the protected mode
switch on its own in the early assembly code right before the zImage
decompression starts. So Grub has to switch back to real mode in order to jump
into this part of the Linux kernel.

Grub has other ways to boot kernels. I once used a method that looks for a
specific signature inside the kernel blob that would tell Grub to stay in 32
bit mode, put the blob at a certain memory address and jump into it. This was
a feature of Grub 1 back in the day. I do not know if it was removed in Grub
2.

~~~
bogomipz
Thanks, yeah I guess I hadn't really thought about who flipped the protected
mode bit Grub or the kernel before. It's documented here in the section "3.2
Machine state"

>"When the boot loader invokes the 32-bit operating system, the machine must
have the following state ... ‘CR0’

Bit 31 (PG) must be cleared. Bit 0 (PE) must be set. Other bits are all
undefined."[1]

However it sounds like Grub also understand the Multiboot Specification so can
keep protected mode set if it's booting a kernel that expects it to already be
set. It's not clear to me how Grub would determine that.

[http://www.gnu.org/software/grub/manual/multiboot/multiboot....](http://www.gnu.org/software/grub/manual/multiboot/multiboot.html#Machine-
state)

------
billpg
Decades ago, I wrote a very simple operating system. It was an MSDOS EXE that
when run, took over all available memory and rewrote all the interrupt
vectors. You couldn't "exit" back to DOS because by then it had gone - you
could only power-cycle the machine and let MSDOS boot up normally.

Because it launched from DOS, many insisted to me it wasn't really an
operating system. What was it if it wasn't an operating system?

~~~
gmueckl
Novell Netware actually used DOS as a glorified boot loader. Also, there was
at a point a small DOS program that acted as a Linux boot loader. It brought
you from a DOS prompt straight to a Linux virtual terminal without reset. I
think SuSE shipped it for a while as an alternative means to launch their
setup, but memory fades.

~~~
bri3d
loadlin.exe! It ran under memory-unsafe Windows (95/98/Me) as well.

------
drngdds
"This programming tutorial is great, but it could really use some giant meme
images," said no one ever.

------
visarga
Oh, that kind of kernel. There are many kernels - GPU kernels, SVM kernels,
etc.

~~~
mar77i
...what Colonel?

------
varunagrawal
Am I the only one who's pretty much done with reading these sort of rinse and
repeat articles about just writing a bootloader in the name of a kernel?

~~~
vidarh
Nobody is forcing you.

I wish we had more of these, not least because of the first comment: "great
post. Didn't know it was this easy ;)"

People assume a lot of low level stuff like this is near magic not for mere
mortals, and it's a great shame, because there's a lot of this kind of low
level stuff that more people would benefit from playing with.

And because demystifying it is a great path to get people into kernel hacking
even if they don't end up writing their own complete kernel.

~~~
gmueckl
This stuff is not magic. But it requires a lot of reading and hunting for
proper documentation on a platform as convoluted as x86. An ARM board would
probably be way easier to play with. I never looked at the Pi. How well does
it lend itself to this type of hacking?

~~~
vidarh
_Specific_ ARM boards might very well be easier, in that a lot of them are
clean slate designs, so you're not dragging along nearly 40 years of PC
history. But ARM CPU's themselves is plagued by the fact that OEMs can put
pretty much whatever they want in there, so there is a proliferation of really
weird beasts.

