
How to Make a Computer Operating System in C/C++ - SamyPesse
https://github.com/SamyPesse/How-to-Make-a-Computer-Operating-System
======
Jagat
I am a graduate student, currently working on building a x86_64 unix like
preemptive kernel from scratch, as part of a course. Most of the OS dev guides
and books focus on 32 bit arch and I haven't found a single guide so far that
is based on 64 bit arch. Since this this guide seems to be in its inception, I
hope someone (hopefully me) will send a pull request for a 64 bit tutorial.

Building an OS has been in my bucket list for long and it has been one heck of
an experience so far.

Here are some good step-by-step OS-from-scratch programming guides

[http://jamesmolloy.co.uk/tutorial_html/index.html](http://jamesmolloy.co.uk/tutorial_html/index.html)

[http://www.osdever.net/bkerndev/Docs/title.htm](http://www.osdever.net/bkerndev/Docs/title.htm)

[http://www.ijack.org.uk/](http://www.ijack.org.uk/)

[http://www.brokenthorn.com/Resources/OSDevIndex.html](http://www.brokenthorn.com/Resources/OSDevIndex.html)

[http://viralpatel.net/taj/operating-system-
tutorial.php](http://viralpatel.net/taj/operating-system-tutorial.php)

and here's a good wiki for reference

[http://wiki.osdev.org/Expanded_Main_Page](http://wiki.osdev.org/Expanded_Main_Page)

~~~
mvanotti
I think that AMD Developer Guides & Manuals [1] are very good resources for
x86_64 System Programming (better than Intel Manuals). They are better
explained and focus on x86_64 arch.

[1]: [http://developer.amd.com/resources/documentation-
articles/de...](http://developer.amd.com/resources/documentation-
articles/developer-guides-manuals/)

~~~
dkersten
I still have a soft spot for the intel manuals.. probably because I managed to
get print copies right before they stopped giving them out :)

------
clarry
It's not C/C++. C/C++ is not a language. The project's goal is to write a very
simple UNIX-based operating system in C++ (does anyone else smell a
contradiction?). The code is predominantly C++.

~~~
alexgartrell
> write a very simple UNIX-based operating system in C++ (does anyone else
> smell a contradiction?)

I'm going to use this as an opportunity to champion C++ for systems
programming despite the fact that it wasn't really your argument. I feel
obligated to do this because I agreed with you for a long time on this but
have changed my mind over the past year or two.

The argument against C++ as a systems programming language has always been
that it gives you too much freedom to hide complexity, but I've written
systems code in both C and C++ professionally and it's been my experience that
the extra burden of having to write _every_ _damn_ _thing_ over again in C (or
use a relatively poorly tested container library, as compared to something
like the STL) is not worth the "obvious performance characteristics."

I'd further contend that in non-trivial software all perf characteristics are
non-trivial, so even though a + b might be doing all kinds of crazy things in
C++, you're much more likely to run into problems invoking store_value() which
might interact with a number of underlying libraries and system calls. At the
end of the day you're hopelessly screwed without profiling and the utility
perf works just as well on C++ programs as it does on C programs.

Finally, the majority of code in my experience is either slow path or at least
not the bottleneck anyway. You can almost always optimize a systems program by
making smarter system calls (or hardware invocations), while optimizing CPU is
rarely worth it. Even if you're CPU bound, it's likely that it's in doing
something like SSL, wherein you're almost definitely wrong to try to jump into
a project like OpenSSL with micro-optimizations. With that in mind, why not
allow yourself to move faster by creating a std::vector? When you use the
proper conventions, it's generally as fast anyway.

Just because people can (and have, a lot) misused C++ and created way too much
complexity with it doesn't mean that it's not the preferred language in the
hands of someone who knows what they're doing. (And yes, it's much harder to
learn how C++ works in a meaningful way to avoid these pitfalls, but if it's
your job, learn it anyway).

~~~
Peaker
You can have generalized containers in C, using the intrusive pattern.

For example, Linux's list.h has an intrusive list that is far better than the
abysmal std::list from the STL.

~~~
alexgartrell
Intrusive datastructures were the thing I missed most from C, but
boost::intrusive satisfies my desires when it's absolutely necessary.

In general, I'm anti-boost, but I think it's a personal bias and we use the
hell out of it at work to great effect. The one thing I'll give
boost::intrusive over sys/queue.h is that the type system helps you a lot more
to catch issues and the common case is a bit simpler (a struct that exists in
a single linked list and a single hash table, for example).

I've been burned many times by something like the following

    
    
      struct foo {
        TAILQ_ENTRY(foo) lru_entry;
        TAILQ_ENTRY(foo) hash_entry;
      };
      ...
      void some_magic_func(struct foo *f) {
        TAILQ_REMOVE(static_tailq, f, hash_entry); // oops, should be lru_entry!
      }
    

Boost intrusive templatizes on the member as well (IIRC) and eliminates this
whole class of problem.

~~~
Peaker
In C, you can use a bit of macro trickery to get the same safety, e.g: by
defining a 0-sized array of the some type alongside the intrusive node. Then,
macros that do the "cast down" from the intrusive node to the container
element can also do a type-comparison (using some trickery) between the
anchor's array and the container type.

I've seen it implemented, but everyone uses the typical non-type-safe one
anyway :)

I don't think I've ever had such bugs in a lot of code though, since I tend to
wrap the "containerof" call with a little function like: foo_of_lru_entry and
foo_of_hash_entry. A bit of boilerplate for each data structure, but worth it.

------
exDM69
A hobby operating system project - everyone's gotta have one, right?

There are many like it but this one is mine:
[https://github.com/rikusalminen/danjeros](https://github.com/rikusalminen/danjeros)

It's an x86_64 bare bones kernel project, doesn't do much except for boot and
handle some interrupts. There's a little multitasking and some related stuff.

Unfortunately there's only a finite amount of time in the world, and not too
much of it is available for me to dedicate to this project.

------
waterside81
Who remembers Nachos?

[https://en.wikipedia.org/wiki/Not_Another_Completely_Heurist...](https://en.wikipedia.org/wiki/Not_Another_Completely_Heuristic_Operating_System)

3rd year OS class we had to use Nachos as our base OS and build your usual
file management, pipes etc. functionality on top of it.

Really taught you how an OS works.

~~~
austinz
Our school used Pintos, a derivative of Nachos. Taking the OS course there was
an eye-opener for a student as technically immature and inexperienced as I was
at that time, but it was also a tremendous learning experience. I've always
wished I could go back and retake that course, the 'right way'...with these
resources, though, maybe there's a better way.

------
slunk
Fyi, at least at first, this looks like it will be very similar to a few
wonderful tutorials and projects I have been going through lately:

* [http://www.jamesmolloy.co.uk/tutorial_html/index.html](http://www.jamesmolloy.co.uk/tutorial_html/index.html)

* [http://www.osdever.net/tutorials/view/brans-kernel-developme...](http://www.osdever.net/tutorials/view/brans-kernel-development-tutorial)

* [https://github.com/klange/toaruos](https://github.com/klange/toaruos)

...and just in case anyone is curious, my own project (about half way through
James M's tutorial - working on virtual memory management):

[https://github.com/slunk/Hobbyos](https://github.com/slunk/Hobbyos)

Oh, and btw, this looks AWESOME. I am looking through the finished chapters
right now!

------
rst
Might also be worth looking at xv6, which is a rewrite of Unix v6 in modern C,
and comes with an exegesis modelled on the classic Lions book:

[http://pdos.csail.mit.edu/6.828/2012/xv6.html](http://pdos.csail.mit.edu/6.828/2012/xv6.html)

(Although if you want to run the original Unix v6, written in a now-archaic
proto-C --- unsigned integer variables are declared as pointers because
"unsigned" itself wasn't directly supported in the language yet --- PDP-11
emulators and system images are readily available.)

------
winocm
I just wish more people dealt with ARM instead of x86. It's a far better
architecture for people to learn system design.

~~~
maaaats
Why?

(I'm curious as I know little of these kinds of things, and because citing
sources or giving arguments is always a good thing)

~~~
winocm
For example, look at all of the legacy stuff in an x86 system. You have the
strange structure of the interrupt descriptor table/global descriptor table.
You have the good old Intel 8042 keyboard controller, Intel 8259 programmable
interrupt controller, old-style programmable interval timer, A20 gate and so
much more.

There's so much legacy baggage inherent in the x86 architecture, all because
of software compatibility. (ROM-BASIC will still run on a modern PC!)

ARM on the other hand has less to really worry about, sure there's some legacy
infrastructure, such as the ARM vector table. Modern ARM architectures such as
ARMv8-A remove old legacy baggage in favor of completely renovating it. (Look
at the old coprocessor interface for instance. In A64 mode, it is completely
gone. You must use 'mrs/msr' instructions compared to 'mrc/mcr'). There's also
far less legacy software to worry about.

------
djvu9
For those who are interested in developing a realistically useful small OS in
C++ (or even C++ 11), you may probably want to check out OSv
[https://github.com/cloudius-systems/osv/](https://github.com/cloudius-
systems/osv/) developed by previous KVM guys.

------
agumonkey
Been reading Massalin's Thesis on Synthesis OS (partially evaluated,
dynamically generated kernel code), it's a refreshing and very inspiring read:

[http://valerieaurora.org/synthesis/SynthesisOS/](http://valerieaurora.org/synthesis/SynthesisOS/)

------
e12e
Maybe a little irationally, the idea of programming an os in c++ strikes me as
very opaque. I think the vipri[1] approach of layering dsls, or the smalltalk
idea of a relatively simple vm to _seem_ more understandable than an os that
embeds a c++ runtime...

As a side note, when looking up [1] I also ran across [2].

[1] [http://piumarta.com/software/cola/](http://piumarta.com/software/cola/)
[2]
[http://www.acm.uiuc.edu/sigops/roll_your_own/1.helloworld.ht...](http://www.acm.uiuc.edu/sigops/roll_your_own/1.helloworld.html)

~~~
dnautics
BeOS was primarily written in C++, although the kernel was dominantly C. For
actual parts that require user interaction, C++ and the object model makes a
whole lot of sense, although there are probably more mature languages out that
would be better candidates now.

~~~
crusso
I loved BeOS and the C++ application framework... except for the linking
model.

You had to provide re-linked executables for every new version of BeOS that
came out. It was exceedingly lame to go to an application download page and
have to select the version of the application that corresponded to the version
of BeOS you installed.

Then when you upgraded your OS, you had to upgrade ALL of your applications.
Sure, there were some framework upgrades that didn't necessitate a re-linked
application, but as a user you never really knew which those were so had to do
a full upgrade of everything to be sure.

~~~
dnautics
Agreed. Although, I don't understand why they didn't provide "shim" .so files.
I _think_ this is what linux does, though in practice these days I just use
ubuntu packages... And at the time to do the same thing in linux, or FreeBSD
(which is what I migrated to when I left BeOS) you had to - or were encouraged
to - recompile everything from source.

------
xradionut
How about making a language, compiler, OS and applications and not use C or
C++ ?

[https://news.ycombinator.com/item?id=6829464](https://news.ycombinator.com/item?id=6829464)

~~~
reirob
You forgot to add in your list: making a completely new processor ;) Which the
linked project actually achieves.

~~~
xradionut
There's that too. Although I'd like to have the time to port Oberon to MIPS,
maybe the PIC32 since the more powerful, modern MIPS chips are out of budget.
Haven't done the analysis, but I think it would be a good match.

------
Spl3en
Another very complete tutorial :
[http://www.brokenthorn.com/Resources/OSDevIndex.html](http://www.brokenthorn.com/Resources/OSDevIndex.html)

------
devhinton
This is great. I am just jumping into open source from an academic background.
I had assumed that open source was great for directly practical website
creating and hosting (back-end, client side etc...) but weaker in terms of
learning for OS / compilers. This post and its comments (with linked
resources) are making me unsure about this assumption. What do you think?

------
zplesivcak
Here [1] you can find another example of writing OS from scratch. Going
through chapters progress is made iteratively by making limited changes in
each step. This OS is intended to be (RT)OS for embedded computers, but
example arch-level is implemented for x86.

[1] [https://github.com/l30nard0/Benu](https://github.com/l30nard0/Benu)

------
vezzy-fnord
Obviously this is still in its very early stages, but I can see this becoming
a very informative and practical course.

I'll definitely be keeping an eye on this. Do you intend on turning this into
a book and perhaps publishing physical copies?

~~~
SamyPesse
Yes it's very early stages I'm looking for feedback on how to structure the
course.

No I don't think so, I just want to an informative resource from what I've
learned.

~~~
SyneRyder
In case you haven't heard of it, you might want to take a look at LeanPub, who
specialize in books that are published while they are being written, have a
particular focus on Software Development books, and accept manuscripts written
in Markdown stored in a Dropbox folder. You can then take the finished LeanPub
book to a print-on-demand company like Lulu or CreateSpace to created printed
books as well:

    
    
      https://leanpub.com/authors
    

I actually assumed you were making a LeanPub book, and was considering buying
the work in progress.

------
stonewhite
"It was written several years ago as one of my first projects when I was in
High School" High school? Really? Are there any proper highschools like these?
I am jealous now.

------
tejasm
Very interesting. A friend and I tried creating our own OS back in 2002 using
interrupts to write the boot program. I wish I had this tutorial back then.

