
Sortix: small self-hosting OS aiming to be a clean, modern Posix implementation - ingve
https://sortix.org/
======
lomnakkus
(Sorry, mini-rant incoming...)

Before I get all negative, I'll say that this looks very good for
education[1], but I'm really curious about why the word "modern" was used.

POSIX is ancient, is overspecified in very important areas (which, thankfully,
many/most implementations ignore/work around), is underspecified in very
important areas, holds us to a C-based API/ABI, doesn't even have sane
"atomic" file operation semantics, etc. etc.

POSIX is not something to aspire to, IMO. We tolerate it because we have to,
AFAICT.

(Rant over. I can only apologize, I just sometimes get a little angry that
we're held back technologically because of this nonsense.)

[1] Even given the limitations, which are _very_ clearly acknowledged. Kudos
to the project people for that!

~~~
LukeShu
> _holds us to a C-based API /ABI_

This shows a bit of a misunderstanding about what POSIX is. It isn't about
operating systems, it (IEEE 1003.1) is about source-code compatibility for 2
languages: C99, and Shell. POSIX doesn't say anything about an ABI, and
there's nothing holding the system as a whole to a C-based API. It's
specifying C compatibility, of course it's talking about a C API! If you want
a different language API, POSIX (1003.1) isn't relevant to you. Totally non-
UNIXy systems like z/OS are POSIX certified, because they have source-code
compatibility with C99 and Shell programs written against POSIX; that doesn't
mean it is held to a C-based API.

~~~
lomnakkus
> POSIX doesn't say anything about an ABI, and there's nothing holding the
> system as a whole to a C-based API. It's specifying C compatibility, of
> course it's talking about a C API! If you want a different language API,
> POSIX (1003.1) isn't relevant to you.

(Double-replied because I wanted to separate this bit out.)

Effectively, it seems you're saying: "Ok, you don't want to implement a C
API"[1], well, jolly good luck to you, then. The premise of this whole
conversation was that POSIX was a dominating force, and now you're saying you
don't have to participate...?

[1] A C API, obviously, requires an _extreme_ amount of either 1) being C, or
2) being compatible enough with C that it doesn't matter either way, or 3)
giving up entirely on the POSIX ecosystem. So... how is that _not_ being
absurdly badly influenced by POSIX?

------
abenedic
One cool thing I like about Sortix is how it handles threading. TLS is set up
by the kernel, so errno can be a thread local variable.

~~~
jeffreyrogers
I think errno is thread local in most modern unix variants.

~~~
sortie
Author here. POSIX requires errno to be thread local, but with few
requirements. Commonly libc's #define errno (*__get_errno_location()). Why?
It's because the thread local storage isn't online very early on. The libc
literally has to open its own program executable and locate the the thread
local storage and mmap a copy of it for each thread. Thread level storage
comes online late in Linux, which bloats statically linked executables with
code to parse their own executable.

In Sortix, I just have the kernel set up the TLS for statically linked
executables and just declare errno as:

__thread int errno;

That means errno is online from the very first instruction in _start in
statically linked executables, without the need to parse the program file in
early libc. There's no error conditions between _start and main. This is the
kind of implementation quality I want to have in Sortix and part of why I
develop Sortix.

~~~
jeffreyrogers
Ah, that's very neat. Thanks for explaining.

------
notacoward
Looks pretty cool as an educational exercise. Therefore, if the author's
reading: how would you distinguish this from MINIX?

~~~
abenedic
Not the author, but the thing that got me interested in the project was some
posts the author made on OSDev[0] about trying to minimize the amount of
interdependencies in the libc code. A lot of the kernel is written in C++[1]
which is a little rare (Haiku, and what else?). They also have a cleaned up
zlib that I have used and think is quite nice[2]. Also I almost forgot about
os-test[3] which tests POSIX compatibility.

[0][https://wiki.osdev.org/User:Sortie/Sortix_Thread_Local_Stora...](https://wiki.osdev.org/User:Sortie/Sortix_Thread_Local_Storage)

[1][https://gitlab.com/sortix/sortix/tree/master/kernel](https://gitlab.com/sortix/sortix/tree/master/kernel)

[2][https://sortix.org/libz/](https://sortix.org/libz/)

[3] [https://sortix.org/os-test/](https://sortix.org/os-test/)

~~~
chasil
Linus Torvalds had (in)famous commentary on the use of C++ within kernel
design.

[http://harmful.cat-v.org/software/c++/linus](http://harmful.cat-v.org/software/c++/linus)

I wonder if things have changed in the decade since he penned that.

~~~
jcranmer
Well, the newer comment basically amounts to "Real programmers know to use C,
so we keep the uncultured plebes out." Which isn't really any argument against
using C++, but is rather the kind of attitude that should be expunged from
system developers.

There are reasons to avoid using C++:

* The main benefit of C++ is that it has a standard library that's somewhat usable. Except you can't use standard libraries anyways in kernel code.

* Templates, particularly as they intersect with name lookup, are difficult semantics both to the compiler and the user. Until a decade ago, using templates meant more or less playing roulette as to which compilers could handle it correctly.

* RTTI and exceptions are broken features. It has more or less been accepted that these features are optional in C++.

* The famous saying goes "there's a safe subset to C++, but no one agrees what that subset is." That's still more or less true today.

The main advantage of C++ is that it gets you RAII (which makes dealing with
errors much easier, since you're less liable to have issues like gotofail),
and you can use templates to build safer ADTs. These are strong benefits in
that it's a lot harder to make mistakes as a developer, but there is a strong
tendency in this corner of systems developer to assume that stupid mistakes
are a fault of the programmer and not the language, and so such benefits are
considered flaws because it means letting in stupid people.

~~~
flyingfences
> "Real programmers know to use C, so we keep the uncultured plebes out." ...
> is rather the kind of attitude that should be expunged from system
> developers.

I have to disagree in this case. I think the rationale here is that a working,
productive mastery of C suggests a developer of a certain caliber. For many
projects you would be right, it would be good to encourage the more junior
developers to join the project. But, for the Linux kernel, for the core of one
of the most widely-deployed pieces of software on the planet, the foundation
of countless systems, many of which are performing critical functions, I,
frankly, think it's good to filter the dev team for properly competent devs at
every possible juncture. If rejecting C++ keeps the noobs out, that's a good
thing.

~~~
eropple
_> I think the rationale here is that a working, productive mastery of C
suggests a developer of a certain caliber._

Possibly, but that's still pretty silly. Like...I am a cautious programmer by
nature. I can write C. I understand the language and its semantics at a fairly
intuitive level. I am still cognizant that a moment's inattention means that I
am _going_ to screw up in ways that I literally can't with a language that has
a scope guard (to say nothing of typing that isn't based on how wide the data
structure is, that sort of thing). Not using a language more featureful than C
--even to the point of literally banning every feature from the language that
isn't a nonvirtual dtor--doesn't "keep the noobs out". It does, however,
create classes of bugs. If anything, a "noob" is less likely to realize the
danger they're in when fearlessly[0] writing code!

Recognition of my own limits is most of why I don't write C, and if I
absolutely must interop with C (passing through to Ruby or whatever) I write
the thinnest, dumbest layer I can between it and C++. (These days, I'm trying
to go towards Rust for these tasks, but I know C++ much better.) C just does
not make you the hard man that many C devotees think it does, that's all.

[0] - This is mostly a compliment and only a little tongue-in-cheek. Novices
being willing to write code that I have to think about analytically to feel
safe when writing it can, in the general case, often work well enough/long
enough to get by.

~~~
nine_k
I can agree that C is a poor language (as opposed to "rich"). It's a pretty
thin layer over machine code, and its means of abstraction, and means of
static checking that these abstractions hold, are very weak. It was designed
around 1970 to work on hardware from 1960s.

The problem is that in early 1990s, when Linux started, no viable alternatives
existed. Pascal required a standard library; Modula-2 did not, but it was late
to the scene. C++ was poorly supported, and even more full of hidden traps
than C. Ada was theoretically a good choice, but accessible compilers did not
exist.

C had widespread use, 20 years of history, many compilers, including a free
(GPL'd) GCC, wide platform support. Because the language is so compact and
simple, these compilers were mostly reliable, there are no complex features
that are tricky to implement.

Hence the choice.

Now we have a much wider choice, we have Rust, we have ATS-lang, we have good-
quality C++ compilers, we have various tools to synthesize verified C code
from higher-level code, etc. We can hope than in 10-15 years we could see e.g.
a viable OS kernel written in Rust.

------
convolvatron
I've been poking at something similar lately, and if you're mostly running on
VMs, I would suggest taking a look at the virtio interfaces. they are high
performance, asynch, and relatively easy to implement.

~~~
sortie
Yep. I already have an issue about virtio filed at
<[https://gitlab.com/sortix/sortix/issues/473>](https://gitlab.com/sortix/sortix/issues/473>).
Not too high priority to implement but something I'd love to get to. Feel free
to post additional hints there if you have additional tips for when I get
around to doing it.

~~~
convolvatron
I started a note on your gitlab issue, but saw you already had an mmio
reference linked. i did the pci version and it looks about the same
complexity.

aside from a reference, being able to instrument the device side on qemu makes
debugging drivers a total cakewalk. so do that. best of luck with your
project. i wish more people went through the exercise. even though a mature OS
after decades of maintenance might exceed 1M lines, you really dont need
anywhere close to that to get what you need done. you learn alot, and can make
your own model

------
feelin_googley
Is sortix kernel compliant with multiboot specification?

If not, is it possible to use syslinux or another bootloader instead of grub?

~~~
Samis2001
Not the author, but the kernel is Multiboot compliant. See:
[https://sortix.org/man/man7/installation.7.html](https://sortix.org/man/man7/installation.7.html)
'If you wish to dual boot, you need an existing operating system with a
multiboot compliant bootloader such as GRUB.'

~~~
feelin_googley
Thank you. Further down in that page:

"Bootloader

The kernel(7) is a multiboot compatible binary that can be loaded by any
_multiboot specification compatible_ bootloader such as GRUB. You need to use
such a bootloader to boot the operating system. You will be offered the
_choice_ of installing GRUB as the bootloader."

Allowing the user to choose their own bootloader makes this particular small
project a rare find, IMO.

I use another multiboot specification compatible bootloader that I prefer over
grub.

I also like how he offers both 32 and 64bit.

I guess this project has been posted to HN before. Perhaps at that time it
still did not yet have networking.

~~~
sortie
Author here. Yep, I don't want to force a particular bootloader on anyone. The
bootloader is a piece of software owned by the system administrator, not
necessarily by any of the several operating systems installed side by side. If
I force a bootloader on the user, that makes dual boot configuration harder,
and it's important to allow that. I happen to follow GRUB's multiboot
specification, which has the advantage that a lot of bootloaders implement it.
It's not perfect though and I'll probably invent and adopt my own bootloader
protocol in the future, but the sysadmin's choice of bootloader continues to
be important on my mind.

------
chasil
Xv6 is the classic reroll of the UNIX 6th edition sources for the x86
architecture.

What does Sortix offer that is lacking in Xv6?

[https://pdos.csail.mit.edu/6.828/2014/xv6.html](https://pdos.csail.mit.edu/6.828/2014/xv6.html)

~~~
ronsor
TCP/IP support

~~~
sortie
Author here. Yep, though the network stack hasn't yet been merged to the
master branch (but that's next and it'll be in the next stable release).

