
Hobby x86 kernel written with Zig - netgusto
https://github.com/jzck/kernel-zig
======
jackhalford
Hi, author here! I've just finished writing the pre-emptive multitasking
[0](only round robbin though, nothing fancy).

I'm currently writing an ATA driver [1], the idea is to implement ext2.

I used to do this in Rust but I switched to zig for maintainability and
readability over Rust. It seems that with `comptime` I'm able to make a lot of
things optimal.

Overall I have to say kernel programming is _hard_ but very rewarding when it
finally works, it's really demystifying computers for me!

[0] [https://wiki.osdev.org/Brendan%27s_Multi-
tasking_Tutorial](https://wiki.osdev.org/Brendan%27s_Multi-tasking_Tutorial)

[1] [https://wiki.osdev.org/IDE](https://wiki.osdev.org/IDE)

~~~
guggle
> I used to do this in Rust but I switched to zig for maintainability and
> readability over Rust.

Can you expand on this ? I'm asking out of curiosity because I want to learn a
"system" programming language (for whatever definition there is to this term).
So far I briefly tried Rust and Nim and found the former more difficult to
read. I know nothing about Zig, how would you place it between these two ?

~~~
jackhalford
In general you'll find that zig is easier to read than Rust (see the first
version of this project in Rust [0]) because it's a simpler language. For
kernel programming this is even more so the case:

* zig has native support for arbitrary sized integers. In Rust I used to do bitshifts, Now I just have a packed struct of u3/u5/u7 whatever (see `src/pci/pci.zig`). Of course Rust has a bitflags crate but I didn't find it handy, this is a case where native support vs library support means a world of difference.

* zig has native support for freestanding target. I used to have to build with Xargo for cross compiling to a custom target, I also was forced to `[nostd]`, and some features I was using forced me to use nightly Rust. In zig I have a simple `build.zig` and a simple `linker.ld` script, just works.

* zig has nicer pointer handling. A lot of kernel programming is stuff that Rust considers unsafe anyway. It's not uncommon to have to write lines like `unsafe { _(ptr as_ const u8) }` to deref a pointer in Rust, which is a pain because this kind of thing happens all of the time. Also you have to play around with mut a lot like this: `unsafe { &mut _(address as_ mut _)`. It just felt wrong a lot of the time, where in zig you have either `const` or `var` and that's the end of it.

* zig is really fun to write! this is something that comes up often in the community, after years of C it's just very refreshing.

Some things zig is missing:

* Package manager, this is coming soon. [1]

* Missing documentation for inline assembly (I think this part is going to get overhauled, as Andrew Kelley is writing an assembler in zig atm [2]).

I don't know Nim, but I believe it has a garbage collector so it could be
tricky to use for kernel programming.

[0] [https://github.com/jzck/kernel-rs](https://github.com/jzck/kernel-rs)

[1]
[https://github.com/ziglang/zig/issues/943](https://github.com/ziglang/zig/issues/943)

[2]
[https://www.youtube.com/watch?v=iWRrkuFCYXQ](https://www.youtube.com/watch?v=iWRrkuFCYXQ)

~~~
simias
It's true that for bare metal programming Rust is quite far from 1.0. You
can't even do inline assembly in stable rust...

Arbitrary sized integers do sound very convenient, although I guess the
counterargument would be that they simply can't map to machine types so you'll
have to have magic behind the scenes which may not make complete sense in low
level languages. Still, C has bitfields in structs which are sort of the same
thing so why not.

Pointer handling however I'm not sold on. I'd argue that the very cumbersome
pointer handling in Rust is a feature, not a bug. It's explicitly discouraged,
and for good reasons. Map your pointer into a clean and safe reference, slice
or wrapper object ASAP and leave raw pointers to the messy details of C FFIs
and ultra low-level code. If you end up having to mess with raw pointers
everywhere in your codebase you're doing it wrong, as far as Rust is
concerned.

I think this is reasonable even for kernel code. After all typically you
wouldn't dereference register pointers directly in C code because of
volatility issues and making it obvious that you're accessing hardware and not
RAM (readl/writel or similar), so having a safe wrapper adds no overhead in my
experience.

~~~
jackhalford
Ok maybe I was user pointers wrong in Rust. And I must confess that I never
really understood lifetimes either, however

* I don't think that low level code needs to be "messy", and

* I don't think that adding an abstraction on top solves anything complexity wise.

For memory paging I used to have a whole library in Rust that handled
hierarchy of page tables with very abstract pagetable classes and interfaces
[0]. Now I just have 80 lines of zig that handles everything [1]. I much
prefer the latter.

> I think this is reasonable even for kernel code.

It's not, for some of the reasons that Linus doesn't want C++ code in the
kernel [2]

[0]
[https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354...](https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354cb82023031cb9535/src/bits64/paging.rs)

[1]
[https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/pagi...](https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/paging.zig)

[2]
[https://yarchive.net/comp/linux/c++.html](https://yarchive.net/comp/linux/c++.html)

~~~
CDSlice
The Rust code you wrote seems to be extremely heavy on boilerplate. A couple
of macros could have drastically reduced the line count.

I don't think what Linus wrote in 2004 on C++ has much relevance for 2020
Rust, especially for a new kernel that doesn't have a ton of contributors.
Most of his complaints seem to be on how C++ abstractions can make it hard to
review unknown code and how 2004 C++ code bases often had extremely messy
code. Rust IMO doesn't suffer from those problems near as much as 2004 C++
did.

Also, Google is using Rust for part of their new Fuchsia kernel so at least
they think Rust has something good to offer kernel dev.

~~~
roblabla
Small nitpick: Google isn't using Rust for the Fuchsia kernel. The kernel,
Zircon, is written entirely in C and is based on Little Kernel. Google is
using Rust for some userspace drivers (Fuchsia being a micro-kernel).

~~~
loeg
> The kernel, Zircon, is written entirely in C

No, it's written in C++[1]. They're very different languages.

That said, Google seems to think C++ has something to offer to all kinds of
development. It seems to work for them, but they are a heavily C++ shop. I
wouldn't read too much out of their use of C++ anywhere; it'd be more
surprising and interesting to see them use anything else.

[1]:
[https://fuchsia.googlesource.com/fuchsia/+/master/zircon/ker...](https://fuchsia.googlesource.com/fuchsia/+/master/zircon/kernel/kernel/scheduler.cc)

~~~
roblabla
Oh that's interesting. Zircon is based on LK[0] which is in C, which is why I
thought it was still written in C. I wonder how much is left of the original
LK code then.

[0]: [https://fuchsia.dev/fuchsia-
src/concepts/kernel/zx_and_lk](https://fuchsia.dev/fuchsia-
src/concepts/kernel/zx_and_lk)

~~~
loeg
Interesting, I didn't know about LK. Thanks for sharing :-).

------
knebulae
I tried to do this with Zig about 13 months ago. It was not where it needed to
be at that time; the biggest impediments were its rudimentary handling of C
pointers to one vs pointers to many (which has long since been fixed), and its
meta programming issues (lack of a macro language or pre-processor) that made
OS development tedious. I have not revisited it as much as I would have liked
simply because I chose to step back a bit on implementation and focus on
theory.

I'm pulling for Andrew. He busts his rear-end, livestreams, and is generally a
good dude. Zig has a TON of potential.

~~~
Bekwnn
I've been waiting for something that looks like a firm step forward in the
domain of games programming and has ideals which align with the domain. I'm
extremely excited for zig and have been messing around with getting a smallish
simulation running with SDL on Windows.

At work in C++ I've switched from working on fairly isolated types, where my
changes had fast recompile times to lower level changes which cause a good
chunk of the engine to recompile. 10 minute compile times, with a lot of tech
trying to get that time down as much as possible, are a huge killer to
productivity and I can feel myself getting much less done than I was before.

Zig tossed away a lot of the constructs that make C++ slower to compile. I
haven't had a chance to see its timings on large projects, but stuff like Jai
compiling 90k LoC full commercial game project live on a laptop in 1.4 seconds
(which caused Jonathan Blow to say "what? That's weirdly slower than it should
be...") gives me hope that Zig is similar.

------
vips7L
Here is a microkernel written in Zig:
[https://github.com/AndreaOrru/zen](https://github.com/AndreaOrru/zen)

------
FpUser
I really wish Zig picks up and goes mainstream. I also wish it gets a little
(not too much) of higher level features. Something in line of simple OOP.

I would even start using it now for some smaller projects that are not vital
but I was stalled on Zig not being able to compile some C (it claims to do
that and it does but not in my cases). Sure I could do import but would rather
prefer for Zig feature to work

------
aganame
Check out zig's "Safety" project
[https://github.com/ziglang/zig/projects/3#card-27896159](https://github.com/ziglang/zig/projects/3#card-27896159)
to get a view of what sort of safety is being planned.

I'd like to state that after significant time and effort with Rust, I also
think it's too complex for what it's protecting us from. Zero-cost indeed does
not refer to the cognitive price. It's better than C++ though, in every way
except popularity.

------
rumanator
Very interesting. Keep up the good work!

------
blackrock
It would be nice if you put up a design document.

I'd rather read that first to get my bearings, as opposed to opening up a
folder full of gibberish code.

Usually a well written design document explains the high level constructs of
the project. Then some mid-level documents ties in the design document, with
the implementation code.

This makes the project more intelligible, and allows for a layman to jump in,
and follow along.

------
kburman
I'm also trying to do something very similar. I can use it as a reference.
Thanks for sharing!

------
archsurface
The hello world examples for master and 0.5.0 seem quite different - any
trouble keeping up with the changes?

~~~
jackhalford
migration from 0.4 to 0.5 was ok. AFAICT there the big change for 0.6 syntax
wise is the drop or varargs in favour of tuples. I'll take some time to do the
switch when 0.6 hits but I'm confident it won't take long because of how
concise the language is.

~~~
AndyKelley
0.6.0 is scheduled for April 13, 2020. Let's make sure your project builds
successfully and correctly with master branch 1-3 weeks before that, and I'll
prioritize any bugs before the release that affect you.

(This goes for all zig community members with active projects)

