
RISC-V OS using Rust: Filesystems - pavehawk2007
http://osblog.stephenmarz.com/ch10.html
======
webgoat
I've been following along while building an armv7a kernel and I really like
how you go back and refactor old parts to match newer rust features like your
change to using cargo instead of a makefile and inlining asm files into the
rust binary.

My only criticism is that sometimes there's parts in your tutorial that are
missing but present in your os. When going through chapter 3/4 I think you
skipped over explaining your kmalloc implementation. It's fine that you do
since I feel your tutorial should be more about navigating embedded rust than
learning os basics, but a note about this task being implemented at some point
would've helped.

~~~
pavehawk2007
Thanks for the feedback. It does get tricky to balance going too far into the
weeds and keeping it above board. I think a note would be helpful as you
suggested.

I do try to put lots of comments in the Rust code to describe what I didn't in
the blog for those more interested in the implementation.

------
StanAngeloff
Excellent series of posts. I've been following along with no prior Rust or OS
programming knowledge and have found it immensely refreshing.

~~~
pavehawk2007
Thanks! Glad it could serve some purpose!

------
Animats
I'd like to see a clone of the QNX kernel, which is very tiny, implemented in
Rust. It has no drivers or file systems in kernel space; those are all user
processes. But it offers a POSIX-like API. Unlike L4, you don't have to run
another OS on top of it to do anything.

This would be good for embedded applications on Raspberry Pi sized machines,
where the Arduino environment is too weak and Linux is too much.

~~~
speed_spread
Aren't there patents protecting it? I remember reading something about the
fast interprocess call mechanism... Microkernels are prone to high overhead
because everything requires context switching. Maybe this isn't true anymore.

~~~
Animats
US 8,434,086, "Process scheduler employing adaptive partitioning of critical
process threads", is still in force until 2030. That's a more recent feature
for hard real time, where you want to guarantee that some thread gets, say,
2ms of CPU time every 10ms of elapsed time. Few schedulers do that sort of
thing at such fine granularity, but that's what you may need if you're
controlling some actuator. Good for things like audio players, too. It's not a
commonly used feature, and went into QNX some time after 2005. I remember it
being listed as a future feature back then.

The fast interprocess call mechanism is what makes it all go. It's mostly
about it being simple. Mach tried to get too cute, moving pages from one
address space to another. Turns out that copying is usually cheaper than
stalling out the CPU and flushing caches while you mess with the page tables
of recently active pages. L4 just offers shared memory, dumping the problem on
the next level up. QNX offers useful send/receive message functionality that
goes fast. Yes, there's a penalty, maybe 20%. But you usually get that back
because you're not forced to do interprocess communication with something far
slower, like HTTP.

------
a1369209993
A couple of minor, but very irritating errors:

> The singly-indirect pointer can address 1,024 * 1,024 / 4 = 1,024 * 256 =
> 262 KiB of data.

256 kilobytes, not 262 'kibi'bytes.

> 1,024 * 256 * 256 = 67 Mi

> 1,024 * 256 * 256 * 256 = 17 Gi

64 and 16 respectively.

~~~
pavehawk2007
If that irritates you, please don't check the grammar and errant placement of
commas!

All jokes aside. Fixed, thanks.

~~~
a1369209993
Actually, the grammar seems fine? Or at least I haven't noticed any problems
with it. TFA as a whole seems pretty good actually. The file sizes are still
incorrect, though; it calls the units kilobytes now, but it still claims
there's 262 of them.

The seven blocks at the start would make it 26 _3_ KB, but I don't think that
what you meant.

~~~
Arnavion
>The file sizes are still incorrect, though; it calls the units kilobytes now,
but it still claims there's 262 of them.

1024 * 1024 / 4 B = 262144 B = 262.144 KB = 256 KiB

~~~
a1369209993
Yes, exactly; 262144 bytes is 256 kilobytes.

~~~
Arnavion
KB is kilobyte. KiB is kibibyte.

~~~
Koshkin
> _kibibyte_

I wonder if anyone uses this abomination of a word in real life though.

~~~
a1369209993
Not that I know of; they're called kilobytes.

------
bogomipz
I had a question of the about a passage under the section: "Our OS's
Filesystem Read Process" where the author states:

>"Recall that the block driver makes a request and sends it off to the block
device. The block device services the request and then sends an interrupt when
it's finished. We don't really know when that interrupt will come, and we
can't wait around for it. This is why I decided to make the file system reader
a kernel process. We can put this kernel process in the waiting state so it
won't be scheduled until the interrupt is received from the block device"

Then further down: >"Now, when we handle the block device's interrupt, we have
to match the watcher and awaken it."

Is the watcher here similar to the filesystem driver at the VFS layer in Linux
i.e the ext4/xfs etc? I believe this is how Linux handles it - the file system
driver creates a block IO request and hands it off to the actual block device
driver and when the block device driver get's invoked on an interrupt from the
disk the block device driver informs the filesystem driver that the IO request
is now complete correct?

------
sl1ck731
Is there a significant limitation in using Rust on the stable branch? Every
interesting tutorial, book, or tool I find seems to use nightly. I know it
shouldn't be a big deal but it "feels dirty" and perhaps I should just view
nightly as any other language's "stable".

~~~
gpm
> perhaps I should just view nightly as any other language's "stable".

It depends on the features you enable. Stable is basically just a pinned
nightly version with no features enabled, and I'd definitely view "nightly
with no features" as more stable than most languages.

Using some of the more experimental features would make it less stable, you
can generally get a feel for how stable a feature is by looking at it's
tracking issue.

These days most tutorials/books/tools don't seem to use nightly in my
experience, though many still do. But if you're primarily looking at blog
posts that are pushing the boundaries of the language (e.g. by writing
operating systems) I expect that most of them would be using nightly.

~~~
estebank
Stable is pinned nightly with disabled nightly features _and beta backports
applied_. The difference might sound like semantics, but it is present and can
impact you.

------
marricks
This looks awesome! Yesterday I just started another OS rust walk through and
now I have a hard choice between that and this.

Any thoughts on this[1] walkthrough in comparison? This one looks more
explicitly up to date.

[1] [https://os.phil-opp.com/](https://os.phil-opp.com/)

~~~
pavehawk2007
I think you'll get something different for each. The purpose of mine is to
explore the Rust programming language, but more specifically on the RISC-V
architecture. Rust "recently" became out-of-the-box for RISC-V, which has
opened up many more avenues for exploration.

As a note, I believe os.phil-opp.com is using the AMD/Intel architecture.

~~~
jimbob45
What do you mean by out of the box for RISC V?

~~~
pavehawk2007
Originally, I required a GNU toolchain from the RISC-V organization for both
assembling and linking. Now, I don't need any external tools (except QEMU to
run it). Everything you need can be downloaded using Rust's configuration tool
rustup for RISC-V. If you take a look at my github, you'll see that I added a
.cargo/config file that will build AND run the operating system using cargo.

~~~
sitkack
Wait, we can now host a whole OS w/o directly requiring a C compiler?! I mean
Rust still requires one to build clang/llvm, but maybe someday when
alternative backends land.

~~~
steveklabnik
There are several projects that do this, yes. They're generally not production
ready, but it's absolutely possible.

------
loeg
The minix fs seems to be basically the classic Unix FS design, like UFS/FFS or
Ext2, prior to the concept of cylinder groups (at least, per this description
and Wikipedia's) and without some of the more advanced features of those
filesystems (softupdates, directory trees, etc).

~~~
yjftsjthsd-h
I mean, that's what you'd expect from an OS that was originally for teaching,
right? A worked example of the fundamental parts common to most other unix
filesystems, just as I would naively expect the minix scheduler to just be
round-robin or something, etc.

~~~
loeg
Well, that was especially true for Minix 1 and 2. Minix 3 is purportedly a
production operating system, and ships in every Intel CPU for the last decade
or so, right? So while admittedly it may not have significant filesystem
needs, it is not purely a teaching operating system anymore.

Nowadays I might point at MIT's xv6, which implements a very similar simple
unix filesystem: [https://github.com/mit-
pdos/xv6-public/blob/master/fs.c](https://github.com/mit-
pdos/xv6-public/blob/master/fs.c)

~~~
pavehawk2007
xv6 also was switched to RISC-V (not exactly sure when, but sometime in 2019),
and it has a nice book for the actual course.

Course:
[https://pdos.csail.mit.edu/6.828/2019/xv6.html](https://pdos.csail.mit.edu/6.828/2019/xv6.html)
Book: [https://pdos.csail.mit.edu/6.828/2019/xv6/book-riscv-
rev0.pd...](https://pdos.csail.mit.edu/6.828/2019/xv6/book-riscv-rev0.pdf)

Take a look at chapter 7 for the xv6 filesystem referenced here.

------
jeffdavis
Is the asm!() macro going to stabilize? It seems important to ever make a real
OS, right?

~~~
steveklabnik
It's closer than it's ever been! The rough plan:

* move asm! to llvm_asm. [https://github.com/rust-lang/rust/pull/68404](https://github.com/rust-lang/rust/pull/68404)

* implement the actual asm! we want to stabilize: [https://github.com/rust-lang/rfcs/pull/2873](https://github.com/rust-lang/rfcs/pull/2873)

* stabilize it

~~~
pavehawk2007
I stand corrected! Thanks for the info!

~~~
steveklabnik
It's all good. I think that was the prevailing sentiment for years, but
Amanieu put in the work, and things are looking bright. It hasn't been
accepted yet, of course, so anything could happen. But stuff is moving in that
direction, it seems.

------
fluffything
This is awesome. I've been following along for a while, so please keep it up.

I wish someday we'll get something like the ESP32 boards but with RISCV.

~~~
pavehawk2007
Thanks!

You might want to check out the Kendryte K210. I'm trying out these boards for
development. They are designed with two (I think) 64-bit RISC-V cores with a
hardware FPU. [https://www.seeedstudio.com/blog/2019/09/12/get-started-
with...](https://www.seeedstudio.com/blog/2019/09/12/get-started-
with-k210-hardware-and-programming-environment/)

I've flashed mine a few times, but I'm still working to getting it to work.
The documentation isn't very good, so I'm working to reverse engineer some
aspects of it.

This is a low cost alternative. Of course, we could go with the SiFive
Unleashed (64-bit) or the SiFive HiFive1, which uses a 32-bit RISC-V CPU in an
Arduino form factor.
[https://www.sifive.com/boards](https://www.sifive.com/boards)

The SiFive Unleashed cost me a cool $1k, so I'm still scared to mess with it.

~~~
FullyFunctional
The Kendryte is pretty limited in many ways, not least that it's permanently
limited to 8 MiB of DRAM (AFAIK).

The most interesting route is using an FPGA. There are many examples of
running RISC-V softcores on FPGAs (for example, Berkeley's RocketChip) and
many boards that support a decent amount of memory (even beyond 1 GiB).

~~~
pavehawk2007
I tried some of SiFive's evaluation cores. They limit you quite a bit unless
you pay. Furthermore, many of the FPGAs worth their salt were quite expensive.
The K210 in the MaixBit form is only $20, and it comes with a camera and LCD.
For hobbyists, I think it's a good compromise.

I looked at the Xilinx FPGAs--the ones SiFive recommends, but they're quite
pricey.

Which FPGAs had you had luck with?

~~~
fluffything
I was looking at getting this Maixduino kit since it appears to be the
cheapest solution that satisfies my constraints (RISCV core + WiFi):

[https://www.seeedstudio.com/Sipeed-Maixduino-Kit-for-
RISC-V-...](https://www.seeedstudio.com/Sipeed-Maixduino-Kit-for-RISC-V-AI-
IoT-p-4047.html)

I'm not sure what's the status of programming these from Rust. I suppose that
you will probably need to build your own toolchain that links the Seeedstudio
libraries.

~~~
pavehawk2007
RISC-V has their own supported toolchain. You can get it here:
[https://github.com/riscv/riscv-gnu-
toolchain.git](https://github.com/riscv/riscv-gnu-toolchain.git)

With Rust, you can do what I did with my OS. It can compile out of the box.
The kendryte has a "BSP" (base support package) that is essentially a minimal
OS, but it is written in C.

~~~
fluffything
Yeah, I was thinking more about what to do with the OS afterwards. That board
has lots of things on it, including wifi, so I'll probably would like to use
some of that.

It would be unfortunate to have to re-implement the wifi driver in Rust, but I
suppose I'll just have to link parts of the C toolchain instead, creating Rust
wrappers, etc.

