Hacker News new | past | comments | ask | show | jobs | submit login
Redox: A Rust Operating System (github.com)
194 points by mfincham on Sept 29, 2015 | hide | past | web | favorite | 51 comments

I know this is a young project, but right now it looks like a terrible waste of Rust's features. Raw pointers everywhere, most allocations unchecked, and 264 unsafes in just the kernel.

unsafe is a necessary escape hatch, but the temptation to use it to write C-style code is strong.

Hello, I wrote Redox, and I completely agree. I have been doing pretty substantial cleanups in this manner recently. At first, I had to start with pretty nasty code to get things going. Within the last week, I seperated out fundamental applications and moved them to the filesystem. This also meant writing a syscall interface, which is currently compatible with Linux system calls. Unsafe is often necessary in kernel land, and I am always rewriting code to make its use span smaller scopes.

Hello! I think it's possible to encapsulate unsafe pointer operations so they're only required in memory allocation, I/O primitives, and device DMA. I'll open a github issue or two with thoughts.

Out of curiosity, are you modeling this on an existing code base/architecture, or is this a new design?

This is a new design, but the syscall interface is like Linux, the read/write/open/close/ syscalls are the same ABI.

As for unsafe, it would be possible to make much less of the kernel unsafe. The kernel allocation could be exposed in an interface like this:

struct Memory { address: usize }

impl Memory { fn new() -> Option<Memory>; fn realloc(&mut self, size: usize) -> Option<Memory>; fn size(&self) -> usize }

impl Drop for Memory { fn drop(&mut self) }

PIO could be wrapped up MUCH better:

PIO8, PIO16, and PIO32 could handle inb, inw, ind. A lot of my driver bugs have come from using the wrong size!

MMIO could be handled similarly.

The PCI driver could give drivers a limited set of PIO and MMIO structs to operate on

Maybe you could add some constructive criticism in a few GitHub issues? I'd say these kinds of things are easier to fix earlier rather than later.

Rust really seems like the next step in system programming.

Would it be possible to have this lifetimes/borrowing stuff in a scripting language?

Like, I care about who owns something, but I don't care about what something is?

Depends on the precise details, but generally speaking the nastiest problem would probably be the fact that `region analysis + type erasure = sadness`.

For instance, you can't just have all your objects be some opaque thing, you need to know if it holds borrows into other types. On one end you can certainly do runtime checking (basically everything is a RWLock) which is slow and still just crashes at runtime. Or you can maybe do whole-program analysis (slow, weird non-local errors).

As far as I can tell, Rust strikes a really clean balance for region systems. Everything about signatures needs to be declared, and the rest can be locally inferred. But when you use type erasure (runtime vtable-based dispatch) you need to decide whether the erased objects are allowed to contain borrows, and what they borrow, upfront.

Sad to read, but thank you :)

Don't scripting languages typically not exhibit the problems that Rust attempts to address?

Rust, among other things, prevents mutable aliasing, allows to create non-escaping pointers and provides deterministic destruction and unique ownership.

These things are essential if you don't have a GC but have a way to free memory, because the bugs they prevent would result in accessing reading or writing memory at invalid addresses.

That's not a problem with a GC, but they are still very useful because they prevent these issues that cannot be statically prevented in Java/JavaScript:

- Exceptions due to one part of the program manipulating a data structure while another is accessing it: the canonical example is someone iterating a dictionary while something else mutates it

- The program keeping a reference to mutex-protected data after unlocking the mutex

- Two threads manipulating the same object, causing races

- Something accessing a File that is closed, or in general a type in an unexpected state (this is because with a GC you can't "destroy" the previous object, but merely set it to a "closed" state that you can't statically check)

- Exceptions due to null pointers

So in general Rust is a safer language than Java, JavaScript and languages equivalent to them.

I had the impression Rust tries to get memory safety without the need/overhead of GC.

I don't know about the borrowing system, but I believe there are several Lisps that avoid mark-sweep garbage collection. NewLisp is one such example, though I think they sidestep the issue by copying everything by default and using efficient allocation/deallocation.

Memory safety is compile-time guaranteed except in unsafe blocks.

It's a little less strong than your impression. Memory absolutely can be a problem, but instead of looking over the entire program to find it, you know the problem comes from a marked unsafe block.

Not only do scripting languages generally use GC or at least refcounting, but most scripting languages (Python, Ruby, JavaScript, Lua) are either difficult to use with guaranteed thread-safety or simply do not support multithreading at all. If you don't have multithreading, you can't have data races (by definition), and it's harder to hit race conditions. It's still possible -- you can queue a callback that operates on data that's invalid -- but lots of programs will never care.

Rust is intended for performant, close-to-the-metal (hence the name) programming, so mandatory GC/RC won't cut it, but a single-threading restriction won't cut it either. And I think the way the borrow system implicitly gives you all the properties needed for thread-safety is one of the best things about Rust's design.


The same region/ownership discipline that Rust provides to handle memory turns out to be useful in other contexts. One is resource usage: you have to close a file handle after opening it, etc. Patterns for cleaning up after yourself are common in many languages. What Rust provides is a way to describe how long something should live and a way to enforce correct usage through compile time type checks. This can be used to ensure people using the library are unable to do the wrong thing. As a simple example, using a mutex incorrectly in safe Rust will not compile. More complex examples:



Lifetimes are part of the type, so in that sense you would care what it is.

Seems like this would go against the nature of scripting languages.

The blog was a good read. I agree with acconsta that the description doesn't seem to utilize Rust's better attributes. Plus, that it's "starting to look like Linux" (author) may or may not be good. I'd look at Wirth's Oberon or A2 Bluebottle for a simpler start: type-safe, memory-safe (mostly), GC'd OS with good documentation, source available, and a simplicity focus for easy re-implementation. Rust, in theory, can do whatever it did and faster.

Additionally, if doing clean-slate, might be worth looking into alternative models for constructing or securing OS's. EROS security OS, SPIN OS's type-safe linking for acceleration, Minix 3's reliability scheme, JX OS's architecture, Microsoft's verified VerveOS scheme, maybe even Amoeba distributed OS just for kicks to see what it could do today. Lot of stuff that might be better than Linux model with a Rust implementation in terms of reliability, security, extensibility, or developer productivity.

Just a thought for Redox author or someone else wanting to try an OS in Rust. However, best thing to come out of Redox project, imho, isn't the OS so much as this article:


Great write-up on an equally great strategy of developing with hardware that's well-supported by both native OS's and virtualization. That's worth copying and expanding in other projects. Maybe worth a dedicated list like the HCL's where there's a list of computer builds that are easiest to develop on with or without virtualization. Or list it one piece of hardware at a time.

Interesting, looks like a few generations of Toughbooks are available cheap on ebay. Looks like a nice platform overall: unbreakable titanium chassis, touch screen, pretty standard hardware (for running OSs with limited hw support). And a few (of the newer, but still cheap) versions even come with SSD disk.

Looks like a great toy for making all sorts of single-purpose devices/terminals -- not just testing OS kernels.

That was so good I put it on the blog!

Didn't see that coming haha. Glad you liked it. At 135pts & front page, you apparently are cool enough for Hacker News. Probably a cool project in a HN-favorite language led to this. ;)

I am Jackpot51, I wrote Redox and would be able to confirm with a master push.

To start, this wasn't supposed to be public yet. I was preparing cleanup before release, and I hope it is not seen as complete.

I am excited to see how much attention this has gotten, and I would be willing to answer any questions about Redox, how to use it, and how to contribute.

I have a blog at https://redox-os.org/ where I recently posted screenshots.

We can IRC at irc.mozilla.org #redox

"Redox is not correct, secure, or documented. It does a lot of things in a cavalier, leave it up to the application, manner. Applications can malloc and forget to free, Redox will not clean this up. Applications can access missing memory or kernel memory, Redox does not care. All applications run in Ring 0, and have all permissions. Some syscalls use vtables that rely on exactly the same struct defintions in userspace as kernel space. Syscalls cannot return values, except by writing them into a passed pointer. Memory allocation is done in the application, and uses a global memory allocation table that applications directly modify. None of its functionality is documented, outside of example source files." -- https://redox-os.org/


OS development takes time and he clearly states this will change, criticizing such a young project with halfquotes is unfair and discouraging for the developers. shame on you.

I'm waiting for OS development to start up in rust as it is clearly the better choice then C/C++ for a kernel.

How easy is it to deal with raw memory in rust? That's the main reason for using C in my opinion.

I don't know what you mean by "deal with raw memory" but rust allows you to do silly things like:

    let foo = unsafe { std::slice::from_raw_parts_mut(0xdeadbeef as *mut u8, 42) };

That gets you a slice (pointer+length that can be used as an array) that points to 0xdeadbeef. Bit more verbose than the c equivalent

    char* foo = 0xdeadbeef;
but arguably this is pretty rare usecase even for handling raw memory

You have to use `unsafe`, but you can do anything you can do in C. Raw pointers are the primary mechanism, same as C pointers http://doc.rust-lang.org/stable/book/raw-pointers.html

Not sure is it like this anymore, but might be worth giving a read:

> But allocating memory seemed like a fun exercise. To allocate something on the heap in Rust, you can do

> let a = ~2

> This creates a pointer to a 2 on the heap.


That syntax has been changed, actually. Now, to allocated an owned pointer allocated on the heap, you use the Box type:

> let a = Box::new(2);

I believe it's an unstable feature currently to use the "box" keyword to heap-allocate a value, which can also be used in pattern matching. I saw somewhere that "box" will be made into an overloadable operator, such that you can easily create reference-counted and atomically reference-counted types.

Yes, that's generally correct. We still haven't fully decided on the exact syntax. If anyone wants to get involved, this RFC was the one that got merged, but hasn't been made stable yet: https://github.com/rust-lang/rfcs/blob/master/text/0809-box-...

And this is the active, in-final-comment-period one which modifies it: https://github.com/rust-lang/rfcs/pull/1228

Read this like Linus Torvald's comment about how Linux "won't be big and professional like gnu" comment from its early days.

I'm not saying Redox is the next Linux. But big things come from small beginnings, and good engineers are intensely aware of the limitations of their creations.

Which is immediately followed by.

"I do, however, have a plan. I will make all applications run in Ring 3. I will do malloc and free through syscalls. I will move significant pieces of functionality into userspace libraries (a new libredox). I will use file numbers to identify file descriptors, and create read/write/seek/close syscalls. I will document the syscalls interface, provide more example programs, including programs that test the correctness of the implementation."

So the creator is aware of and wants correct those issues.

Sounds a lot like a Commodore 64 or TempleOS, but in those this was/is considered an advantage.

I do not understand much about low level languages or operating systems, but could someone explain why does this project need C in its codebase?

Probably doesn't, but it does currently contain a copy of lua, which is written in C.

There are Rust OS projects with nothing but some assembly and then Rust code, so it shouldn't be strictly required.

Disclaimer: haven't had time to read the code yet.

It is probably easier to do all of the massively unsafe things required to initialize in C than it is in Rust. Then Rust is just used after C does its dirty things.

That shouldn't ever be necessary. Part of the point of Rust is that you can do unsafe low-level work, so long as you tag it with the "unsafe" keyword to enable pointer arithmetic and so forth.

The OS is completely Rust, with a minor amount of ASM for the bootloader and other requirements.

C is a third party possibility, I allow C programs to run on Redox.

> Redox is a Rust based operating system, designed to be modular and well documented.

How does one design something to be well documented?

The other comment about the Oxford comma is probably correct, but I'll answer the question anyways: I have found that I design things differently if I'm doing the documentation up front. I have known myself to re-write functions because it seemed like too much work to describe what the function is supposed to do in plain English. If I had put off documentation for later, I would have built a different system.

Well, depending on your definition of "Design", you do things like

  1. Ensure you have the tooling to produce nice docs.
  2. Build time to write docs into your planning.
  3. Actively try to find people who will write good docs.
  4. Focus on making your code understandable, so that the
     eventual docs will make sense. Spaghetti code is hard
     to document.

(IMO) that is taking documentation in consideration while designing, and not designing to (it's not the actual purpose of the design).

Pedantism aside, I guess it was a phrasing issue.

Literate programming.

Though obviously in this case it was ambiguous punctuation.

I am the author of Redox. I didn't choose that line well, so I simply removed that text.

I meant that I would like it to be well documented, not that it would be designed around documentation.

the sentence is missing the oxford comma before "and"

Like all designs. You write it down and immediately throw it away and do w/e you want.

Can someone clarify, but it looks like it's going to be a monolithic approach here correct?

In the current Redox design, everything runs in privileged mode, so it's kind of the ultimate monolith?

I kinda got that from the blog, but I guess I was more curious if anyone got a whiff of longer term design goals and if that was where things were headed.

Judging by the syscall implementation, seems to be 32 bit only? Apologies if I missed something in the docs, such as they are.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact