
An Experimental Course on Operating Systems - jzoch
https://web.stanford.edu/class/cs140e/
======
jlebar
If any Stanford students are reading this wondering if they should take this
class:

Engler was one of my favorite professors at Stanford. He's kind, sharp, well-
prepared, and consistently delivered great lectures.

One of the things that made him stand out to me are the insights he provided
beyond what's in the papers we read in CS240. You can read the papers
yourself, but the papers won't tell you things like "I think the reason this
paper was accepted is different from the reason the author likely thinks it
was accepted."

He's awesome, take his class. Find a good team, and, um, don't underestimate
how much time CS140 takes. :)

~~~
srcmap
Is the goal of the this class teaching student how to re-write the Pi3 's
kernel/driver in Rust?

It is not that clear from the course syllabus.

"This course covers the following topics:

Disks, File systems, I/O, Threads & Processes, Scheduling, Virtual Memory,
Protection & Security, Interrupts, Concurrency & Synchronization."

In theory, if those are the goals, one doesn't need Rust as part the first
week's lecture.

Rewriting the Kernel in Rust sure makes a very interesting (and very tough)
way to learn OS in 3 months.

~~~
habitue
It's not writing a new kernel in Rust, it's an OS course, where they touch on
all the fundamentals. Instead of assignments being in C, they're in Rust.

------
Dowwie
If anyone has looked at web frameworks in Rust, they'll have noticed one known
as Rocket ([http://rocket.rs](http://rocket.rs)). The Rocket project is
authored by none other than Sergio Benitez, who is _teaching_ Stanford CS140e.

~~~
akhilcacharya
I really should not have read that resume. Really doesn't help the overall
inadequacy problem.

~~~
shawn
On the other hand:

 _At Google I worked with the Gmail performance team to decrease server
downtime by detecting anomalous behavior before server failure. I designed and
implemented an original anomaly detection algorithm based on local outlier
factors._

Does anyone have any info about anything related to this?

I want to write a program to detect anomalous behavior in my systems too.
(Mostly so I can completely ignore the useful information, but still.)

~~~
adtac
I can talk about my experience writing such a tool during my internship at a
bank that rhymes with Oldman Bachs. Bear in mind, this wasn't my original
project; since I had finished my first project, I was allowed to work on
something else I find interesting. So it will probably not be very robust, but
it worked.

The tool would run in trading engines and monitor different metrics of
performance. CPU usage, while allowed to burst, shouldn't be sustained at 100%
for a long time. Typically, these machines are monsterous and have hundreds of
gigs of memory; therefore, they shouldn't come anywhere near 100% memory usage
either. There's a bunch of other proprietary internal metrics that I can't
really explain (stuff like latency of a particular kind of response).

Anyway, since I was most comfortable with writing the tool in Python (also,
time was a constraint as I started working on the project towards the end of
my internship), I wrote a modular tool with pluggable metrics. The reason why
it wasn't a monolithic block: if the company decides to add a new metric they
want to measure and follow, they should just have to write the logic necessary
to retrieve that metric. The rest should be handled by my framework: anomaly
detection, alerting relevant parties, and so on.

So banged out the framework and had just enough time to write two modules: CPU
usage and memory usage (probably the easiest ones). With a simple moving
window average and standard deviation, you can identify outliers: I marked
everything outside 2*sigma as an outlier as it's better to have false
positives than false negatives (but not excessively many false positives as
that would be counter-productive and make humans ignore reports).

The alerting part was fairly straightforward as there was a built-in tool to
do that. I ran a few tests with historical data and it was pretty good at
detecting the anomalies that I would mark myself as a human. It could be
better if someone smarter than could come up with a machine learning algo to
do the same thing more accurately.

(I didn't productionize it as I didn't have the time.)

------
nafizh
I have made a list of all the materials needed for this class with amazon
links. Hope this helps people who would like to pursue this course from
outside Stanford.

1\. 1 Raspberry Pi 3 [https://www.amazon.com/Raspberry-Pi-
RASPBERRYPI3-MODB-1GB-Mo...](https://www.amazon.com/Raspberry-Pi-
RASPBERRYPI3-MODB-1GB-Model-Motherboard/dp/B01CD5VC92)

2\. 1 1⁄2-sized breadboard [https://www.amazon.com/Qunqi-point-Experiment-
Breadboard-5-5...](https://www.amazon.com/Qunqi-point-Experiment-
Breadboard-5-5%C3%978-2%C3%970-85cm/dp/B0135IQ0ZC/)

3\. 1 4GiB microSD card [https://www.amazon.com/Samsung-MicroSD-Adapter-MB-
ME32GA-AM/...](https://www.amazon.com/Samsung-MicroSD-Adapter-MB-ME32GA-
AM/dp/B06XWN9Q99/)

4\. 1 microSD card USB adapter [https://www.amazon.com/Adapter-Standard-
Connector-Smartphone...](https://www.amazon.com/Adapter-Standard-Connector-
Smartphones-Function/dp/B01BXSKPES)

5\. 1 CP2102 USB TTL adapter w/4 jumper cables [https://www.amazon.com/KEDSUM-
CP2102-Module-Download-Convert...](https://www.amazon.com/KEDSUM-
CP2102-Module-Download-Converter/dp/B009T2ZR6W/)

6\. 10 multicolored LEDs [https://www.amazon.com/Multicolor-Flashing-Changing-
Electron...](https://www.amazon.com/Multicolor-Flashing-Changing-Electronics-
Components/dp/B01C19ENG4/)

7\. 4 100 ohm resistors
[https://www.amazon.com/dp/B0185FCR66/](https://www.amazon.com/dp/B0185FCR66/)

8\. 4 1k ohm resistors
[https://www.amazon.com/dp/B00CVZ46FM/](https://www.amazon.com/dp/B00CVZ46FM/)

9\. 10 male-male DuPont jumper cables, 10 female-male DuPont jumper cables

[https://www.amazon.com/Haitronic-Multicolored-Breadboard-
Ard...](https://www.amazon.com/Haitronic-Multicolored-Breadboard-Arduino-
raspberry/dp/B01LZF1ZSZ/)

Edit: edited for solderless breadboard link rather than a solderable one.

~~~
tzs
[Report of incorrect link deleted, as it has been corrected]

For the 100 and 1k resistors, I'd recommend something like this:
[https://www.amazon.com/PIXNOR-Resistor-1ohm-10Mohm-
Resistors...](https://www.amazon.com/PIXNOR-Resistor-1ohm-10Mohm-Resistors-
Assortment/dp/B00SD6F6FC/)

That includes 20 1k and 20 100 ohm resistors, plus 20 each of 54 other
assorted resistor values, and it costs slightly less than just buying the 100
ohm and 1k 10 packs that you suggested.

~~~
jzoch
No that is not the correct one, we are using a solderless board.

------
pmarreck
Does anyone else think it's time for a new and promising operating system? The
hegemony of OS X/Windows/Linux has basically gone on for a generation.

Shout-out to BeOS (the old geeks will know of it) which was the last promising
new OS I encountered... and that was many moons ago

~~~
danbruc
Here is another thought. One has to realize that nobody wants an operating
system. People only use them because it is the only way to run the
applications the users actually wants to use. If you could run an office
suite, a browser and a mail client without Windows or Linux, that is what
people would do. At best an operating system is absolutely invisible,
realistically it causes additional pains over those inflicted by the
applications. I can not really imagine how this could work, but a new non-OS
would probably be a better idea than a new OS.

~~~
AnIdiotOnTheNet
Application management, window management, resource management, document
management, etc. OSs are more than just the applications they run, they
provide the user ways to manage their workflow.

On the other hand, you know what OS was practically invisible? DOS. And DOS
was pretty great, so maybe you're on to something.

~~~
spc476
MS-DOS wasn't that great. Oh, don't get me wrong, it's great at being a singer
user, non-reentrant interrupt handler on a painful to program architecture
[1]. A lot of applications skipped using device drivers entirely for things
like the keyboard, video and serial I/O (using such made the program run more
slowly).

Do you know how much work it was to redirect the input/output of an MS-DOS
application? It can get pretty insane [2].

[1] You had to pick a memory model to compile against for one thing---tiny,
small, medium, compact, large or huge. You had to deal with FAR and NEAR
pointers, and it really drove home the point that sizeof(int) != sizeof(long)
!= sizeof(void *) when writing C on that system.

[2]
[http://boston.conman.org/2015/12/13.1](http://boston.conman.org/2015/12/13.1)
More info here:
[https://github.com/spc476/NaNoGenMo-2015](https://github.com/spc476/NaNoGenMo-2015)

~~~
StillBored
I wrote a fair number of DOS programs. The memory models wern't that bad, at
least you had choices... I also wrote code on machines that had 64k address
spaces, with soft banking. That was worse... What I remember being a huge PITA
was memory management on macOS in the system 5/6 time-frame. I vaguly have
memories of struggling with locking/unlocking handles.. Heck googling about
it, turns up a whole wikipedia article which spends a bit of time describing
it in general.

[https://en.wikipedia.org/wiki/Mac_OS_memory_management](https://en.wikipedia.org/wiki/Mac_OS_memory_management)

Anyway, in dos, .ost little utilities and such worked just fine in tiny (and
you could use .com which simplified everything even more). Medium and compact
were sort pointless because if you were writing anything you expected to grow
beyond 64k then you used a HLL and let the compiler deal with most of the
fiddly stuff. IIRC One of the compilers (may have been turboC) had an
project/link option which let you cycles through all the memory models, and
with a quick rebuild you could check executable sizes or run little
benchmarks. I did a fair amount of programming in Turbo Pascal, and I just
checked the manual, it basically hid the entire memory model argument, even
going so far as to have that "overlay" functionality which was basically a
software paging unit for your program which would load groups of methods from
disk as necessary.

Yah, and I wrote TSR's too... and hacks to take over bios/dos INT calls, etc
etc etc. These days it seems any software project that has been around for a
few years is orders of magnitude more difficult than the stuff people claim
was hard about DOS. Heck there are _network_ drivers in linux that are orders
of magnitude more code (and more complex) than DOS or probably the vast
majority of DOS applications. People simply didn't write 100 million line DOS
programs so dealing with little hardware oddities is simple vs trying to
ferret out VM barrier bugs in linux (or whatever).

------
kaycebasques
Sounds cool!

Mildly related: I found "Nand To Tetris: The Elements of Computing Systems" to
be an amazing, bottom-up, hands-on approach for learning about the fundamental
layers of computer architecture, from hardware to assembly to OSs.

~~~
chucksmash
A wonderful complement (albeit with significant overlap) is Charles Petzold's
Code. It's written as prose so doesn't include exercises but the book is
infused throughout with the mastery attained by its author. His synthesis of
different ideas and the historical context he provides give a very nice
perspective on computing. While nand2tetris makes for a good textbook to work
through, Code makes for a better subway read.

------
jzoch
I am currently taking the class. Its an embedded systems + OS class that
undertakes the challenge of writing an operating system for a raspberry pi 3
in Rust. The lectures aren't recorded sadly, but its the first time offering
something like this and both Sergio and Dawson are fantastic professors!

------
StillBored
I guess its "experimental" as in your will in a lab setting with experimental
(rust!?) code for traditional operations on an rpi.

Not that you will be experimenting with new OS concepts.

Which is sort of a shame because it seems much of OS research might be turning
back to concepts that haven't been explored since the "RISC/unix" revolution
in the late 80's early 90's proclaimed that multiple privilege level machines,
capabilities, full ACL controlled operations, message passing kernels, and
dozen's of other concepts wern't "fast" or fell to the wayside because the
RISC and traditional unix model couldn't support them, while we continue to
pay a huge hidden tax for the flat address/paged memory model...

------
wcrichton
Note that the "e" is inspired from CS107e [1], the experimental version of
"Introduction to Computer Systems". 107e also uses a Raspberry Pi to help
students incrementally build up a working knowledge of basic system components
including the processor, memory, and peripherals.

[1] [https://cs107e.github.io/](https://cs107e.github.io/)

------
kissiel
Looks similar to "Baking Pi" from Cambridge.
[https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/os/](https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/os/)

~~~
sigjuice
I'm afraid I disagree. An OS course needs to at least cover virtual memory,
processes and filesystems. Doesn't look like "Baking Pi" does.

~~~
barbegal
That depends on what you mean by "Operating System". Lots of real time
operating systems don't have virtual memory or file systems. But I would agree
with you that the "Baking Pi" course covers little of what one would typically
associate with an operating system, it is much more of a course on low level
ARM assembly and interfacing with a frame buffer.

------
raverbashing
Will this become the Rust's era Minix (which encouraged a certain student to
do an improved version?)

~~~
pjmlp
That would be Redox.

[https://www.redox-os.org/](https://www.redox-os.org/)

~~~
drizze
Redox looks very cool!

It is very funny that their "Screenshots" tab is full of pictures of real
computer screens:

[https://www.redox-os.org/screens/](https://www.redox-os.org/screens/)

~~~
hinkley
When HDMI was new one of the things that went around gamer circles was the
fact that it was possible to do video capture on a second computer by daisy
chaining them together. That way your game isn't compromised (as much, you
probably still have higher latency) by trying to capture video.

I don't know if any modern video cards can still do that, but it might be good
for these guys to look into it.

~~~
extrapickles
You can still get HDMI capture cards as the more popular streamers use them so
their main computer is unencumbered with video mixing.

Most video cards these days have built in support for capturing video and
encoding it on the fly in hardware, with a <5% impact to performance. This
wouldn’t work for these guys as its probably a feature enabled by the driver.

------
lalwanivikas
It looks like you need to be a Stanford student to be able to watch the
lectures. Can anybody here recommend a good OS course(video preferably) for
beginners?

I am a self-taught developer and would love to learn it in my free time.

~~~
atak1
You don't really need the lectures - I took the main version (cs140) with
ousterhout. His notes are condensed and give you a good summary of the basics.
They're also open to the public:

[https://web.stanford.edu/~ouster/cgi-
bin/cs140-spring14/lect...](https://web.stanford.edu/~ouster/cgi-
bin/cs140-spring14/lectures.php)

------
polskibus
I hope they turn this course into a MOOC.

~~~
2_listerine_pls
Sadly, Stanford isn't as commited as MIT to open their courses. Only a handful
of them can be found on Coursera/Lagunita.

~~~
0xFFC
Do you know why?

~~~
polskibus
Don't they have their own edx platform?

~~~
2_listerine_pls
It's a fork of edx's platform called "Stanford Online –Lagunita".

------
g051051
I'd love to follow along, but I don't see a parts list to get the matching
parts.

~~~
woodrowbarlow
Assignment zero lists the following parts:

    
    
        1 Raspberry Pi 3 (Model B)
        1 1⁄2-sized breadboard
        1 4GiB microSD card
        1 microSD card USB adapter
        1 CP2102 USB TTL adapter w/4 jumper cables
        10 multicolored LEDs
        4 100 ohm resistors
        4 1k ohm resistors
        10 male-male DuPont jumper cables
        10 female-male DuPont jumper cables

~~~
eli_gottlieb
Oh _hey_ , so it's actually a course in operating systems combined with
embedded programming? That's really neat!

~~~
HankB99
I'm not sure I'd consider a system running full blown Linux with self hosted
development tools, graphical environment and so on to be a typical embedded
system.

Edit: I suppose they may not be running Linux on the Pi, certainly not the
graphical variants on a 4GB card.

~~~
figgis
Not sure where you are getting that info... but xfce is just above 2gb. You
could easily run a graphical desktop on that.

------
anocendi
Sweet! I have been looking for a good opportunity to build something with Rust
in a domain I already know, and this looks like a right mix of everything!

------
lgessler
See also: [http://rust-class.org/](http://rust-class.org/)

(Note though that this uses Rust as it was in 2014.)

~~~
steveklabnik
Yeah, with the caveat that Rust in 2014 was a very different language; we had
a runtime back then!

[http://rust-class.org/0/pages/final-survey.html](http://rust-
class.org/0/pages/final-survey.html) was very instructive to read at the time.

------
capkutay
Is this supposed to be a replacement for using PINTOS to build an OS in C? I
remember the PINTOS projects as one of the most rewarding things I worked on
in my college career.

[https://web.stanford.edu/class/cs140/projects/pintos/pintos_...](https://web.stanford.edu/class/cs140/projects/pintos/pintos_1.html#SEC1)

~~~
jzoch
Its an alternative. Rather than using a virtualized environment, you build
everything yourself for a raspberry pi 3. No scaffolding, nothing. and its in
Rust!

------
neduma
Interesting. Can anyone take this course or Stanford students only?

~~~
krat0sprakhar
Well, the lecture notes and assignments seem are available online, so one can
follow at your own pace

Assignment 0:
[https://web.stanford.edu/class/cs140e/assignments/0-blinky/](https://web.stanford.edu/class/cs140e/assignments/0-blinky/)

~~~
chrissnell
That's awesome. It's very well written. In just five minutes of reading, I
learned what registers and memory-mapped I/O are and how they apply to the Pi.
Old news to CS students but for this guy with a liberal arts degree, super
helpful. I'm grateful that the document took the time to explain this kind of
thing.

~~~
busterarm
Did you complete assignment 0 yet?

------
Jupe
It seems the 2 biggest hurdles of creating a new OS (aside from designing and
actually writing the damn thing) is software support and driver support...

Silly idea, but how about a "super" terminal OS... that does a few primary
things:

1\. Responsive clients for various UIs (X, VNC, RDP, SSH, Powershell, etc.) -
Having a full UI to use my smart phone would be awesome. 2\. Run VMs (which
can run docker, etc.) 3\. Uses some mechanism (wave hands here) on the remote
machines to facilitate storage, computation?, graphics, audio, clipboard and
printing on the local terminal. 4\. Strong support for a few devices. Pick a
few network cards and graphic cards to support, perhaps a few other basics,
and leave it at that...

I know it would still take years of writing a LOT of code, but this could
reduce the lack of software support. Drivers are still a sticky issue...

(Yeah, like I said, a silly idea!)

~~~
sprayk
Plan 9 sorta does 3. Can execute on one device, display on another, play sound
on two others, and store wherever. This was always my pipe dream w/ that OS.

------
_nalply
Have a look at the cheat sheet, it's extremely illustrative for intermediate
Rust programmers:

[https://web.stanford.edu/class/cs140e/notes/lec3/cheat-
sheet...](https://web.stanford.edu/class/cs140e/notes/lec3/cheat-sheet.pdf)

~~~
conradev
Here is the original link – it was made by Raph Levien:

[https://docs.google.com/presentation/d/1q-c7UAyrUlM-
eZyTo1pd...](https://docs.google.com/presentation/d/1q-c7UAyrUlM-
eZyTo1pd8SZ0qwA_wYxmPZVOQkoDmH4/edit#slide=id.p)

~~~
steveklabnik
A quick note: while this is a good way to get a grasp on how these types work,
it also exposes/implies things that are implementation details and not
guaranteed. This is why we haven't put it in the docs, awesome of a resource
as it is.

~~~
raphlinus
My feelings are hurt!

Nah, just kidding. An example of the kind of thing that might change, the
current stdlib mutex implementation currently depends on pthread mutex
objects, which can't be safely moved, so there's an additional allocation to
box them. The parking_lot crate is an alternative implementation that
interfaces with the system at a lower level, so can avoid this allocation.

I found I learned a lot about Rust's low level implementation details when
making the chart, so can definitely see how it would be useful for such a
class, and am very happy it's being used in this way.

~~~
steveklabnik
<3

By the way, I'm extremely sad to be missing your talk at the Recurse Center;
I'm in Canada until Monday :(

------
vardhanw
For anyone interested in following up on the course, I have created a cs140e
subreddit [0] as I think it will be useful for those interested. Feel free to
join.

[0] [http://www.reddit.com/r/cs140e](http://www.reddit.com/r/cs140e)

------
edcarter
Wow, I would have loved to take this class in my undergrad. This looks really
cool!

------
baumbart
Could someone explain to me how Golang would compare to Rust for this purpose?
I'm a Golang newbie and I always wondered if it would be possible to write an
OS in it (+ some assembly).

~~~
chillydawg
Golang isn't generally suited to real time or near real time systems (like
operating systems) because of its garbage collector. It will pause (not for
very long) whenever it likes to tidy up and that can cause problems when
interacting with hardware where timings are potentially critical.

~~~
baumbart
Thanks, makes sense!

------
adamnemecek
I'm pumped too see the new OS architectures Rust will allow for.

~~~
dmitrygr
Same as C. Rust is basically C with hardcoded checks enforced and put in by
compiler that good programmers already know to put in.

~~~
jimktrains2
> Rust is basically C with hardcoded checks enforced and put in by compiler
> that good programmers already know to put in.

I have a feeling you know absolutely nothing about rust.

Even the best programmers and programs in C have issues with what the borrow
checker will catch. Moreover, the expressiveness of the type system is world's
above and beyond any other main-stream (so not Haskell or ocamel) language.
Saying it's C with checks is like saying French and Hungarian are the same,
but with some different words.

~~~
cmrdporcupine
I think you are correct about Rust being substantially different from C, but
that's a nasty way of saying it ("you know absolutely nothing about..."). Is
it really necessary to speak that way?

~~~
kevindqc
It wasn't that nasty. Might just be because I have no sympathy for people
saying things like they are fact, yet don't know what they are talking about.

~~~
cbHXBY1D
Except the context of the quote was OS architecture so dmitrygr was correct --
Rust will not provide different OS architectures.

Moreover, it _was_ mean-spirited. Probably because they were reacting to the
second part of dmitrygr's quote.

~~~
steveklabnik
On a fundamental level, sure, a new language doesn't inherently make a new
architecture possible. But you have to think of it from an affordance
perspective: what architecture does Rust make _easier_ or _better_?

Consider the Stylo project in Firefox, for example. Yes, Mozilla could have
done the parallelization in C++ instead of Rust. They even tried! Twice! But
it failed both times. That doesn't mean that it's impossible.

In my experience, this is what people mean when they say things like this.

~~~
cbHXBY1D
Sure, but you're moving the goalposts :)

Look, I'm a big fan of Rust - I want to see it succeed. I go to Rust meetups
in my city, I've advocated for it's adoption in my company, yadda yadda. But I
don't these vague half-truths are good for the language or the community. In
fact, I think they'll be harmful to the community over time as the language
will fail to live up to expectations. When I started programming, Java was in
the position of Rust. It was being given so many vague platitudes that it
experienced push back a few years later as developers realized it didn't fix
all their problems.

I don't mean for this post to be mean or condescending. Tone is hard to
transmit over the internet.

~~~
steveklabnik
I don’t think it’s moving the goalposts. It’s where they were set in the first
place. Furthermore, nobody is making promises of solving all the problems, the
OP said they were interested to see what might happen, which is a very
different statement than “this will happen” or “this will happen and fix
everything.”

That is, I 100% agree with your comment, but I don’t see it happening in this
thread.

------
peonicles
Rust + Rpi interests me and I might wanna follow along. Are they using an
available kit from an online retailer ? Getting/shipping individual parts is a
pain where I'm from.

------
juanmirocks
I wonder: is there room for a new OS to truly surpass Linux/Windows ?

I would love to see more commercial success for micro/nano-kernels (vs. the
staple monolithic kernels)

------
nafizh
Can someone point to a reliable website where I can buy the required
materials? I think this is going to be a big question for beginners.

------
wemdyjreichert
Welp. I dont even go to Stanford, but applied anyway cause it looked good.
Wish me luck getting in :)

------
foobaw
I'm glad they transitioned into Rust. OS classes I've taken before were
nowhere this cool.

~~~
clw8
UV actually had a Rust based OS class four years ago! The language was at
something like 0.7 so I'm really excited to have this course with up-to-date
Rust.

------
emh68
Rust + OS design seems like the sweet spot for teaching future systems
programmers. Nice idea!

------
trisimix
Seems the chances of a non stanford student getting this class is pretty slim.
Super lame.

------
godelmachine
Can anyone from any corner of the world register for this class?

------
ihuman
Why was the title changed? It used to be "Stanford CS140e: Writing a Raspberry
Pi OS in Rust"

~~~
steveklabnik
HN usually prefers the <title> of the webpage, in this case, that's "Stanford
CS140e - Operating Systems". However, it's already a link at stanford.edu, so
the first half is redundant, but "Operating systems" is a zero-information
title, so "An Experimental Course on Operating Systems" was used instead, as
it's the sub-title of the page.

I'm not a mod, so I can't say for sure, but I bet that was the thought
process.

~~~
hinkley
Proving again that heuristics don't solve problems, they just make them less
obvious.

