
An immutable operating system - agumonkey
http://augustl.com/blog/2014/an_immutable_operating_system/
======
procrastitron
I wrote an operating system in a purely functional dialect of Lisp;
[http://losak.sf.net](http://losak.sf.net)

There are a few things I learned in the process that I think are relevant
here:

1\. Forget garbage collection in the sense of any variant of mark-and-sweep.
For an operating system latency is more important than through put, so you
really want real time memory management. If you use reference counting, then
there is a lazy variant that gives you just that; no pauses due to reclaiming
free memory. The downside with reference counting is that it doesn't work with
cycles in heap-allocated memory, but if your data is all immutable then you
can prevent cycles from being created in the first place.

2\. Treat mutable memory as a form of I/O. Even if you just want to use memory
mapped I/O, you will need some way to read and write to (some portion of)
memory. Set aside a fixed block of memory (say, for instance, the first 10
MB), and only allow that memory to be read and written in an imperative
fashion. Keep the managed heap completely separate from that mutable block,
and don't allow any direct access to the managed memory.

3\. You can actually perform preemptive multitasking entirely at compile time
(or at program load time for apps not included when you build a binary for
your OS). I've done this for both my OS and for Scheme
([https://github.com/ojarjur/multischeme](https://github.com/ojarjur/multischeme)).
It works extremely well and winds up being much faster than using the native
primitives usually used for building multitasking support. For more details on
exactly how to do it see the link to my Scheme implementation.

4\. The biggest difficulty the author will hit, and the most likely cause for
performance issues, will be in code generation. Since the proposal is for a
new bytecode, the author will have to implement their own interpreter and/or
JIT compiler for it, and writing an efficient one is an extremely large chunk
of work. I've built an operating system and a programming language, and the
language is by far the harder project. This is why I went from working on a
new OS written in a new language to re-implementing my compile-time
multitasking technique for Scheme; it allows me to take advantage of all of
the hard work people have done to make compilers for that language which
produce fast code.

~~~
augustl
Author here. The language is the part I also dread the most, and is also the
part that will be the least unique and the least interesting to "re-invent"..
I'll look into finding an existing language that runs on bare metal and that
has (or supports) immutable values.

~~~
gohrt
Haskell

~~~
chongli
There's actually been a bit of work on this very idea!

[http://programatica.cs.pdx.edu/House/](http://programatica.cs.pdx.edu/House/)

~~~
agumonkey
funny sidenote, one of House dev (Jeremey Bobbio) had a talk at FOSDEM2014
about reproducible builds for debian, similar to Nix (also immutable in
spirit) I believe.

[https://fosdem.org/2014/schedule/event/reproducibledebian/](https://fosdem.org/2014/schedule/event/reproducibledebian/)

------
mtrimpe
I gave an un-conference talk at Clojure days Amsterdam called "Purely
Functional OS" on this exact topic and we had an incredibly interesting
discussion about the topic.

The conclusion we kept coming back to is that it's technically not all that
difficult to implement, but that to make it usable in the real world would
mean that computers would have to get a lot more cautious about source vs.
derived data.

The main thing is that on the one hand it seems pretty feasible to store all
_meaningful data_ a user generates over the course of using a computer and
building an immutable persistent OS on top of that.

On the other hand though, when you think about doing this in the context of an
actual computer as in use today you will very quickly run out of storage
space.

What is separating these two scenarios is that a _lot_ of data is
(deterministically) derived from source inputs, but in the current state of
technology it's impossible to determine what is _derived_ data which can be
easily recomputed and what is _essential_ data without which the current state
could not be reached.

I happened to just be reading this article on unikernels which coincidentally
is something that would help get us a bit further along in that direction:
[http://queue.acm.org/detail.cfm?id=2566628](http://queue.acm.org/detail.cfm?id=2566628)

P.S. When thinking about this one key concept is to realize that you could can
do things like implement a 'torrent file system' where a file is just a
torrent hash which can be requested and signal when (parts of it) become
available.

~~~
sehugg
_in the current state of technology it 's impossible to determine what is
derived data that can be easily recomputed and what is essential data without
which the current state could not be recomputable._

Can you elaborate more on that statement? I see it as an expensive problem,
but not impossible.

If you have a completely deterministic VM/instruction set, you can recompute
any function output from any set of inputs. If you have non-deterministic
input (network, input devices, randomness, etc) you can store those inputs to
replay at a later time.

I don't know if it's _always_ necessary to store non-derived data. Do you need
to save all keystrokes? All mouse moves? All network packets? Probably not. If
those network packets just result in displaying a graph on the screen, they
can probably be thrown away. Maybe you can throw away the headers and just
compute based on the packet data. Or maybe you can simply store the fact that
a stream of data that matches a cryptographic hash was received. It depends on
what function receives that non-derived data.

So I think the difficulty would be designing a system that would isolate
derived data and replayable/recomputable/refetchable data, and the processes
that compute that data -- and do so with reasonable efficiency. I think this
could be done at the process level rather than the instruction set level
(though you would have to have a restricted instruction set -- i.e. limited
floating-point).

~~~
mtrimpe
At the process level there are already plenty of programs that work this way
and it's exactly the kind of thing the Clojure ecosystem is geared towards.

That's why the Purely Functional OS is essentially a thought experiment in
what it would take to extend this to the entire operating system.

When I said "impossible given the current state of technology" I meant exactly
what you said in your last paragraph though: impossible without designing an
OS that isolates derived data.

I believe such a system will come in the form of persistent data structures
coupled with reactive programming but, while I believe it's inevitable in the
long run, extending that concept all the way to the OS will be quite the
challenge.

------
stcredzero
_A garbage collector that knows that all values are immutable will be rather
interesting, I think. Typically, a garbage collector will stop the world (i.e.
halt execution) to do heap defragmentation of the old generation. When all
values are immutable, though, you can defragment the heap by copying a value
to another fragment and just swap the internal pointer to the value._

This might actually have significant performance benefits. When everything is
immutable, GC can also be inherently incremental. You just do your defrag copy
faster than new object creation. Then you cam limit your stop-the-world to a
small "catch-up" so your copy-from can become a copy-to. Incremental GC
nirvana!

~~~
RyanZAG
I don't see how immutability makes much difference to garbage collection.
Garbage collectors don't look at the value inside memory they are collecting,
only whether something live is still retaining that part of memory. Whether
the value is mutable or immutable has absolutely no bearing on the performance
or logic of the GC. Either something is still using the memory, or it is free
to collect.

Also you've just described existing GC systems - The Java G1 collector is very
similar that (it's a lot more complicated though).

~~~
barrkel
Generational GC usually depends on write barriers to detect creation of
references to newer generations inside older generations.

Immutability prevents the mutation of older generations, so no new references
can occur.

So I expect generational GC to be easier to implement. I wouldn't expect it to
be faster or slower though, because programs will need to be written
differently to cope with the limitations on mutability. O(1) algorithms will
likely be replaced by O(log(n)).

~~~
stcredzero
In the case of Clojure, log is log base 32, so it's close enough to O(1) as to
make no difference in most circumstances.

~~~
barrkel
Many immutable algorithms replace arrays with trees. Trees are pointer-heavy,
and pointers are less cache-friendly than arrays.

Memory access is often over 100x the cost of an L1 cache hit. It doesn't take
too many of those to make a big difference if you're CPU bound.

My comments are general, I'm sure Clojure has access to arrays where necessary
for interop at least.

~~~
stcredzero
Again, true, but the big problem I'm facing isn't raw performance so much as
it's guaranteed latency.

------
thinkpad20
I don't really know how feasible this is without pretty significant
performance impact (esp. memory usage), but as a research project, it sounds
fascinating. Go for it.

~~~
pmr_
> don't really know how feasible this is

And I suspect neither does the author. I looked at the previous project
(OpenGL Renderer) which comes in at 250 lines of code and barely any
functionality. The current project is has no code at all.

I'm all for sharing ideas, but this is somewhat lacking. The design document
touches on the memory model, but ignores all the other problems an OS has to
solve (scheduling, networking, disk IO...).

~~~
jmpe
The author could focus on an existing embedded RTOS and fork from there.
Enough cherries to pick one.

~~~
pmr_
But is he going to find one that follows his "all has to be immutable"
approach? I somewhat doubt it. Do you know of any existing OS that works like
what the author describes?

~~~
jmpe
The reason I posted the idea was to start with a system that is working and
where you can forego the filesystem and other parts that are typically found
on a full blown OS. It's not uncommon for an RTOS to dismiss the GC and leave
it to the programmer.

For example, there are discussions in the RTOS world to take a closer look at
Rust (Google embedded Rust) which is why the terms "new OS", "GC" and
"experimental" triggered my response to look over there for a starting point.

------
efnx
Why make the system language a Lisp instead of a language that is by default
immutable? I'd think Haskell is a shoe in for something like this and you'd
have a head start with House:
[http://en.wikipedia.org/wiki/House_(operating_system)](http://en.wikipedia.org/wiki/House_\(operating_system\))

~~~
augustl
Personally I'm more of a fan of immutability than purity, which is why I
haven't considered haskell. As mentioned in the post, though, I'm not sure if
the language should be static or dynamic or something else.. And I don't
really want to invent my own language either. Food for thought :)

~~~
codygman
Pure functions would let many of your functions become immutable :)

EDIT: Well... not really I guess. But idempotence has similar properties I
think.

------
hardwaresofton
Just curious have you started looking into some of the LISP machines of old?
You might be able to get a head start by studying their structure (and of
course their decisions):

[https://en.wikipedia.org/wiki/Lisp_machine](https://en.wikipedia.org/wiki/Lisp_machine)

~~~
_delirium
There are definitely interesting things there, but fwiw Lisp machines weren't
based on immutable data structures. Common Lisp in general has an ethos of
being multiparadigm and using imperative updates whenever convenient or
performant. Functional constructs are probably used more than in C-like
languages, but not exclusively, and are probably a minority approach in
"production" Common Lisp. That's one place where Clojure differs quite a bit
from CL; it sort of combines the "Schemey" preference for functional style,
non-destructive functions, recursion, etc., with the CL ambition of a
"batteries included" library and industrial-strength tooling.

However if you look at the first-order similarity as not being immutable data
structures, but instead the idea of an OS entirely built within a managed
runtime, the Lisp Machines and Microsoft's Singularity OS (C#) are two
interesting examples.

~~~
augustl
Op here, thanks to both of you :) I would very much like to expand prior art
to more than just Linux and Clojure.

------
cscheid
Related, a purely functional package manager, Nix, and a Linux distribution
based around it: [http://nixos.org/nix/](http://nixos.org/nix/)

~~~
agumonkey
I see a number of projects sharing similar traits:

    
    
      - DVCS
      - btrfs,zfs
      - rethinkdb
      - persistent datastructures (cons cells, fp trees)
      - nix package management, virtualenv, containers
      - react/pedestal (pushing ux delta upgrades)

~~~
cfallin
Mutability/immutability goes pretty deep philosophically... all of those
projects are basically built on persistent data structures, but persistent
data structures can be seen as just one type of "making time explicit". In
other words, with mutability, a variable (or data structure) takes different
values at different times, while with immutability, you can still have time
but you name each version through time explicitly. (Values vs. variables,
etc.) Same concept exists in e.g. single static assignment (SSA) in compiler
IR, or register renaming in high-performance processor design... pretty
powerful concept.

~~~
imsofuture
As an aside: I'm pretty mind blown that I've never once considered
immutability as "making time explicit". That's an extremely apt comparison.

------
eloff
An operating system whose main goal is not performance is forever going to be
a toy of professors. Linux is a mess compared to the beautiful code that would
comprise this OS, but it's a mess that runs fast on real hardware and let's
people get stuff done. That matters.

~~~
mhurron
Sometimes 'toy of professors' (research) is the point.

Not everything has to run a social network and not everyone is working on a
twitter clone.

~~~
jrockway
Isn't Twitter written in Scala, a programming language designed with
immutability in mind?

~~~
coldtea
Yes, but his comment was about an OS needing to be pragmatic and designed with
performance in mind (which Scala was), not about it not being immutable.

------
bjackman
Awesome! You should have a look at Urbit:
[http://www.urbit.org/](http://www.urbit.org/)

~~~
DougMerritt
There appears to be no "About" on that site. Who knows what on earth it is?

On the left sidebar, the "documentation" link is to a video. No thanks, I'm
look for a sentence of text.

The next link down is a 404 error.

The community link is for insiders...lots of comments about people leaving the
building,whatever that means.

Bottom of page, how to install. Why am I installing something when I don't
know what it is?

Sites that expect you to already know what they are about, and do not deign to
give so much as a sentence of explanation, are extraordinarily frustrating.

~~~
urbit
Sorry our site sucks. We released accidentally and are more or less in "semi-
stealth mode." You probably shouldn't be interested yet. Unless you like
playing with semi-broken stuff. When it's not broken we'll let you know.

Urbit is indeed a purely functional OS, not super dissimilar to the OP's
project. And it works at least well enough to run a chat server on its own
network (which generated all those logs with "leaves the building"). But we
ship no code before its time and we probably shouldn't even have a site up.

You'll enjoy the video though - it has good music, at least.

~~~
DougMerritt
That's fine -- but _please_ add an "About" link that says what "urbit" is.

That lack of explanation, that expectation of speaking to insiders, is
unfortunately very common on web sites, not just yours.

Please add the About explanation even if you decide to take the site offline.
At least that way, it will already be ready, later, when the project is
further along.

I was just reading about the defunct House functional OS yesterday,
coincidentally, so I'm interested in the general topic.

~~~
urbit
Thanks again for the advice.

Do you have a link to House? I wish people naming projects would realize that
the Internet is dead and all we have is the Googlenet...

~~~
DougMerritt
I totally agree, it's sad when interesting links die, and it's sad when a
search term is extremely generic.

Here's what I've got, pasted out of my notes:

House has been successfully implemented in Haskell, I'd assume that Haskell
would fit your criteria. Admittedly, House is an experimental OS, rather than
production one. But they have managed to implement everything from kernel
though network stack, to a rudimentary GUI. [link broken, see search results
below] 2005 [http://lambda-the-ultimate.org/node/943](http://lambda-the-
ultimate.org/node/943)

House is a demo of software written in Haskell, running in a standalone
environment. It is a system than can serve as a platform for exploring various
ideas relating to low-level and system-level programming in a high-level
functional language. More details are available in our ICFP 2005 paper: A
Principled Approach to Operating System Construction in Haskell.
[http://ogi.altocumulus.org/~hallgren/ICFP2005/](http://ogi.altocumulus.org/~hallgren/ICFP2005/)
[http://programatica.cs.pdx.edu/House/](http://programatica.cs.pdx.edu/House/)

terse wikipedia article on House
[http://en.wikipedia.org/wiki/House_(operating_system)](http://en.wikipedia.org/wiki/House_\(operating_system\))

short LTU thread on functional systems programming [http://lambda-the-
ultimate.org/node/943](http://lambda-the-ultimate.org/node/943)

...oh, also there's a book, "Unix System Programming with Standard ML" by
Anthony L. Shipman 2001

Was free online, link is broken, available via archive.org:

[http://web.archive.org/web/20040531113417/web.access.net.au/...](http://web.archive.org/web/20040531113417/web.access.net.au/felixadv/files/output/index.html)
[http://web.archive.org/web/20040531113417/web.access.net.au/...](http://web.archive.org/web/20040531113417/web.access.net.au/felixadv/files/output/sysprogsml.pdf)

I just found that, have not read it.

~~~
urbit
House is an OS in a very different sense than Urbit - the "boots on bare
metal" sense. The core of House is a monad which represents the hardware state
of a common Intel box.

Urbit is intended to run virtually in the cloud and is an OS only in the sense
of "stores your data and runs arbitrary programs". As a cloud computer its
"function" is simply (old state, packet in) -> (new state, packets out). For
communicating with the host OS this generalizes to "event in" and "actions
out." Eg, I got an HTTP connection so I send an HTTP response. So not only
isn't it done, it isn't even very smart... but it is a function and it is an
OS.

------
ams6110
Sort of a sidebar: The term "atom" doesn't seem to have the same meaning in
all languages.

Atoms in Clojure are reference types, and they are mutable in a controlled way
(they can be changed to refer to another immutable value).

In Erlang, an "atom" doesn't refer an immutable value, it _is_ the immutable
value. An Erlang atom by definition cannot ever be any other value.

------
obblekk
I'm not sure how this would look very different from current pure functional
languages. Think about programming in Haskell. Everything is immutable but you
can compile down to machine level binaries. So at some point the programmer is
presented with an abstract immutable model and under the hood some runtime
does real pointer arithmetic and all. Even garbage collectors could be
optimized already for knowing that the programmer has no knowledge of memory
locations.

If this kind of OS encourages hardware changes, that's one thing. But aside
from performance, it probably won't look too different from current FP
languages.

~~~
thinkpad20
He's not designing a language; he's designing an operating system.

~~~
chongli
The GHC RTS (used in most Haskell programs) is practically a mini-operating
system already.

------
timclassic
I haven't written any Clojure, but I have written a good bit of Erlang. My
understanding of the Erlang Virtual Machine tells me that it solves a lot of
the problems you describe regarding immutable data. For example, per-process
garbage collection due to immutability, message passing via copying data, etc.

I think you would be well-served to experiment with and learn some Erlang to
help inform your design. In fact, Erlang often feels to me like an operating
system due to the independence of processes (and the multitude of tools built
on top of it, but that's not directly relevant here). I've often dreamt of an
Erlang Machine like the Lisp Machines of the past.

As an aside, Erlang's primary goal is fault-tolerance. It's other properties,
such as immutable data, message passing, and functional properties were all
design decisions made to achieve this goal. My point is that your OS could be
very well suited to fault-tolerant systems.

~~~
augustl
I'm an Erlang noob, but afail the model is pretty different. I'm imagining a
shared-everything system, where immutable values are freely passed around with
no copying. Erlang is more of a shared-nothing system. Also, Erlang requires
multiple machines to shine, but an OS is more about managing one machine.

I do think message passing have a bright future, though, and my OS will not
work at all in a shared-nothing manycore system..

~~~
timclassic
I see. You are correct, Erlang definitely falls on the shared-nothing side if
you ignore some uses of ETS and ref-counted binaries (binaries above 64 bytes,
I believe).

I disagree with the statement "Erlang requires multiple machines to shine,"
but my motivations, and my definition of "shine," are likely different from
yours. Erlang only _requires_ >1 machines if you are creating a system with
proper fault-tolerance. In my opinion, it has good SMP support and it performs
well for general-purpose applications. You wouldn't use it to implement matrix
multiplication, though :)

Anyway, the gist of my post is that the lessons learned from writing Erlang
programs might be useful in your work.

Good luck! I look forward to hearing about your progress.

~~~
augustl
I once skimmed Armstrong's dissertation, it was mentioned how the JVM is bad
for fault tolerance, because it's possible for a run-away thread to break the
entire VM. Apparently, the Erlang VM manages isolation better, in such a way
that it's impossible for one process to take down the entire system. That is
definitely something I want to bring along to this OS :) I'm toying a lot with
Erlang these days, to ensure Clojure and Linux isn't the only prior art..

And you're right, "shine" is a generalization, I actually had fault tolerance
in mind when I wrote that.

------
ihnorton
_But garbage collected memory of immutable values and atoms, in hardware?_

Hardware transactional memory might help here.
[http://en.wikipedia.org/wiki/Transactional_memory#HWIMPLEMEN...](http://en.wikipedia.org/wiki/Transactional_memory#HWIMPLEMENTATION)

------
optymizer
This idea reminds me of what Go is doing with their channels to solve
concurrency issues: they're essentially sharing data by copying the data.

Copying data is slow and that's why Go's channels aren't the fastest solution,
but they're less complex than dealing with the traditional concurrency issues.

I've written a toy kernel and I just can't imagine copying all process data
because 1 little thing changed. However, there's no reason not to pursue this
idea, as it may prove to be useful in certain cases. If real-time schedulers
have their place, why wouldn't such an OS have its uses? I think Go (and other
projects) have proved the concept works, so maybe it's time to see an OS?

Best of luck to you and I hope to hear good news in the future!

~~~
masklinn
> they're essentially sharing data by copying the data.

No. Go only copies the immediate values, if you send a pointer the pointee is
shared for instance. Go's concurrency is shared-memory.

Erlang copies (almost[0]) the data sent between processes, and has process-
local heaps.

[0] binaries above 64 bytes (IIRC) are refcounted and shared.

------
RivieraKid
I've always been baffled by Clojure's view on state. It seems like just a
rename of the concepts present in most languages today. Ie atom is mutable
variable and value is an immutable value. What's the novelty here?

~~~
rplacd
The sole fact that one has to reach over themselves and make a great deal to
invoke it when thinking within Clojure's mechanics - they become neither cheap
to mentally fiddle around with, nor actually implement within.

(FWIW, though, Clojure's novelty to me was an STM system that I could get up
and running without a night's worth of tweaking, and otherwise fall safely
into a great deal of attention if something were to go hilariously wrong -
most of the other solutions around "back then" seemed to regard themselves as
research curios lagging behind in fit and finish.)

------
mostly_harmless
You mention that one of the benefits would be that memory can be shared across
processes without any defensive copying or protection semantics, but isn't the
entire idea that immutable values _are_ a special type of defensive copying
and protection semantic?

I imagine that this would be useful if doing many calculations on readonly
values, but would take a performance hit for readwrite operations.

I'm not one for purely functional programming, but this seems like an
interesting concept. I'm interested to see where it goes

~~~
augustl
Interesting definition of immutable values, and a valid one. My perspective is
different. An immutable value has a value add more than that of avoiding
defensive copying, such as structural sharing. When you add a key to an
immutable map, the new map can point to the old.

It has become more and more common to model systems with append-only as the
only form of write (Event Store, Datomic), so readwrite can probably be
avoided in some cases. And things like IP packets certainly makes sense to
represent immutably - anything else is a bit of a lie :) But there is
certainly a risk that the trafeoff of immutabulity becomes too costly for an
OS..

Also, the system language will not be pure, like Haskell. Just immutable
values, like Clojure :)

------
puredanger
Maybe consider Fressian for EDN-like bytecode format...

~~~
augustl
Ah, thanks. When I said EDN, I meant Fressian :)

------
tkirchner
I really doubt the practical usefulness of such an operation system other than
pure fun and skill development (and maybe some tools that fall out at the
end...).

Isn't the main purpose of an OS to bridge between the applications and the
hardware? The hardware is pretty accurately abstracted as sth. with a state
and input/output streams.

What is the advantage of abstracting the hardware as sth. without a state?

------
tkirchner
I really doubt the practical usefulness of such an operation system other than
pure fun and skill development (and maybe some tools that fall out at the
end...).

Isn't the main purpose of an OS to bridge between the applications and the
hardware? The hardware is pretty accurately abstracted as sth. with a state
and input/output streams.

What is the purpose of abstracting the hardware as sth. without a state?

------
tkirchner
I really doubt the practical usefulness of such an operation system other than
pure fun and skill development (and maybe some tools that fall out at the
end...).

Isn't the main purpose of an OS to bridge between the applications and the
hardware? The hardware is pretty accurately abstracted as sth. with a state
and input/output streams.

What is the purpose of abstracting the hardware as sth. without a state?

------
DanBC
Is there a list of research / alternative operating systems anywhere?

I don't know how an OS would qualify to get on the list other than not DOS,
Windows, OSX (or any of the Apple OSs), Linux (especially not 'ubuntu with a
different DE'), or the BSDs (i guess there are exceptions if they're on a
toaster).

~~~
FrozenCow
I was making a list of a few I knew, but it would be better to just check
wikipedia:
[https://en.wikipedia.org/wiki/Comparison_of_operating_system...](https://en.wikipedia.org/wiki/Comparison_of_operating_system_kernels)

Those are more serious kernels I think. There are also a ton of research-
related kernels. I guess you can find a few for some programming languages.

------
iElectric2
We already have a purely functional linux distro that is immutable and
stateless, called NixOS

------
ij2bun
'Clojure has transients, which basically means "inside this function, use a
mutable value under the hood while building up the value, but make it
immutable before it's returned"' is this essentially a ... closure?

~~~
augustl
Transients in Clojure is an added feature to the built in immutable values.
Essentialy, they are immutable. But Clojure does have mutable-ish versions you
can use to build up a semi mutable transient, and when you're done, convert it
into an immutable value before it's returned.

The key is to make it fast. Mutable -> immutable is not an O(N) copy.

------
unfamiliar
Can someone tell me where I can find a primer on immutable vs mutable, state
etc? I have tried to piece together an understanding from wikipedia etc but
I'm not really "getting" it.

------
tekknolagi
So how do you load this into a VM?

~~~
augustl
The makefile builds a virtualbox image. But it doesn't do anything interesting
yet. I didn't submit to HN, the project is a long way away from being demoable
and testable.

------
billsix
how would a user interact with such a system?

