
Lisp Operating System (2015) - antipropaganda
http://metamodular.com/Common-Lisp/lispos.html
======
heisig
Summary:

\- We don't need separate processes if the language we use is inherently safe
(e.g., modern Common Lisp).

\- Without process isolation, it is possible to share large, complicated and
possibly mutable data structures (graphs, arrays, user-defined objects)
system-wide.

\- Once it is possible to cheaply share and communicate arbitrary data
structures, it is pointless to maintain designated 'file system'.

\- Not having a file system raises the question of what data should be
persistent and what shouldn't. But on a modern computer, it is feasible to
just treat all data as persistent (maybe excluding the youngest GC
generation).

The best news is that the author is actually working hard to implement this
operating system. The first part - the Lisp implementation - is already in a
pretty good shape and could be finished within the next two years:
[https://github.com/robert-strandh/SICL](https://github.com/robert-
strandh/SICL)

~~~
magicalhippo
> We don't need separate processes if the language we use is inherently safe
> (e.g., modern Common Lisp).

I note the year of the article, which was before Spectre and friends showed
that this is not a reasonable assumption these days[1].

So does this mean LispOS is doomed?

[1]: [https://arxiv.org/abs/1902.05178](https://arxiv.org/abs/1902.05178)

~~~
rjsw
If you have a single address space you don't need Spectre to be able to
discover any aspect of the running system.

~~~
admax88q
Only if you're given raw memory access, which you are not in Lisp.

~~~
sametmax
Without raw memory access, good luck supporting anything than basic hardware.
Even some USB features requires it.

~~~
msla
It's entirely possible to write a system where application code only gets
sanitized handles and OS code inside the implementation gets raw memory
pointers at least some of the time. All running in one address space doesn't
mean all running in one security context, necessarily.

Really, address spaces are just a hardware implementation of a more general
concept: Namespace-based security. Application code wouldn't even be able to
know the names of privileged objects, and if someone told that code the right
names, it wouldn't be able to use them, because resolution of names to things
is, itself, privileged.

In a simple example, assuming a Common Lisp-like system: Everything which
handles raw pointers is in the SYSTEM package. Application code can't inspect
the SYSTEM package, and, even if you told an application that the function
SYSTEM:WRITE-DATA-TO-DISK was a thing, it couldn't call that function because
the real evaluation code, which _can_ see into the SYSTEM namespace, knows not
to let application code call anything in SYSTEM; only the functions in SYSTEM
and SYSCALL can do that.

------
neilv
For the incremental development the author describes, you can do that atop any
GNU/Linux host platform on which your CL implementation runs.

To play with a bootable user experience, before you've replaced the host
kernel and device support, you can strip down something like Debian Live. I
previously did this for a Racket-based living room appliance project. A
simpler example, without Racketisms, that uses only 2 shell scripts (you only
need 1; the other is to play partition table tricks), is my earlier Debian
Live variant:
[https://www.neilvandyke.org/lildeb/](https://www.neilvandyke.org/lildeb/)

I'm also a fan of a variation on the author's approach: have your
better/different language get a foothold on a working GNU/Linux system,
_alongside traditional apps_ , in a system the user could use as their daily
driver, and incrementally replace all of userland. This was my plan for
PostmarketOS (though I've suspended my open source work on "non-employable"
things):
[https://www.neilvandyke.org/postmarketos/](https://www.neilvandyke.org/postmarketos/)

The author's approach also seems valid, and worth playing with.

~~~
TeMPOraL
> _I 'm also a fan of a variation on the author's approach: have your
> better/different language get a foothold on a working GNU/Linux system,
> alongside traditional apps, in a system the user could use as their daily
> driver, and incrementally replace all of userland._

That's incidentally what Emacs is to many of its users (myself included).

~~~
neilv
I used to do this with Emacs, too (VM, Gnus, w3m, and more besides all the
programming tools), and some of my Emacs packages are in support of that:
[https://www.neilvandyke.org/emacs/](https://www.neilvandyke.org/emacs/)

For a (mostly) single host process, without preemptive multitasking/threading,
Emacs as an operating system works surprisingly well.

------
pushpop
I’m all for thinking outside the box but many of the complaints described in
that are blamed on UNIX when in fact they were also adopted by a multitude of
other non-UNIX platforms and reason they became common was because it simply
made more sense for general computing.

For example their “file system” (for want a better description) approach was
tried on an IBM mainframe and it proved more cumbersome to maintain than the
typical file system approach we have currently.

Also when talking about address spaces he acknowledges traditional memory
management exists for security reasons then says that’s not needed in his OS
without actually proving how his method either prevents the same attacks nor
new ones that effectively result in the same kind of vulnerability (I could
think of a few very easy attacks against his OS design).

Similarly the way he combines physical storage and system memory into one
device completely overlooks technical limitations of the hardware, the risk of
data loss if you don’t manage things correctly, the vastly different storage
capacities between the two, nor the fact that basically every OS does this to
some extent already via caching.

While it’s nice to go back to the drawing board and rethink certain idioms,
this felt more like a half baked rant at UNIX than an objective look at the
past and current OS designs.

~~~
hnlmorg
That was my take away as well. Some interesting ideas - most of which aren't
new - but the ideology felt very biased. To a fault even. I would still love
to see a working version of this. Or even just a more traditional general
computing OS but one which used FP (not necessarily LISP) as its primitive.

~~~
TuringTest
This article I read yesterday has one interesting take on what an OS designed
on modern programming principles could look like. It has a loose treatment of
technical FP terms, but the insights behind it are at least thought-provoking.

[https://medium.com/@reinman/monoids-to-
groupoids-492c3510511...](https://medium.com/@reinman/monoids-to-
groupoids-492c35105113)

I believe we're in for a redesign of OS architectures, now that the 50's
assumption of a single machine with a single OS kernel running all programs
(or the 60's equivalent of several few mainframes communicating through a slow
network) is no longer the main paradigm.

~~~
all2
The article reminds me of what Erlang is meant to do. With pure message
passing wouldn't the need for some foo() calling some bar() be down to
shooting off a message?

------
shalabhc
Great write-up and I agree with many points. I don't think the 'Lisp language'
is the key point here though, but the other OS properties like a single level
store and composition methods richer than byte arrays.

Would be nice if the author referenced historical systems that overlap these
ideas.

One is the Symbolics Genera system with 'data level integration' so apps
shared rich data structures instead of uninterpreted bits.
[https://www.ifis.uni-luebeck.de/~moeller/symbolics-
info/gene...](https://www.ifis.uni-luebeck.de/~moeller/symbolics-
info/genera/genera.html)

Another is Appolo Domain/OS which had a single level store:
[http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_...](http://bitsavers.org/pdf/apollo/014962-A00_Domain_OS_Design_Principles_Jan89.pdf)

~~~
twic
Yes, absolutely astounding that this article doesn't mention Genera.

If we're going to proceed to list every single address space operating system
we've ever heard of, i'll mention SPIN, written in Modula 3 (although i don't
know if it did anything clever about sharing data between applications):

[http://www-spin.cs.washington.edu/external/overview.html](http://www-
spin.cs.washington.edu/external/overview.html)

------
ngcc_hk
Whilst difference is good, I am not sure many of the “attack” on the process
etc is right. Whilst you can bring anything on old hardware (car and cdr come
to mind and those strange intel register format), some gem can be argued
instead of rubbish. Process is an isolation, small address is easier to debug
if you do not need 64bit address etc. And driver issue ... lisp driver.

Really not sure the whole game.

------
GorgeRonde
Makes me think of loper os: [http://www.loper-os.org/?p=4](http://www.loper-
os.org/?p=4)

------
rakoo
The criticisms may or may not be correct, but I fail to see why a Lisp (and
not another language) would be the answer to all of the issues here.

~~~
heisig
The points for Common Lisp are:

1\. It is ANSI standardized and mature.

2\. Common Lisp programs are safe. Some implementations violate safety for
performance reasons, but this one certainly won't.

3\. Common Lisp has the best error handling mechanism that I know of
(Conditions, handlers and restarts).

4\. The language is programmable, both at the syntax level (Macros) and at the
object level (CLOS with the metaobject protocol). So new ideas from other
programming languages can often just be ported.

5\. The language has been designed for interactivity and safe incremental
redefinition.

6\. Lisp has already been used very successfully for operating system
development
([https://en.wikipedia.org/wiki/Lisp_machine](https://en.wikipedia.org/wiki/Lisp_machine)).

This makes it pretty much the ideal choice for such a project.

------
effie
This document talks a lot about deficiencies of the unix way, yet fails to
give any concrete examples of advantage of the proposed new way where
everything runs in the same address space, uses object storage, and is
allowing users to run (!) _" only code that has been produced from the high-
level notation of the language"_.

I agree that there are technical problems with interoperability of programs
and partially this may be due to the unix way. But the article does not
analyze this and it is not clear that such a radical change is a good way to
solve those problems. Even if theoretically yes, there are practical things to
consider, such as the insane amount of work needed to switch. Perhaps the
author just likes Lisp and has a solution in need of a problem.

------
newsoul2019
Does anyone remember the TI Explorer?
([https://en.wikipedia.org/wiki/TI_Explorer](https://en.wikipedia.org/wiki/TI_Explorer)).
They were sold in the 1980s, I first saw one in person in 1993. It was
astonishing back then that it had an optical mouse and an optical cable
connecting the keyboard to the rest of the unit. My understanding is that it
was a full stack LISP machine, down to CAR/CDR support in hardware.

------
mwkaufma
How does failure-recovery (software bugs, hardware, or otherwise) or
resilience-engineering more generally work on a system that demurs isolation
and redundancy (honest question)?

------
ryanolsonx
They spelt emacs wrong

~~~
sagartewari01
They didn't name anything, they just provided a description.

~~~
hnlmorg
I assumed his comment was a reference to the old "joke":

> _" Emacs is a great OS, but it lacks a decent text editor."_

The comment presumably made because emacs is extended with LISP.

~~~
somatic
Emacs is written in Emacs LISP, which is its own language. The really
performance-critical stuff is written in C, as is the interpreter, but that’s
it.

~~~
hnlmorg
That's ostensibly what I said wasn't it? Sorry but I can't tell if you were
agreeing or disagreeing with me.

~~~
somatic
I interpreted your comment to mean something like “Emacs is configurable with
third-party LISP code”, in much the same way as Vim is configurable with
third-party Vimscript.

But Vim is written in C, Vimscript being just a bolt on, whereas Emacs LISP is
its own language entirely, a distinct variant of LISP, and much of Emacs
implemented in it.

~~~
hnlmorg
LISP is a family of languages (much like BASIC). So Emacs Lisp is still a
LISP.

I don't even know why you're bring Vim into the discussion when it has nothing
to do with the topic being discussed. It doesn't invalidate the statement I
made; I just makes your reply look a little snobby because you seem concerned
that someone might compare Emacs and Emacs Lisp to Vim and Vim Script (I
clearly wasn't given the complete lack of reference to Vim in my post).

To be honest I thought we had gotten past those tedious flamewars in the 90s.

------
hardtoreason
I literally stalled on the first sentence where he asserts without any
rationale that an OS written in Lisp "would be a good thing in itself".

WHY

WHY WOULD ANYONE THINK THIS MATTERS

~~~
GrinningFool
Conversely, why would anyone think that because a thing doesn't matter to
them, it shouldn't matter to anyone else either?

~~~
AnimalMuppet
I think that if you were to poll the users of an OS about what matters to
them, the language it was implemented in would be at the absolute bottom of
the list.

It may matter to the author of the article. That's fine. To the users? I'm
pretty sure that hardtoreason correctly reflects their attitude.

And if you're writing an OS, but don't care about the users? Well, write it
however matters to you, because you aren't likely to have users who care,
because you aren't likely to have users at all.

~~~
yellowapple
> And if you're writing an OS, but don't care about the users? Well, write it
> however matters to you, because you aren't likely to have users who care,
> because you aren't likely to have users at all.

Counterpoint: Windows 10.

~~~
AnimalMuppet
Touche. (Though even with Windows 10, my larger point still stands - nobody
cared what language it was written in.)

