
Why do we need a new OS? - thecombjelly
http://3lproject.org/blog/why-do-we-need-a-new-os
======
unsatchmo
"Windows, Linux, Mac OS X, and the BSDs we have today are still built using
the architecture developed when computing resources were limited even though
it is no longer technically necessary."

Nonsense! Why would you waste any resources on infrastructure when you could
use them for applications. Every cycle that your garbage collector runs is a
cycle that the cpu could have been sleeping and saving battery life. This
concept that we don't need to care about resources anymore is absolute
poppycock.

~~~
thecombjelly
> Nonsense! Why would you waste any resources on infrastructure when you could
> use them for applications. Every cycle that your garbage collector runs is a
> cycle that the cpu could have been sleeping and saving battery life. This
> concept that we don't need to care about resources anymore is absolute
> poppycock.

It isn't that simple. Depending on your program it can actually be slower to
do manual memory management. malloc isn't free. In some cases allocating
memory and performing GC is actually faster.

It's not that we don't have to care about resources at all just that we can
sacrifice some of the resources to improve the situation for users and
programmers. If that isn't your thing then feel free to write everything in
assembly.

~~~
unsatchmo
This isn't an either or. The reason your phone gets all day battery life is
because the kernel was designed to run well on an i386. Let the app frameworks
have cushy memory management but implement the OS in C.

~~~
thecombjelly
> The reason your phone gets all day battery life is because the kernel was
> designed to run well on an i386

This is false. The reason your phone has the battery life it does is because
the OS and libraries and applications have been written to allow the hardware
of the phone to drop in to lower power states as much as possible. Unless the
language you have written your OS in has some bizarre requirement to wake up
all the hardware constantly then the language doesn't matter a ton when it
comes to battery life. What matters is writing programs that wakeup the
hardware as infrequently as possible.

------
sedatk
The article is full of gold:

\- OS's were designed for limited resources back in time. Now it's time to
design something for unlimited resources.

\- Attempting to retrofit a new security model to a current OS would require
everything would be rewritten so we decided to rewrite an OS, which will
require everything to be rewritten for it, including the OS this time.

\- Sandboxing, local-scope FS access, user controlled permissions have never
been done before, so we're doing those.

\- Runtime analysis will give out better performance

\- Compilers have always been hindered by OS design, so our new OS will break
the barriers of compilers (finally! )

(and more!)

~~~
eloisant
When they say we no longer have to design for limited resources, I really want
to know what we can gain from dropping that constraint.

Do I, as a user of the OS gain anything, or is it just easier for the kernel
developers to not have to care about that?

~~~
ksec
The Desktop are like unlimited resources, that is true to a certain extent.
But the design has been switched to Phones in recent years, and they are
limited. And we will move to Watch soon, which is even more limited resources.

Not to mention we are bound by Battery technology. In that sense, we are still
very limited in resources.

~~~
thecombjelly
Very true. When it comes to the resource intensive development features in 3L
they can actually be disabled. But also this is partially a bet on the future,
a bet that hardware resources will continue to become less constrained at all
levels. Of course that is not for sure when it comes to that but it seems more
likely than not that we will continue to advance in some ways. It's important
though to start work now so that we are ready when the time comes.

Regarding batteries, the real issue there is how often you wake up the
hardware from low power states. An OS like 3L could actually control that
better than what we have today and give us better battery life.

------
nickpsecurity
Very interesting project, Thomas. Here's one for inspiration:

[http://www.symbolics-dks.com/Genera-why-1.htm](http://www.symbolics-
dks.com/Genera-why-1.htm)

About time we have something modern that can do everything in that ad that's
relevant today. I want my live-updating, live-fixing, per-function compiling,
whole-system debugging, crash-resistant OS. If they could do it in the 80's
then... you know... ;)

~~~
unsatchmo
This is the best argument for a High Level Language OS that I've seen yet.

~~~
nickpsecurity
Thank you!

------
panic
Does the language-based security model of 3L mean that all programs must be
written in Lisp? Single-language systems tend to miss out on a lot of cross-
platform software (which is typically written in or bootstrapped using C).

~~~
thecombjelly
(creator of 3L)

No. Almost any language we use today can run on 3L and transpilers are planned
for the future of 3L. The security features will still be provided via the
underlying lisp though. Technically the same security mechanism could work for
most languages but it is much simpler, easier, and more natural to implement
in lisp.

Even languages like C can be supported safely via memory emulation (although
right now that would mean it runs quite slowly but likely, although no
guarantee, it will run well enough in the future as hardware becomes more
powerful).

------
rodgerd
So OSes designed in the 70s are bad[1] but we'll use a language designed in
the 50s because it's good.

Old stuff is bad except when we like it, I guess.

[1] And quite dishonest to lump WinNT in with that...

~~~
thecombjelly
OSes designed in the 70s aren't and weren't necessarily bad. It just depends
on the circumstances. When they were written they fit the times quite well.
But things have changed since then.

Lisps have also changed since it was first created. Unlike many languages lisp
has evolved tremendously over the years and has benefited from a tons of
language research. The lisps we have today are much more than we had in the
60s.

~~~
TheCondor
Do you have any concrete examples of things that no longer fit well?

------
thecombjelly
I'm the creator of 3L so feel free to ask me anything or point out how it will
never work and is a terrible idea and is nothing new. :)

~~~
quartzli
What is the reason you decide the "Fully Source Mapped" feature? What's the
advantage and disadvantage? Thanks.

~~~
thecombjelly
The only major disadvantage is it takes time to develop.

The advantage is it makes development of the OS easier and application
debugging easier. It is also makes future proofing the OS more robust. A
program that can rewrite or inspect other programs is very powerful, in the
same way lisp macros are.

One concrete advantage of this is you could write a debugger as a user
application, it wouldn't have to be integrated in to the language runtime.

~~~
hvidgaard
> A program that can rewrite or inspect other programs is very powerful

And is a security nightmare. If I have an executeable, I have verified, I
don't want the instructions to be altered in any way. In fact I prefer if it
is impossible from a hardware level by marking them as executeable bits.

~~~
thecombjelly
This still falls within the 3L security model though. A program can only do
that if you, the user, allows it.

Also, 3L runs in two modes: development or secure. In secure mode you can
completely disable manipulation of source code (or strip it out) and mark all
code as read only and have the system verify the code hasn't been modified
when it is run.

~~~
hvidgaard
But how is that different than what we have today? Windows for instance
already does this for system files, and could be expanded to user installed
files as well.

You still need a debugging build, otherwise compiler optimizations will
obscure the executeable to the point where debuggers cannot reliably debug it.

------
eklavya
I have always been curious about Lisps but the lack of static typing turns me
off. Sure you can have a type system (typed racket) but it doesn't feel
organic, also then why not use any other statically typed language (haskell,
scala, ..). With all the hype (very likely well founded) surrounding Lisps, I
feel like I am missing out.

~~~
efnx
House, an experimental Haskell operating system is exactly that. If I had a
lot of time on my hands I would love to revive that project.
[https://en.m.wikipedia.org/wiki/House_(operating_system)](https://en.m.wikipedia.org/wiki/House_\(operating_system\))

~~~
tathougies
(Shameless plug follows)

Take a look at my own Haskell operating system project Hos
([https://github.com/tathougies/hos](https://github.com/tathougies/hos)).
Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it
easy to generate ANSI C from Haskell sources). Currently supports multi-
tasking, interrupts, paging, garbage collection. Has the beginnings of a
simple user-space ATA driver.

Patches welcome. E-mail me at travis@athougies.net if you want help setting up
a development environment!

~~~
iso-8859-1
Would be nice if you could get Hos listed on
[https://wiki.haskell.org/Applications_and_libraries/Operatin...](https://wiki.haskell.org/Applications_and_libraries/Operating_system)
...

------
jorangreef
Too much network and filesystem data moves through an OS before the user
application can work with it. As data bandwidth increases, this becomes a
bottleneck. It would be great if an OS could push this out to userspace and
focus on zero-copy stacks for network and disk.

~~~
thecombjelly
3L does do this. You can share as much data between the OS and user
applications as you want. The OS doesn't really mediate anything so there is
no overhead from copying data or messages between processes or enforcing some
security mechanism. The language runtime enforces the references that are
available to some code that gets run but that's it and that's all that is
needed. You really don't need a separate OS program facilitating everything to
serve as an OS.

~~~
jorangreef
Thanks, I looked through the 3L website and features section but didn't
realize that from there.

~~~
thecombjelly
Yeah sorry my fault really. I'm working on improving the organization of the
site and I haven't had near enough time to write everything down in an
intelligible fashion. Most things are still in my notes or waiting to be
extracted from various papers.

------
thegenius2000
I know (with software) to never say never, but to be quite frank, I'm content
with the current Unix paradigm, and I haven't seen a compelling reason why any
new OS would be fundamentally superior.

But I would absolutely love to be proven wrong...

[http://www.catb.org/esr/writings/taoup/html/plan9.html](http://www.catb.org/esr/writings/taoup/html/plan9.html)

------
onre
"Another change since the original OSes were initially being built is the need
for security"

I do not know what's referred to as "original OSes" but many old operating
systems had actually pretty decent security architecture. MULTICS and VMS, for
example, and many others too.

~~~
thecombjelly
It's true that some did. I was more referring to the history of most of the
major OSes we use today and specifically when it came to the first
microcomputers. Generally when they were first being used (and the OS was
being expanded) security was not much of a concern.

------
bradley_long
Keep going! To make a new OS successful, there are too many difficulties.
Existing competitors, acceptance of consumers and cost of shifting from the
old to the new. We are happy to see a new OS. Hope it can go well!

------
peterburkimsher
There is a market for a new OS, but security alone is not enough reason to
make people switch. Supporting Unicode would affect users a lot more. I wish I
saw an OS with good support for Chinese.

~~~
DigitalJack
I would have thought the Chinese would have built one.

------
abstractbeliefs
Fascinating project.

I'm glad that alongside the philosophy of this being a FOSS endeavor, there's
the practical aspect of this being adjustable on the fly by users in the old
Lisp machine tradition.

Though I'm sure it's too early for outsiders to shape the PoC code, is it
available anywhere for us all to follow along?

~~~
mchahn
> there's the practical aspect of this being adjustable on the fly by users in
> the old Lisp machine tradition.

Is there the possibility that the OS code will be so mutable that it is
impossible to predict if an application will run on any one installation? Will
there be any posix-like standard to guarantee compatibility?

------
bronlund
Yes! Please make it so you don't have to be time travelling cyborg in order to
sort files the way you like. You have files right?

~~~
thecombjelly
The "In Depth"[0] page addresses that a tiny bit.

In summary, 3L drops explicit support for filesystems. Instead the system just
provides a mapping of labels to storage blocks. From there those blocks can be
utilized in any fashion you like. A filesystem library can be written to
manage its set of blocks as it pleases. A database can skip the overhead of a
filesystem and directly access the blocks to improve performance.

But yes, assuming you have imported the filesystem library (which has probably
already been done assuming you are at the REPL) then you can process files as
regular lists of lisp data (or in whatever language you want really). Not sure
that is really that special though since you can already do that.

[0] [http://3lproject.org/in-depth](http://3lproject.org/in-depth)

------
Yaggo
Entirely new OS would be a waste of resources, but some big player could
donate few millions to boost the development of Wayland. What the markets
lacks is a system with openness / hardware support of Linux combined with
solid graphical layer of OS X. Or maybe AOSP could transform into true open
source (culturally) project?

------
hasenj
Resources aren't really that cheap. Just try running a video game on a Macbook
Air. It doesn't work.

~~~
quartzli
@thecombjelly Can programmer "touch" PreScheme in 3L?

~~~
thecombjelly
Right now, not really. PreScheme currently compiles to C. Eventually the goal
is to create an assembly backend and if that were combined with incremental
compilation features you could manipulate it at runtime to some degree.

------
alexandrerond
When I read this I raised my eyebrow. I hardly think that Lisp (1958) can be
the base of the operating system of the future. Their texts are full of bold
statements about how easy and natural lisp is and how everything else sucks.
Nevertheless it's interesting to look at their references:

[http://3lproject.org/references](http://3lproject.org/references)

I understand than in short, the security model is about how to make sure a
process does not do more than what it's supposed to. In this case it's lisp
environments that provide the necessary sandboxing.

~~~
na85
That Lisp began in 1958 is mostly irrelevant. It (Common Lisp) is a fully-
modern, Turing-complete language with a first-rate object system and supports
all programming paradigms.

On top of that the implementations have all been regularly updated over the
years and many are extremely mature and well-optimized.

Do you equate newness with quality, as seems to be common on HN?

Perhaps an OS written entirely in this month's JavaScript framework, deployed
on Docker, using monads-as-a-service or whatever the hipsters have come up
with lately would be more your cup of tea.

~~~
rodgerd
> Do you equate newness with quality, as seems to be common on HN?

When the link's thesis is "these OSes design years ago are old and therefore
bad" it is hardly unreasonable to point out the language the author prefers is
even older.

~~~
pjmlp
Because the UNIX based OS are based on PDP-11 architecture with the
compromises required to fit in such systems.

Whereas already in the 60s, we had hardware systems like the Burroughs b5000
that could be fully programmed in Extended Algol and Espol, with zero
Assembly.

The idea, as I see it, to base modern OSes in better older architectures that
were lost to the mists of time.

Actually the mobile OSes, with their memory safe languages seem to already be
moving into this direction.

