When you have multiple applications running on the same machine, you need some way to safely share resources between them; for example, incoming network packets are a resource. A kernel handles this by keeping a data structure mapping sockets to processes, and demultiplexing data that comes in from the network card. Hypervisors work the same way, except at the level of virtual machines rather than processes.
Arrakis does the same thing, but relies on hardware support in the network card to dispatch packets to the right process. This relies on a standard called SR-IOV which allows the OS to configure a PCI device to present itself as multiple virtual subdevices. The kernel programs the NIC to dispatch packets to different buffers depending on the incoming MAC address; after that, packets can be dispatched with no kernel involvement at all. Similarly, you can tell a disk controller to present a particular extent of a disk as a new virtual storage device.
The blurb about memory protection seems to be a red herring, because as far as I can see they haven't done anything to change that. There's still a kernel, which handles requests for resource mappings, and processes are still isolated from each other. But once they've requested the mappings that they need, the normal execution path doesn't involve any syscalls, and so there's no kernel overhead. The real contribution of the paper is designing an API around this idea and proving that real applications like Redis can be ported to it.
Disk storage is less obvious. The LSI 2308 and 3008 controller chips probably support it, but I'm not finding a commodity card or a motherboard integrating one.
They mention in the paper that existing devices have problems which prevent them from actually securely isolating clients of the sub devices. Combine this with seeing a lot of web activity about SR-IOV in 2009 and not much now. Either it became too common to mention, or is dwindling into an idea that didn't catch on. The wait for secure SR-IOV might be interminable.
Edit: I see now it's around $500. This list of cards and motherboards using the LSI 2308 and 3008 might be a starting point for finding something less expensive: http://forums.servethehome.com/index.php?threads/lsi-raid-co...
Sure, that's true for browsers, as they mention, and a few other degenerate cases (eg. virtualization software?) - but that's certainly not the case for the vast majority of applications I run (text editor, terminal, mail client, IM client, etc.). How does this argument hold?
* The Microsoft office suite has a VB interpreter for every application,
* 3D (Blendr, Rhino) and graphics (gimp) tools have inbuilt python interpreters,
* And, it seems like there's always notes coming out from postgresql about how to deal with os limitations/gotchas
From there, the same exploits that were previously possible are possible again - they can, if they break out of whatever sandbox is in place, access everything. I guess the OS might work better for apps that don't need these rights to begin with, but then these apps usually aren't much a problem in regular OSes anyway.
For example, see the Chromium architecture: http://www.chromium.org/developers/design-documents/multi-pr...
Most of the time the os doesn't have a general purpose interpreter in the kernel even if they have an interpreter for important os functionality(sh/bash/powershell). And even then most of the time these interpreters aren't meant to be embeded in applications.
It's a fair point about postgresql but dbs that need max speed/control at times are an exception not the rule.
/ | \
Browser | System Services...
/ | \
IM EMail Terminal
I'd guess at the same time all these pieces will continue to become more distributed. Hypervisors and apps will present distributed environments (storage, processing, failover), even more than they do today.
It already does. Just write an extension and schedule a function. The one thing I don't think you can do yet is listen on a socket. At least not without doing a websocket connection to a separate service. Or bringing a native component with you.
The irony is that rather than new kernels, these are being added on as new APIs to the Linux kernel. I suppose that makes a lot of sense because it's much easier to expose a new syscall and see if it gets any adoption rather than convincing everybody to switch to a whole new OS.
Abstract of their latest paper:
Recent device hardware trends enable a new approach
to the design of network server operating systems. In a
traditional operating system, the kernel mediates access
to device hardware by server applications, to enforce pro-
cess isolation as well as network and disk security. We
have designed and implemented a new operating system,
Arrakis, that splits the traditional role of the kernel in
two. Applications have direct access to virtualized I/O
devices, allowing most I/O operations to skip the ker-
nel entirely, while the kernel is re-engineered to provide
network and disk protection without kernel mediation of
We describe the hardware and software
changes needed to take advantage of this new abstraction,
and we illustrate its power by showing 2-5x end-to-end
latency and 9x throughput improvements for a popular
persistent NoSQL store [i.e. Redis] relative to a well-tuned Linux
This is pretty concerning, actually. I don't think I want or trust shady companies like Adobe to be running DRM-laden code directly on my hardware.
Vendor lock-in is an increasingly common phenomenon and I'm picturing a really alarming future if this sort of OS takes off. I like that the linux kernel sits between my software and my hardware.
Want to watch a Sony DVD? Better hope you have a webcam so that the media player application can directly access your facial reactions to the media being played and upload it to Sony's servers.
Read this paper:
You're right, the basics of process isolation, scheduling, and resource allocation are pretty dull.
Maybe the UW project makes the move to userspace official? At 1,000,000 packets per second it is more efficient to put the whole network stack in userspace. Assume similar gains can be had for all types of hardware, and shared libraries eliminate the overhead of loading the whole network stack every time you run 'ping'.
That's just a 30-second pitch for the idea. Userspace is not new (see: Mach). I think what is changing is the average lifetime of each process. Both desktops and servers are beginning to become a commodity to the point where the old reasons for not using a microkernel are starting to become irrelevant: e.g. message passing cost isn't as much of a problem if the file server runs 'smbd' 99% of the time and that is the only really performance-sensitive application. Likewise for a desktop, except the running process is 'Chrome', or for a tablet running 'dalvikvm'.
Really it's a move away from code complexity that wasn't possible before but is becoming possible.
EDIT: Exokernel was what I was remembering (I think; https://en.wikipedia.org/wiki/Exokernel). I can see some of the advantages, but as a current maintenance programmer and previous RTLinux hacker, I have to say I have my doubts about giving most programmers direct access to hardware :) That being said, giving programmers (and admins!) easier access to containerization, virtualization, or just intra-process protection in general would be a good idea. As I've said, there are plenty of options right now, but none seem to be well documented or advertised. There was a CACM article a while back that predicted a future in which you had one smartphone, but it had split domains for work and personal apps and data. Interesting stuff, and as you say, finally becoming possible.
The big change that makes this desirable now but undesirable before is that machines are becoming increasingly single-tasked. When everybody owned a desktop and had a dozen programs on it, you needed to write an abstraction that could support all of those programs. When everybody just runs a web browser that connects up to a web server in the cloud which connects up to appservers and databases running on separate machines, each machine only needs to support one type of application. So let that app link in the abstractions it needs and only those abstractions, ensure that existing daemons don't trample over each other, and have the kernel get out of the way.
This sounds great. Do you know of any literature/writings on these kind of things for programming languages, specifically? For example the thing you mentioned about language runtimes.
How is that George Santayana thing? "Those who cannot remember the past are condemned to repeat it"?
I like seeing new ideas being tested in the OS arena. It's a real shame the two dominant OSs these days are Unix and VMS. I refuse to believe these two are the best humans can come up with.
"The good news is that in 1995 we will have a good operating system and programming language; the bad news is that they will be Unix and C++."
The predicted date was a little early, but otherwise he pretty much got it right. I think that, from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category, and Java/C#/ObjC/whatever are close enough to C++ to count.
wouldn't say so - my first Linux machine (slackware, shoebox of diskettes :) i installed exactly in the 95 :)
>from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category
while psychotropic mind-altering effects of Lisp are well known, i'd doubt that it can produce such perception changes or even in such a direction.
It sounds like the "bad old days" of DOS where games had to have their own implementation of these things.
But yeah, the game was pretty awesome, even on it's own.
I just want a layer (ie all current linux drivers) without virtual memory or a protected mode (context switches) or process management.
Just a nice API to run code on a core of my choosing, and read and write data from device "streams".
If anything should be named Arrakis it should be a terraforming project...or a worm farm.
application-specific library: Yeah, all my circles are squares, too.