
Docker Acquires Unikernel Systems as It Looks Beyond Containers - amirmc
http://techcrunch.com/2016/01/21/docker-acquires-unikernel-systems-as-it-looks-beyond-containers/
======
nickpsecurity
I was thinking: Docker. Hmm. Containers. Hmmmm. Xen developers. Hmmmmm. Seemed
really boring until I saw " Anil Madhavapeddy, the CTO of Unikernel Systems."
Oh... I know _that_ name: it's on quite a few IT/INFOSEC papers I stashed and
shared over the years. A smart researcher with a practical focus. Didn't know
he was CTOing at a startup.

Yeah, Docker is about to get some enhancements for sure. Maybe some real
security improvements, too. You can count on it.

~~~
bcg1
[https://twit.tv/shows/floss-weekly/episodes/302](https://twit.tv/shows/floss-
weekly/episodes/302)

~~~
ZenoArrow
Great video. I follow the Mirage project already, but I thought Anil explained
it very well, and some good questions from the hosts too.

------
hacknat
I used to be in the unikernel camp of "this is the next step in virtualization
tech", but having played around with both containers and unikernels, and now
developing with containers, I think unikernels are going to occupy only a very
niche space.

There are two touted benefits of unikernels, performance and security.
Performance turns out to be a red herring, as the overhead of an OS vs a
Hypervisor turns out to be roughly equivalent (with the OS actually winning in
some use cases).

Security is definitely an issue, but it's so abstract. My company is a
compliance (a very specific industry's compliance) cloud provider and we have
gone with Docker as we get to use the OS as our Hypervisor, which means it is
much more extensible and, in our use case, secure as we are able to auto-
encrypt all network traffic coming out of the hosts with a tap/tun virtual
device.

Two things need to happen to make unikernels attractive. A new Hypervisor
needs to get made, one that is just as extensible as an OS around the isolated
primitives. It should also have something extra too (like the ability to fine
tune resource management better than an OS can). Secondly a user friendly
mechanism like Docker needs to happen.

~~~
Pxtl
Wouldn't the unikernel security benefit be the same for a lightweight OS where
the libraries/services had been carefully pared down to the bare minimum for
the application?

It sounds like the main benefit to the Unikernel security is the implicit
audit of all the services you want to roll into the kernel down to the bare
minimum. I imagine that could be done with a more conventional architecture,
it's just that nobody ever _does_.

~~~
talex5
It depends on which lightweight OS and which unikernel. But e.g. a stripped
down Linux will still have a huge amount of C. If you're going to write your
kernel in something safer, then you might as well make a unikernel, rather
than creating a kernel/userspace split.

It's hard to see how you'd get a traditional OS stripped down anywhere close
to e.g. the mirage-firewall unikernel
([http://roscidus.com/blog/blog/2016/01/01/a-unikernel-
firewal...](http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-
qubesos/))

~~~
EvanPlaice
Not necessarily.

NodeOS cut out everything but the absolutely essential parts of the linux
kernel. No c libraries, no c compiler, no POSIX utilities, no user space, etc.

Instead, everything runs on V8 (ie which also takes care of sandboxing) and
minimal tools were rewritten in pure javascript, incl a git clone tool.

It's amazing how much you can cut out and still have a decent platform to
build servers.

~~~
willtim
What's next PythonOS, RubyOS? Just learn OCaml, its really not that hard.

~~~
Elv13
In itself, beside being a ridiculous amount of work, it would be a good thing.

If all apps ran on its own hypervised runtime instead of a "native" runtime,
the world you be a better place.

The overhead of starting a VM is actually in the 2 digit milliseconds and can
be done while the initial I/O are performed.

The XDG-App project is trying to make it easier to sandbox Linux desktop and
server applications by defining all the interfaces between the sandboxes and
the OS. This work, while initially designed for containers, will eventually
work on VM too.

------
btilly
Hopefully Docker's paternalistic attitude doesn't infect Unikernel systems.

I'm not very hopeful given that their CTO is quite open about wanting to
embrace-extend-extinguish competing technologies. This move embraces
unikernels, and now they are perfectly positioned to go the rest of the way.

The discussion at
[https://news.ycombinator.com/item?id=10904452](https://news.ycombinator.com/item?id=10904452)
may shed light about my complaint.

~~~
seiji
We're in a new weird technology landscape where everything is driven by pride
and ego and money instead of "what's best for everyone" progress or even
stable technological advancement.

Things like Docker and systemd are exactly EEE land grab lock-in plays to
redirect "open source" value back towards private entities and unilateral
control structures (instead of cross-vendor standards bodies, because
standards bodies can't make individuals into billionaires).

How do we as lowly developers with effectively $0 net worth fight companies
with billion dollar war chests? Over time, better _does_ win (see: the
collapse and accelerating irrelevance of Microsoft), but in the short term
it's likely we'll get stuck in another 10 years of platform "dark ages" until
we see the light again.

~~~
scaryclam
With systems like Docker we're just making sure that we don't lock ourselves
in. If you decouple deployment and your code (which you shouldn't have coupled
to a container anyway) from how you run it (containers, a VM, etc) then
there's no problem moving to or away from systems like Docker. This is how
developers can make sure they're not getting the raw end of the deal, and
makes for a much cleaner setup anyway.

------
pjmlp
OCaml and MirageOS FTW!

I hope this brings Unikernels adoption forward, and boosts a little more
OCaml's adoption as well.

~~~
nickpsecurity
Exactly. They've been doing exemplary work. I especially like that, upon
hearing of language-related obstacles, they try to actually fix or work around
them instead of throwing up hands and defaulting to C or C++.

------
chatmasta
Docker has some really smart people leading it to success. But make no
mistake, the economic thesis of Docker depends on a massive landgrab of vendor
lock-in. This acquisition is a hedge against any Unikernel company looking to
make the same landgrab.

There's a reason Docker is so heavily funded by the biggest cloud companies.
They're the ones who stand to benefit from specialized Docker container
optimized for their own platform. It's a great way to package open source
services and leverage the effort of the developer community into centralized
profit.

It seems blatantly obvious that Docker is looking to build the app store of
devops. I wish them the best of luck, but they are going to face some heavy
resistance from open source initiatives. There is nothing about Docker that
makes it fundamentally superior to the systems it's based on, specifically the
LXC project. When developers finally wake up to the fact that they are sleep
walking into a massive walled garden, Docker will lose some of its clout.

~~~
cyphar
Docker doesn't build on LXC, they have their own implementation of a container
runtime known as libcontainer, which has been used for over 2 years. I also
want it noted that the OCI and runC mean that vendor lock-in is very hard.
There is a tonne of work going into making Docker a wrapper for runC so you
can replace Docker usage with runC and containerd without having to rewrite
your configuration.

I've been involved in the Docker community as a free software developer for a
long time. I may have squabbles with some of the people who work at Docker
Inc. but many of them are very passionate about software freedom, so I don't
think it's fair to say the project is bad because the CTO (who hasn't
committed code for almost a year) has views that are unfriendly to free
software.

~~~
shykes
I have to say, after having contributed thousands of commits to free software,
recruited hundreds of people to do the same, and directed tens of millions of
dollars to open-source contributions... this stings a little.

For the record I don't have views unfriendly to free software. But I have
upset many people by disagreeing with their design opinions, and as a result
not merging their pull requests. Sometimes those people imagine a conspiracy
of evil VCs smoking cigars, instructing me to reject their patch for an
obscure capitalistic motive that they can't quite express. When in reality the
explanation is simpler: I disagree on their opinion, and I have commit rights.

If you point out an example of my supposed hostility to free software, I will
be happy to refute it with objective facts. But then everyone is free to
believe what they want to believe.

------
div0
OCaml is a fine language that most people don't use. If I want a unikernel in
my own language, do I need to build one myself? I wonder if someone is
building a unikernel that have external language bindings, which will allow
one to create "High-level" unikernels. This will open up the possibility to
completely bypass the installation of language runtime. For example, I can
just type some Python code into a browser editor, the backend can take the
source code and fork a Python unikernel to run the code. Docker can currently
do this but one still has to rely an underly OS to manage all the packages
etc. Wouldn't it be nice if you could simple write "import xyz", and the
unikernel takes care of fetching them automatically?

~~~
jorge-fundido
Will probably be a while before could build a clean-slate unikernel with
something like python, but doing what you want in a rumpkernel looks to be
just around the corner:

[https://github.com/rumpkernel/rumprun-
packages/tree/master/p...](https://github.com/rumpkernel/rumprun-
packages/tree/master/python3)

~~~
rday
It's getting really close! I have a post detailing how you would run Flask in
a rumpkernel[0]. Right now any pure Python modules will run just fine.

We get into tricky situations when you need C extensions, e.g MySQL or
PostgreSQL. Since extensions must be statically linked, you have to decide
upfront what you want. Either extension can be compiled in, but including both
in the package by default.... So a production Python build is fairly custom
right now.

Nothing insurmountable, just haven't gotten the workflow perfect.

[0] [http://projects.curiousllc.com/flask-in-a-rump-
kernel.html#f...](http://projects.curiousllc.com/flask-in-a-rump-
kernel.html#flask-in-a-rump-kernel)

------
amirmc
We also put a note on the company website [1]. I'm looking forward to working
with the Docker folks on this. :)

[1] [http://unikernel.com/#notice](http://unikernel.com/#notice) (not to be
confused with the _community_ website at
[http://unikernel.org](http://unikernel.org) :)

------
bcg1
So far Docker seems to be a good citizen when it comes to FOSS, hopefully that
will continue.

I've been following the Mirage and rumpkernel lists for a while and its nice
to see these hackers getting traction (and money!) for their efforts.

Not too long ago unikernel.org was started, which IIRC was billed as a
community driven "one stop shop" for information on the subject, which I
assume is independent of the company "Unikernel Systems". Hopefully Docker
won't go rogue and start attacking others that use the term "unikernel" by
claiming that its trademarked or something like that.

Congratulations Amir et al!

~~~
amirmc
Unikernel Systems was supporting the community site (i.e. paying for stuff)
and that will continue with Docker. It's important that the open-source work
continues to grow and that all the projects benefit from each other.
unikernel.org just got started so we need help to grow it and make it the
place to go.

Also, the word 'unikernel' is used like a generic term and as far as I know
it's not trademarked.

------
seliopou
Congratulations to the all the OCaml homies up in Cambridge! Well-deserved.

------
crudbug
I have been following unikernel development for sometime. The work done by
Atti Kantee and others on Rumpkernels [1] is most promising and has the right
abstractions (POSIX userspace using NetBSD stack). Also, in the demo video,
unikernel folks should acknowledge rumpkernel work as they are using it :)

[1] [http://rumpkernel.org](http://rumpkernel.org)

~~~
anttiok
I sort of agree with you and I sort of don't. For example, I think an
interesting future path for the Rumprun unikernel (which, I always stress, is
_not_ the same thing as a rump kernel) is running without the POSIX-y
userspace interfaces. In fact, that's pretty much where I think e.g. Golang
support for Rumprun should go, i.e. remove the userspace abstractions from the
stack, since they, conceptually, do exactly nothing. Parts of the
implementation of Rumprun are wrong, because I didn't previously see the
importance of no-userspace, but I'm slowly converting those. (ironically,
Rumprun -- before the codebase was even called Rumprun -- started out as no-
userspace, and then grew too much userspace ... but that's really another
story)

(edit added later so that we get credit going where it's due: It occurred to
me that it was Sebastian Wicki who was campaigning for the no-userspace mode
last summer as part of his lowRISC project, and did the initial work to be
able to again use Rumprun without "userspace". Apparently my memory of events
goes only a few weeks back if I don't think about things carefully ...)

Now, if I'm allowed to summarize rump kernels, I'd say the goal is to build a
framework which incorporates enough of the past to allow things to work, but
tries to be as flexible as possible so as to enable the future. I'm a firm
believer in "there's no such thing as #1", which means you shouldn't produce
software components which work only in one type of tool, because it's easy to
foresee that right around the corner you'll have to build components for the
next tool.

p.s. "Atti"? That one was new, usually it's "Antii" or something like that ;)
;)

~~~
crudbug
From the comment:

Having everything in-kernel (single memory space) with POSIX-y API for
applications the right direction ?

or I just brain farted here ?

~~~
anttiok
Assuming that was directed at me, what's "the comment"?

Anyway, that's the right direction if that's what you want you want to
accomplish. Otherwise it's the wrong direction.

sincerely, anti

------
EvanPlaice
Holee shit. I totally called this. Even got made fun of on twitter (via
@ShitHNSays) for mentioning it.

I didn't expect unikernels to gain mainstream notice for at least 6 months to
a year.

------
tyingq
I didn't quite get what a unikernel was. Reading up on approaches that were a
bit different than MirageOS was helpful:

[http://osv.io/](http://osv.io/)

[https://github.com/rumpkernel/rumprun](https://github.com/rumpkernel/rumprun)

~~~
ksec
I thought OSv is dead?

~~~
tyingq
There are recent commits: [https://github.com/cloudius-
systems/osv/commits/master](https://github.com/cloudius-
systems/osv/commits/master)

I posted the links because (at least for me) it was hard to get my arms around
what a unikernel was by looking at MirageOS. Largely because I have little
knowledge of functional programming, OCaml, etc.

------
erichocean
Our latest distributed database uses a mono kernel too. We use Pure64[0] to
boot the system and then the "kernel" is derived from QK[1], but it's also
just our database software.

Other than reducing complexity, our distributed database use the virtual
memory hardware in a unique way, so a mono kernel was essential.

Having said that, the easiest way to develop such a system is _not_ on the
bare metal, it's by running Linux in such a way that in only uses the first 1
or 2 cores, and then running your "custom kernel" on any other cores in the
system. Then you can use a normal debugger and utilities during development.
It's only when you actually want to put it into production that you can
consider not using Linux at all.

[0]
[https://github.com/ReturnInfinity/Pure64](https://github.com/ReturnInfinity/Pure64)

[1] [http://www.state-machine.com/qpcpp/struct.html#comp_qk](http://www.state-
machine.com/qpcpp/struct.html#comp_qk)

------
hueving
Considering the way docker tends to feature-creep, they will eventually just
be re-implementing a full kernel. :)

~~~
SEJeff
This ^

Docker is trying to be the next systemd

------
bch
Why don't I see Antii Kantee's[0] name all over this?

[0] [https://archive.fosdem.org/2013/interviews/2013-antii-
kantee...](https://archive.fosdem.org/2013/interviews/2013-antii-kantee/)

~~~
amirmc
Antti isn't part of Unikernel Systems. List of people involved can be found at
[http://unikernel.com/#notice](http://unikernel.com/#notice)

~~~
anttiok
That, and my name not being Antii (as already mentioned in the discussion).

Pick whichever reason suits your agenda ;-)

~~~
bch
:)

My 'editability' timed out on my initial comment before I had a chance to fix
that, Antti.

That said, there's a footnote about rumprun, and if Justin Cormack of
Unikernel is
[https://twitter.com/justincormack](https://twitter.com/justincormack), its
all so close to Antti (Justin and Antti are both of NetBSD). For -me- Antti
has been such a big part of my unikernel experience, I figured there'd be more
involvement in this project or at least more recognition of his work.

------
takeda
I'm a bit surprised. Isn't that a different approach than containerization?

I hope this wasn't acquisition to simply kill unikernel approach.

~~~
justincormack
No, it may look a bit different but it has the same aims. And they are not
intending to kill us. Disclaimer: I work for Unikernel Systems, now Docker.

~~~
takeda
The problem I have with is that those are two different philosophies to
address the same problem. I have concern whether docker can pursue both of
them at the same time. I see that sooner or later they'll have to decide which
direction to pursue.

If they decide to drop containers and go the unikernel route, then a lot of
what they already did to this point is no longer relevant. If go the
containers direction then unikernel technology won't be too useful, unless
using it to create hosts OS that would run the containers, but that is a huge
challenge and at that point it no longer would be simple kernel.

------
dukedougal
They seem to have sold very early in the lifecycle of the Unikernel
technology. Possible they've left billions on the table?

Were the terms of the deal disclosed?

Perhaps Unikernel Systems ran out of money and it was an "acqui-hire"?

~~~
mwcampbell
I wonder what the planned business model of Unikernel Systems was pre-
acquisition.

Even if the acquisition was a result of Unikernel Systems running out of
money, I have no doubt that these folks will continue to work on unikernels,
rather than, say, more tools for managing containers.

~~~
dukedougal
Maybe Unikernel Systems was a supernova from the early unikernel universe.

Its people will be the elements that go to make up a new generation of
unikernel companies.

------
AkihiroSuda
Interesting demo materials(`docker-unikernel`): [https://github.com/Unikernel-
Systems/DockerConEU2015-demo](https://github.com/Unikernel-
Systems/DockerConEU2015-demo)

------
mwcampbell
I'm curious about how support for building unikernels will be integrated into
Docker. The current Dockerfile-based build process doesn't support separate
build-time and run-time environments, but when building a unikernel, the
build-time environment is completely different from the run-time artifact.
Support for separate build-time and run-time environments is also useful when
building container images, so the image doesn't include things that are only
necessary at build time. So I hope that problem is solved first; I think the
addition of unikernel support will be more natural that way.

------
e_d_g_a_r
So Docker will now have some OCaml openings? :)

------
dap
This feels specious:

> The result of this is a very small and fast machine that has fewer security
> issues than traditional operating systems (because you strip out so much
> from the operating system, the attack surface becomes very small, too).

Obviously traditional operating systems provide a lot of interfaces that
represent attack surface, but they're generally able to be secured. On the
other hand, much of the operating system actually _implements_ security, so if
you throw it out, you're losing that.

~~~
EvanPlaice
Implements security theatre. If the OS is immutable then there's nowhere to
persist an exploit.

A server could be killed and restarted at the first sign of compromise. If you
aren't loading GBs worth of modules that'll never be uses, the OS will be able
to boot up in a few seconds max.

~~~
binarycrusader
What do you mean by immutable?

If you're talking about the on-disk files, that's possible to do without
throwing away almost everything -- Solaris does this using read-only zones
(ROZR).

If you're talking about the kernel itself; that can never be truly immutable.
An OS had to read and write memory.

I also don't understand the claim about "GBs worth of modules"; if you're
talking about kernel modules, there shouldn't be "GBs" to start with.

As for security theatre; I hardly think crypto support, auditing frameworks,
qualify. So you need to define that as well.

~~~
EvanPlaice
Immutable as in filesystem.

Meaning every restart brings the system back to a fresh state. Most
trojans/backdoors persist because they're able to modify the underlying OS to
load them on boot. No, immutable OSes are -- by no means -- a new idea. This
just uses the approach by default.

Not talking about the kernel specifically. Although there's no reason the
bootup time should be slowed down by loading unnecessary HAL, drivers,
protocols, utilities that will never be used in a server setup. GBs of modules
was referring to the rest of the OS made up mostly of features that will never
be used by the server.

Crypto support is one big issue. I'd assume if the OS uses V8 at the base
level, it should be able to be packaged with OpenSSL.

Auditing for what exactly? File access? The filesystem is immutable. Network
requests? Can ve handles at the network level or a tool can be added to log
network requests via an offline logging service.

Security auditing at the OS level isn't required to the same degree it is on
other OSes because the attack surface is so small. For instance, you don't
have to check for SSH vulnerabilities when there's no SSH. The userspace
sandbox isn't necessary because everything that's network-facing already has
to run through V8's sandbox.

Application-level vulnerabilities will always be an issue but that's nothing
new. This doesn't solve all problems. Just reduces the potential for issues.

~~~
binarycrusader
_Meaning every restart brings the system back to a fresh state. Most trojans
/backdoors persist because they're able to modify the underlying OS to load
them on boot. No, immutable OSes are -- by no means -- a new idea. This just
uses the approach by default._

Ok, then yes, Solaris read-only zones provide that level of immutability.

 _Not talking about the kernel specifically. Although there 's no reason the
bootup time should be slowed down by loading unnecessary HAL, drivers,
protocols, utilities that will never be used in a server setup. GBs of modules
was referring to the rest of the OS made up mostly of features that will never
be used by the server._

A good operating system will be smart enough to only load what it needs; there
will be some inefficiency, but protocols, drivers, HAL are actually used in
server setups and are not usually the source of significantly increased boot
times unless they're misconfigured. Also, with a proper service management
facility and packaging, it should be possible to simply omit unnecessary
items; services are usually what contribute the most to startup time.

 _Crypto support is one big issue. I 'd assume if the OS uses V8 at the base
level, it should be able to be packaged with OpenSSL._

Right, but you generally need more crypto administrative capability than
OpenSSL alone provides to manage a sufficiently complex application. (e.g.
gnupg, etc.)

 _Security auditing at the OS level isn 't required to the same degree it is
on other OSes because the attack surface is so small._

Sorry, I'll have to disagree on that point. Auditing is always important
because it allows you to determine what happened and why when privileged
operations are involved. The overhead of auditing is almost completely
determined by what an administrator chooses to audit and how detailed that
auditing is.

 _For instance, you don 't have to check for SSH vulnerabilities when there's
no SSH._

Perhaps you misunderstood what I referred to when I spoke of auditing; I don't
speak of checking for vulnerabilities -- packaging should generally handle
that in combination with metadata and reporting. I was referring to the
recording of the assumption of privileged operations, system authentication,
and related details.

 _The userspace sandbox isn 't necessary because everything that's network-
facing already has to run through V8's sandbox._

Disagree again; the point of having a userspace sandbox is in case something
escapes V8's sandbox, which has been done in the past. It's an extra line of
defense.

 _Application-level vulnerabilities will always be an issue but that 's
nothing new. This doesn't solve all problems. Just reduces the potential for
issues._

While it reduces the number of problems in the existing space, I personally
believe it creates _new_ problems. I'm not sure the tradeoffs are worth it.

I would rather see an immutable container with a minimised set of packages
with proper auditing enabled. That's far easier to test and deploy and doesn't
create _new_ problems.

------
e_d_g_a_r
Yes! More OCaml adoption.

