
Tiny Linux distro that runs the entire OS as Docker containers - purak
https://github.com/rancher/os
======
oneplane
This is starting to smell like a system on top of a system to fix something
that could be fixed in the system. Kind-of like implementing a filesystem on
top op a filesystem... or putting a database on a filesystem to run another
filesystem inside the database, or using a webbrowser as a runtime instead of
an operating system.

~~~
zerocrates
This kind of thing happens when the base system is ubiquitous and therefore
hard to change. It's easier to layer something on top of the base, where
people can "opt in" and there's a large preexisting compatible audience.

Changing the base layer itself at a minimum requires people to upgrade, and
now you don't have that advantage of the preexisting audience anymore. If your
improvement requires a _breaking_ change, then you're in a real pickle. So
people will stack on more and more until it becomes more or less unbearable.

~~~
user5994461
There is an advantage in layering components, or building new software on top
of existing components.

However, a minor improvement or bugfix should be done in the component that is
responsible for it. Creating another layer instead of fixing the problem is
just creating more problems.

~~~
numbsafari
You can go whole hog with "containers" by just adopting unikernels.

But, unless you are willing to abandon Linux/Unix, this is kinda where you are
left.

~~~
hueving
How would that work when containers make kernel calls?

~~~
sp332
Where containers would make kernel calls, unikernels make calls to the VM
hypervisor. (Not exactly, but kinda.)

------
darren0
Creator of RancherOS here. Thanks for the interest in our tiny distro.
RancherOS was created the beginning of 2015 and at the time was quite a novel
concept. We strived to not just use container technologies in a Linux distro
but actually package everything as standard Docker containers. Fast forward
two years, what we were doing back then is now becoming the accepted practice.
Most major distro are adopting more container packaging approaches in the form
of flatpak, snap, and containerd. RancherOS 1.0 LTS was just released a couple
weeks back and we have started development on 2.0. 1.0 was a bit ahead of the
time and honestly had to employ a lot of technics to make it work that we
didn't like. With 2.0 we will shift the focus from Docker to containerd, OCI,
and LinuxKit which will allow a much cleaner design.

~~~
rcarmo
Will there be an ARM version?

~~~
darren0
We released some versions for ARM but in the end ARM requires too much effort
to support right now. As ARM64 matures and Docker gets full multi arch image
support we'll revisit this.

------
gtrevorjay
This is clearly a trend, though it remains to see if it will garner enough
acceptance to actually be "the future". systemd supports launching container-
based services via nspawn and already namespaces "legacy" services very
heavily. In fact, systemd et al were among the heaviest early drivers of
cgroup technology for cleaner starting and stopping of groups of processes.

~~~
frik
I am starting to wonder, why not just execute processes directly with cgroups
commands?

    
    
      $ cgcreate -g memory,cpu:groupname/foo
      $ cgexec    -g memory,cpu:groupname/foo bash
    

[https://wiki.archlinux.org/index.php/cgroups](https://wiki.archlinux.org/index.php/cgroups)

It's the bare basic that libvirt and Docker et al are based anyway. So if you
want to run just one process per "container" it seems rather logical to keep
it simple and use cgroups commands directly. (Similar on Windows, using just
sandboxie is so simple. Or do it like Android, execute every app/process with
a different user.)

[https://en.wikipedia.org/wiki/Cgroups](https://en.wikipedia.org/wiki/Cgroups)

~~~
wmf
systemd already puts each service in a separate cgroup AFAIK, so commands like
cgexec aren't even needed. I suspect people are more interested in namespace
separation, but recent versions of systemd can also do that [1]. I don't think
systemd has image management so that's still a reason to use Docker.

[1]
[https://www.freedesktop.org/software/systemd/man/systemd.exe...](https://www.freedesktop.org/software/systemd/man/systemd.exec.html)

~~~
technofiend
Systemd creates three "slices": system, user and machine. Daemons started by
systemd go in system, user processes in user and virtual machines go in
machine.

Each process is placed in a hierarchy​ so you end up with system-httpd which
makes it easy to assign or limit resources based on the slice.

Redhat covers it extensively in their performance​ and tuning class.

------
thrill
I've been using RancherOS for a few weeks now and I'm quite delighted with it.
A confusion that occurred at first for me was the distinction between Rancher
(the project/company), Rancher (the application) which can be used for
installation and distribution, and RancherOS, which is the concept of
replacing the Init process with Docker. Docker is used in separate instances -
one to manage the containers that make the OS, and one for the remaining
containers that make your apps.

I think the advent of Docker Swarm probably put a crimp on development and use
of Rancher (the app). To me the way forward is Docker's own clustering tools,
and the ease of standing up a cluster of Atom processors at www.packet.net
where they install (as an option) RancherOS is very attractive.

------
hobarrera
I'd really love to see some of this stuff transition to the desktop too.

Like, for example, containerize Skype, so that it can't read my home. Or
contain Firefox to just read `~/.mozilla` and `~/downloads`.

For binary blobs I don't trust that much, I'd really value this.

For FLOSS stuff, it still provides protection from bugs.

~~~
wasted_intel
Look at [http://flatpak.org](http://flatpak.org) for precisely that.

~~~
hobarrera
Nope, it does a lot more: it re-packages software, and basically shoves a
second package manager down my throat; one that actually bundles dependencies
within in package, carrying along all the issues that that flow brings with
it.

I want to isolate data, no libraries. Libraries are there to be shared.

------
B1tchard0
I don't understand why running software on bare metal is viewed as a problem
to be solved.

how many layers of abstraction are necessary, and why?

Obviously virtualizing serves a valuable purpose.

Making development more accessible is great. Simplistic dev services like this
mean reliance on others infrastructure, and being bound to cloud.

Doesn't seem forward thinking.

Can you imagine if Google had decided to run their search app on Microsoft
servers?

~~~
digi_owl
The basic thing is that we have ended up with a world of rock star code
monkeys. And those rock stars can't be held back by some admin or exec saying
no to using some hot new language or framework...

------
cookiecaper
CoreOS works the same way. All containers. You can run `toolbox` to get into a
systemd-namespace'd Fedora container (any other container can be specified;
it's just Fedora by default), from which you're supposed to do all your
troubleshooting/analysis (caveat: systemd-namespace does not seem to support
`auditd` well).

I still strongly dislike "containers". It's not worth the complexity or
instability. Two thumbs way down!

~~~
frik
The concept itself is simple. The complexity is added by the API layer.

[https://en.wikipedia.org/wiki/Cgroups](https://en.wikipedia.org/wiki/Cgroups)

[https://en.wikipedia.org/wiki/FreeBSD_jail](https://en.wikipedia.org/wiki/FreeBSD_jail)

[https://en.wikipedia.org/wiki/Solaris_Containers](https://en.wikipedia.org/wiki/Solaris_Containers)

(more general: [https://en.wikipedia.org/wiki/Security-
Enhanced_Linux](https://en.wikipedia.org/wiki/Security-Enhanced_Linux) ,
[https://en.wikipedia.org/wiki/Sandbox_(computer_security)](https://en.wikipedia.org/wiki/Sandbox_\(computer_security\))
)

~~~
cookiecaper
Yeah, I agree and I think that FreeBSD jails in particular are much better (to
be fair, I am not very well informed on Solaris Zones, so maybe they're the
best). They are certainly much less ostentatious and do not try to redo
everything for their own little subworld like Kubernetes does.

I sat down one day to try to write down what would make Linux
containers/orchestration usable and good, and realized after about 20 minutes
that I was describing FreeBSD jails almost to a T. The sample configuration
format I theorized is very close to the real one.

However, I think that there's good reason for actual deployments of
containerized systems to remain niche, as it did until the VCs started dumping
hundreds of millions into the current Docker hype-cycle, and the big non-
Amazons jumped on board as a mechanism to try to get an advantage over AWS.

What people really want are true VMs nearly as lightweight and efficient as
containerized systems. In fact, I think many people wrongly believe that's
what containerized systems are.

~~~
rhizome
_What people really want are true VMs nearly as lightweight and efficient as
containerized systems_

Like what QubesOS is trying to do?

------
mmrezaie
Probably it was not the vision of these types of projects, but this reminds me
a lot of Qubes OS[1]. I actually have occasionally used docker (lxc) to run
some applications that I was not trusting, and I was controlling them using
cgroups. Right now my chrome browser is running like that.

[1] [https://www.qubes-os.org/](https://www.qubes-os.org/)

------
dkarapetyan
I have to say this enrages me. The system services are still privileged
containers and we are now basically emulating a micro-kernel (very badly I
might add with a monolithic kernel). If you want to use a micro-kernel then
use a fucking micro-kernel. Hacking a micro-kernel with docker is not the
right approach, especially given the stability track record of docker itself.
It's a hack and aesthetically unpleasant on all sorts of levels. Not the least
of which is that docker itself is one giant hack.

~~~
DigitalJack
You have serious problems if this enrages you. Let other people hack in peace.
Go do your own "right" thing and leave the rest of us alone.

------
codebeaker
Didn't Docker release this themselves (different project) just a week or so
ago? I forget the name, and can't seem to find it on their site.

~~~
joshwget
There's definitely overlap in functionality between RancherOS and LinuxKit,
but there are also a few pretty big differences.

\- LinuxKit seems designed to be a piece that can be used to build a Linux
distro but isn't a full distro out of the box like RancherOS

\- As far as I know LinuxKit is still based on Alpine whereas RancherOS is
custom and doesn't have much of a host filesystem

\- LinuxKit is based on containerd and RancherOS is still based on Docker
(though this is likely to change soon)

We're definitely interested in collaborating with LinuxKit since we do have
similar goals. It's probably a good idea for us to write a more detailed blog
post comparing the two since we've been getting this question pretty often
lately.

~~~
justincormack
LinuxKit is not based on Alpine - earlier versions before open sourcing were.
We build our system containers from Alpine though, so there is still a
connection. (I work on LinuxKit).

~~~
joshwget
Thanks for the correction! That's good to know.

We considered using Alpine as the base image for our system containers for a
time. They're still built using Buildroot currently, though we're playing
around with another project that we might use instead.

------
logronoide
I guess RancherOS is not something "new". First time I use it was in 2015...

Anyway, it seems it's design has inspired some people recently.

------
KaiserPro
for the ignorant, why would I want to wrap docker inside docker?

~~~
cookiecaper
This is not Docker-in-Docker. This is an OS that acts like a "container
hypervisor". It provides the bare minimum for hardware interfacing and hosting
containers, just as conventional hypervisor provide the bare minimum for
hardware interfacing and hosting virtual machines. The benefit is that more
system resources are available for your clients (containers, in this case).

I haven't used RancherOS but CoreOS works mostly-fine. However, I would avoid
using these things altogether because containerization sucks.

~~~
KaiserPro
I'd suggest that containerisation as a principle is fine, but its not what
most people want.

What people want is a mainframe where a lump of code is guaranteed to run the
same everytime, regardless of the machine state, and if something goes wrong
with the underlying layer is self heals(or automatically punts to a new
machine, state intact).

What we have currently is a guarantee that the container will run, and once
running will be terminated at an unknown time.

Mix in distributed systems(hard) atomic state handling (also hard) and
scheduling(can be hard) its not all that fun to be productive for anything
other than a basic website.

------
justAlittleCom
That's ungodly.

------
Walkman
I predicted this last year:
[https://twitter.com/kissgyorgy/status/783825879348744192](https://twitter.com/kissgyorgy/status/783825879348744192)

~~~
y4mi
congrats, you predicted years after its inception.

~~~
Walkman
Thanks!

