
How to Use User Mode Linux - xena
https://christine.website/blog/howto-usermode-linux-2019-07-07
======
nneonneo
UML has been around for a very, very long time. I remember using it back in
undergrad for one of my courses, but now I forget which one.

It’s got great documentation. See the kernel.org documentation here:
[https://www.kernel.org/doc/Documentation/virtual/uml/UserMod...](https://www.kernel.org/doc/Documentation/virtual/uml/UserModeLinux-
HOWTO.txt) and a write up from the creator Jeff Dike (which includes a lot of
technical detail on the inner workings) here:
[https://lwn.net/2001/features/OLS/pdf/pdf/uml.pdf](https://lwn.net/2001/features/OLS/pdf/pdf/uml.pdf)
(from 2001!)

slirp is the only _purely usermode_ way to do networking, but UML also
supports TUN/TAP for something a little more sane. UML does a bunch of pretty
crazy things in order to work, including a dedicated thread whose job it is to
ptrace every other process for syscall and signal emulation (and, as a bonus,
it even ptraces external debuggers to make them think they’ve successfully
ptraced a process that’s already being traced!), and a full emulation of
physical/virtual memory translation with paging (done by giving every virtual
process under UML its own virtual address space).

All in all, a super impressive project, and definitely one worth trying out.

~~~
svd4anything
I’ve recently been using proot [1] as a way to have a fully controlled rootfs
for jobs on a HPC cluster/grid service where I am a very regular unprivileged
user.

Is it theoretically possible I could run an entire linux kernel instead with
this? Is there any setup on the host system that is required which needs
administrative privileges?

[1] [https://proot-me.github.io/](https://proot-me.github.io/)

~~~
ktpsns
The HPC usage scenario is exactly the reason why I played with UML a few years
ago. Many HPC clusters nowadays completely lock down the network and only
allow in/outbound SSH connections to whitelisted IP networks. On the other
hand, users are only given an unpriviledged account. Typically, users set up
proxies on a per-application level (i.e. for git, svn, to sync their codes
with the outer world). I found this very frustrating and wanted to use any
usual Linux command line tools. My idea was a UML based virtual machine (ie.
running a Linux distribution within UML) and have it connected to the outer
world by tunneling a single slirp and VDE connection. VDE is quite amazing
because it marries unix pipes with ethernet-level networking. It even allows
to run ethernet switches as processes and all that. My codes based on
[https://github.com/jpetazzo/sekexe](https://github.com/jpetazzo/sekexe)

Turned out: It basically worked, but it was a bit painful to setup and to
manage.

~~~
gnufx
I'd have thought it would be relatively unusual not to be able to use HTTP
into a cluster login node, but I'm puzzled why you can't "sync code" with ssh
otherwise. If you're prevented from doing things you need to work, it sounds
like an institutional politics problem; subverting controls is likely to cause
more friction as well as presumably wasting resources due to inefficiency.

------
vmlinuz
Many many years ago, when I was working at a large 3-letter-starting-with-S
Unix company (no, not the graphics guys, nor the suing-Linux-for-Microsoft
guys), we were planning a Linux compatibility layer... At the time, there were
a couple of options out there - lxrun and uml. Anyone remember lxrun? I tested
them both and did a little write-up, basically saying that neither of them ran
reliably or fast enough to be worth adopting

What we decided was to go with option 3 - we implemented a Linux system call
interface within our own kernel to give us 'native' support. This, of course,
is exactly what the first version of WSL did, many years later. Plus ça
change...

~~~
meruru
>at a large 3-letter-starting-with-S Unix company (no, not the graphics guys,
nor the suing-Linux-for-Microsoft guys)

Is it Sun? Why use more than 20 words to (probably) mean "for Sun"?

~~~
ghostpepper
Presumably they think this is a defense that will hold up in court as
plausible deniability if they are ever sued for defamation relating to this
comment.

~~~
vmlinuz
You guys are weird. It was just a bit of wordplay :D

~~~
ghostpepper
I didn't really have an issue with what you wrote. It made me realize there
are several three-letter, starts-with-S, Unix companies.

------
wmeddie
Brings back memories. My first VPS back in 2003 used UML and it worked great.
You could also run it inside Windows as well. Could even get GUI apps running
by running Xming on your Windows desktop using CoLinux
([http://www.colinux.org/?section=screenshots](http://www.colinux.org/?section=screenshots)).

~~~
zoobab
Colinux was freaking great, I could run an OpenSSH server on port 22 and
sometimes ending up on a Linux box, sometimes on a Windows box. Weird.

------
linsomniac
I have often thought back to how great UML was. I had filed it under "distant
history" when I thought about it last week...

Back in 2003 or 2004, I threw together a "virtual hosting" platform using UML.
It had a web form accepting payment, and a few minutes later you'd have a
fully running virtual machine. I put this together in a couple weeks leading
up to PyCon, where we launched it. But we didn't really have the business
savvy to take it anywhere, we never had more than a few hundred VMs.

We eventually moved to other technologies: VMWare, Xen, Proxmox (KVM). But UML
was my favorite. (Xen was my least favorite :-)

------
EvanAnderson
The next logical thing to do for more fun is to run User Mode Linux under the
Windows Subsystem for Linux.

After that, it would make sense to try WINE.

~~~
userbinator
A Windows app running on WINE inside UML inside WSL of a Win10 which is itself
a VM running on e.g. macOS would be funny to see.

~~~
sigjuice
Does WSL support ptrace now? UML can't run without it.

------
svd4anything
[https://lwn.net/2001/features/OLS/pdf/pdf/uml.pdf](https://lwn.net/2001/features/OLS/pdf/pdf/uml.pdf)

In 2001 this mentions SMP support is possible. First thing tomorrow I need to
check if it was ever implemented. Perhaps it’s late and I’m tired but somehow
this seems to be much better than containers for a number of problems, or
maybe the performance hit is too much.

------
michaelgrafl
Off topic, but here it goes: I really like the styling of this website. I
shouldn't like it, because I don't like non-proportional fonts being used for
prose, but it works great in this case.

Also loads super fast, which is an exception these days.

~~~
opan
I like it as well. It looks like one of those generated static sites from hugo
or haunt. I've been seeing it more often lately and I think it's a good trend.

~~~
xena
My site isn't actually a static site. The code is in
[https://github.com/Xe/site](https://github.com/Xe/site)

------
lordleft
I had no idea this was possible. Excellent article - thank you for opening my
eyes to yet another insanely cool aspect of Linux.

------
zoobab
weberlars/diuid [https://github.com/weber-
software/diuid](https://github.com/weber-software/diuid) runs a UML in a
docker container, which then runs a Docker daemon of a different version
inside. Clever.

------
Tepix
It would be nice to combine UML with an encrypted root filesystem so a nosy
admin on a VPS will have a harder time seeing what you are doing on your VPS.
Still I wonder what the performance will be like. Is the entire UML process
running single threaded on a single CPU core?

~~~
xena
Each process in the guest has its own host thread, but yeah the way I have it
set up there's only one logical "CPU core" being exposed to the guest.

------
twayback
Great work!

I am trying to run this on the latest Ubuntu but getting the below error:

root@kit:~/uml/furry-happiness# docker run --rm -it xena/docker-uml [WARN tini
(7)] Tini is not running as PID 1 and isn't registered as a child subreaper.
Zombie processes will not be re-parented to Tini, so zombie reaping won't
work. To fix the problem, use the -s option or set the environment variable
TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.
Core dump limits : soft - NONE hard - NONE Checking that ptrace can change
system call numbers...ptrace: Operation not permitted check_ptrace : expected
SIGSTOP, got status = 9

~~~
xena
What host OS do you run? I got this working on both Ubuntu and Alpine.

~~~
twayback
I am running this on ubuntu 18.04 :(

------
etaioinshrdlu
When I tried User Mode Linux I found that there was very little in the way of
ready-made, up to date tooling.

Very little "just worked".

I think there's huge potential there, however, as an architectural building
block, just like containers are.

~~~
xena
It's the kind of stuff that drives you insane a bit. You have to hack together
things from software last released in 2006. You have to dig through kernel
source code to really understand what is going on in the user mode Linux
drivers.

------
bencollier49
The article states that processes in User Mode Linux are completely isolated
from the host, and cannot be seen.

Could someone clarify this for me - there must be _some_ way of pulling data
from UML back into the host system?

~~~
nneonneo
Actually, very much on the contrary, the processes running inside UML are
completely visible as separate threads within the UML process, and you can go
poking their `proc` files at-will. Also, UML uses a read/write disk image as
the rootfs, so any writes to the filesystem will be persisted in the disk
image.

~~~
svd4anything
> processes running inside UML are completely visible as separate threads
> within the UML process

Does this mean the UML system can utilize many cores well and SMP was
eventually implied?

------
badrabbit
Very nice. Now I'm tempted to try and run X11 and firefox in UML somehow.

------
meruru
How does this compare to other methods of isolation / running untrusted code?
What are the pros and cons?

~~~
dspillett
[https://www.linode.com/](https://www.linode.com/) used to run off UML, though
they switched quite a number of years ago. It might be worth hunting out their
documentation and/or forum archives from around the time of the transition. I
recall a lot of detailed discussion (useful discussion too: not just people
bashing the technology they don't prefer).

I used to run it myself, way back when. For isolation, it is going to be
better than a container or jail, but not as good as a full VM, though I was
using it for testing personal stuff so wasn't really concerned with isolation
from a security point of view. One key con is that it can perform pretty badly
in some respects, particularly for loads that ned to talk to the kernel much
at all, including processes that perform much IO. It "felt" less cumbersome
than full VMs (IIRC I was playing with a mix of VirtualBox and VMWare' Player
& Server for that sort of thing at the time), in terms of my automation and to
some extent in terms of host resource use (though I didn't test that with any
scientific rigour, and if I did any good tests back then the results would not
be relevant today anyway as they were pre VT-x and other CPU support which UML
can't use but a modern hypervisor can and will).

