
Fakecracker: NetBSD as a Function Based MicroVM - yankcrime
https://imil.net/blog/posts/2020/fakecracker-netbsd-as-a-function-based-microvm/
======
hultner
Intersting, always fun to read some BSD content. One thing that bothered me
though is that no bootup times are specified anywhere in the article which
after all focuses on exactly this, it would be very nice to see boot time of
Fakecracker vs Docker vs NetBSD amd64 w. bootloader (and unmodified), and
compared to the minimal Linux kernel menitoned in the begining.

~~~
sigjuice
Docker is not exactly "booted" in the traditional meaning of starting a (real
or virtual) computer.

~~~
hultner
No but the process is still started and I suppose there's still some sort of
setup time for networking, context group, etc.

------
api
I'd like to see a small lean kernel like NetBSD ported to WASM. You could run
services in the browser or in a relatively safe and portable local WASM VM.
Would be an interesting way to distribute certain kinds of software.
Performance wouldn't be up to par with native, but it wouldn't be too bad.

~~~
greenshackle2
And then let's ship desktop apps in electron running NetBSD. And we will
finally have come full circle :)

~~~
api
Can you build NetBSD to run as a user process like user mode Linux? A cross
platform user mode "kernel" would also be a pretty interesting runtime.

------
winrid
NetBSD is so fun to play with. Highly recommended.

~~~
lbhdc
I don't really know much about NetBSD. Where is a good place to jump in?

~~~
inetsee
The NetBSD Project has its own collection of tutorials here:
[https://wiki.netbsd.org/tutorials/](https://wiki.netbsd.org/tutorials/)

------
yjftsjthsd-h
One limitation of this system as it currently exists seems to be that you need
a working i386 NetBSD host to built the filesystem. It might not be worth the
effort once pkgin gets involved, but in theory I would expect this to be
perfectly possibly to get around; after all, NetBSD's own build.sh is
perfectly happy to build a full system image for ex. i386 NetBSD from source
on ex. an amd64 Linux host. Or, if you didn't want to deal with cross-
compiling, perhaps it would make things easier to build a Fakecracker VM that
itself builds root filesystems, analogously to building docker images from a
container using dind.

~~~
pwdisswordfish2
"One limitation of this system as it currently exists seems to be that you
need a working i386 NetBSD host to build the filesystem."

One does not need a NetBSD host. This is not the most time/space efficient
method I can think of, but it's the easiest to explain...

On Linux host:

1\. Download

    
    
       for x in gnusrc sharesrc syssrc src;do ftp ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-9.0/source/sets/$x.tgz&& tar xzf $x -C /;done
    

2\. Compile toolchain

    
    
       mkdir /usr/obj
    
       PATH=/bin:/sbin:/usr/bin:/usr/sbin ./build.sh -m i386 tools
    

3\. Make filesystem

Using makefs instead of vnconfig + mount + newfs has the advantage that makefs
is part of the toolchain while vnconfig is not

    
    
       makefs root.img fakecracker

~~~
anthk
Wouldn't that crap out /usr/include?

~~~
yjftsjthsd-h
Nope; all the netbsd source tarballs extract to [/]usr/src

------
gdamjan1
Does NetBSD support initramfs (the name of the Linux feature)? That would
avoid having to create a disk image by running a bare system from a
ramfs/tmpfs.

I experimented with something like that with Linux and cloud-hypervisor (a
firecracker fork):
[https://gist.github.com/gdamjan/1f260b58eb9fb1ba62d223495858...](https://gist.github.com/gdamjan/1f260b58eb9fb1ba62d2234958582405)

------
lsofzz
I assume this is a play on the word "firecracker"? :D

[https://github.com/firecracker-
microvm/firecracker](https://github.com/firecracker-microvm/firecracker)

~~~
jlgaddis
No need to assume anything. The article starts out:

> _In November 2018 AWS published an Open Source tool called Firecracker, ..._

~~~
lsofzz
> No need to assume anything. The article starts out:

Aha. Reading now.

------
MintelIE
Think of the overhead which could be saved if services could be "spun up" in
chrooted environments or even just as members of traditional Unix-style user-
private groups! I know, I know, this is a newish concept which seems crazy.
But it could work.

~~~
anthk
Ahem. Chroot has lots of ways to escape.

~~~
MintelIE
The article mentions this but links to a Google search as "proof." Even VM's
have lots of ways to escape these days.

Chroot vulnerabilities have been discovered and fixed over time, as any
security issue in an operating system is. It's not accurate to say that they
are insecure in a blanket fashion especially these days. My opinion is that we
should be using Unix as it is meant to be used, as on operating system, and
using its time worn facilities meant for purposes such as security and
sandboxing. They are very well tested and the solutions are baked-in and
generally pretty small in terms of both code and overhead when compared to
spinning up a VM, for instance. There are arguments for using VMs for security
and scaling but they don't always win over just one modest local server in
either domain. We're not all serving Google search after all.

~~~
lsofzz
> Chroot vulnerabilities have been discovered and fixed over time, as any
> security issue in an operating system is. It's not accurate to say that they
> are insecure in a blanket fashion especially these days. My opinion is that
> we should be using Unix as it is meant to be used, as on operating system,
> and using its time worn facilities meant for purposes such as

I think you are right in general terms but I do not think an attacker will
play by the rule and use `chroot` in UNIX-like OS as it is "meant to be used".
They will use whatever means necessary - whether it be 0day or other un-
patched vulnerability to get a break out.

~~~
smabie
Doesn't this apply to really any sort of isolated environment? What's wrong
with chroot vs a vm?

~~~
lsofzz
> Doesn't this apply to really any sort of isolated environment? What's wrong
> with chroot vs a vm?

If you are implying _PoC GTFO_, then definitely I do not have anything to
suggest today that either is secure but I would rather not base my comment on
what's not seen in the wild and also on the limited breadth of my research.

If anything, we've learnt the from past exploitations of guest
additions/kernel modules in guests/vmm, that all of it is real and possible.
Given enough resource, time and eyeballs, a _lot_ of the bugs are
_exploitable_ \- you just have to ask some of the folks in offsec who develop
exploits for living.

