I found jails kind of accidentally. A few years back I built a NAS at home and wanted to run ZFS on it, so FreeBSD was an obvious choice. I soon found out how easy it is to create and manage jails: you just start a new one from a template, get an ip address, ssh into it and `pkg install` whatever you need. I now run over 10 jails, having home automation, media player, grafana, few different databases, unifi controller and so on all in their own jails.
I like the simplicity and how straightforward it is to add new servers to our home. I have long history with docker, but kind of prefer the way of being able to ssh into the jail and manage it like a normal server distro. Docker again is much nicer in the daily development flow, where I want nine different databases in a clean state every morning.
That’s exactly how I use LXD on Ubuntu. CoW with zfs or btrfs makes running sites on dedicated servers quite easy, and allows to experiment with upgrades, or other random stuff.
LXD is great. I've had it for a while in OpenWRT, but never on a CoW filesystem. Does LXD also utilize snapshots like jails do on ZFS to save some disk space?
I have now only pihole running on LXD and it's ok.
jails are great, coincidentally I started using them too with Freenas (now Truenas). The only big disadvantage is that the FreeBSD ports are less uptodate than docker (Linux).
Most open source applications are released for Linux and have to be adapted to FreeBSD. So FreeBSD maintainers have to keep up with the ports. Sometimes port maintainers drop-off leaving stale ports until they break.
It is not at all uncommon to be running applications that are several versions behind.
For popular packages this is not that common. For less popular packages, yes it can happen, because maintainers are almost completely volunteers. Some years ago I created a port, then at some point stopped being able to find the time to maintain it. Eventually someone offered to take over, and I happily obliged.
The ports Makefile framework is incredibly sophisticated, so one can find lots of examples and be quickly able create and test ports very easily, especially for packages written in C, C++, Go, Java, Perl, Python, Ruby, etc. Poudriere, which is the definitive bulk package builder used by FreeBSD pkg itself, is especially useful here. Ports that a have a billion vendored libraries are a pain. Ports are not allowed to download from the internet at build time, so all the vendor dependencies need to be known beforehand and be marshaled to download at the "fetch" phase accordingly. But eventually it all boils down to a very terse and declarative Makefile. What used to be a bigger pain, was the way to update ports in the ports tree, which used to require someone with access to the ports tree taking your new port and updating it. But recent news indicate that FreeBSD is moving to git and towards a model where port owners can each update their own directly, which will be huge. My personal opinion is that this will lower the ports maintenance bar significantly.
Also, for administrators, switching /etc/pkg/FreeBSD.conf from "quarterly" to "latest" pkg train of your version number (e.g. FreeBSD 12.x) will get you the latest versions of binary packages, as soon as their respective port is released into the ports tree. Quarterly is also pretty good, but you get only security updates and package versions tend to stay stable and change every quarter. I've administered systems switched to "latest" and seldom had any serious problems.
Yep, making sure you're on latest package branch helps for sure. When it comes to popular packages on the latest branch, I always feel like I've had far more trouble getting up to date versions on ubuntu than I do on FreeBSD.
I had some issues running latest on a desktop machine back when it was default. Switching to quarterly made updates smooth sailing; the only issues I remember after that were from a big X upgrade.
https://bastillebsd.org is a great tool if you want to deploy tiny container-like Jails on FreeBSD. I've been using it in production for a while and it's been a great tool.
Jails are nice but it has always seemed like a system where you don’t need another copy of userland would be better (eg sandboxing). It’s wasteful and you might forget to apply security patches.
You do not need another copy of the FreeBSD userland in a jail.
My unbound dns server, for instance, has something like 24 total files in it. The reason it has so many is because I run two different jails from it - my unbound jail and my nsd jail share the same directory/mountpoint.
Which is to say, I run two different jail commands, that start two different jails with different IPs and startup commands, but they both point to the same directory/mountpoint.
Updates consist of replacing a few binary executables.
Not everything will fit into that model, but when it does, it is fantastic.
for instance would run the command "/bin/csh" and jail it so that it doesn't get access to any IPv4 and it allows communications over IPv6 only.
So you can use jails to sandbox applications. You can also mount all paths are read-only and only the paths required for the application as read/write. jails can be very handy for all kinds of operations around that, there is no requirement that you provide the jail with a copy of the user land again, although that is the easiest way to run jails.
Actually, you don't need another copy of userland. You merely need a dedicated space for the jailed OS (assuming we're talking OS-style jails, not process-style ones) to write the things OS writes to (devices, logs, etc.). You could get away with null mounting your host filesystem and then mounting writeable space on top of it. Once I've had read-only mounts for my jails, for funsies.
No, it doesn't and this is where an initially "superior" technology (Jails existed years ago, I was using them at least 15 years back in time) gets overrun by "inferior" tech (Linux containers), because the latter gets a user oriented tooling, packaging and so on (no flame war intended at all). Now the Linux container is much more "superior" as it catch'd up and exceeded Jails in many areas, especially user oriented readiness and integration.
And FreeBSD Jails are still superior in the way they're a "first class citizen" in the kernel, vs. the "hacky" feel that Linux containers has.
I have no doubt that Linux containers are just as secure as FreeBSD Jails, but if the implementation and tooling is complex, there is a much higher risk of something being configured wrong.
And then there's the giant gorilla in the room, Docker, which probably has the best tooling of them all, and initially used Linux containers, but has since moved on to their own container implementation (runC, https://www.docker.com/blog/runc/).
I wouldn't be so hasty in saying that Jails implementation doesn't suffer either - there are dragons there too, it wasn't all just designed and written in one go, there are layers upon layers and it is not all pink and unicorns as it perhaps initially was ;)
RunC isn't "their own implementation" but rather an OCI (Open Containers Initiative) standard that world seems to be adopting and I wish FreeBSD Jails would be a part of it.
Jails has their share of problems, all i was saying is that when the tooling and implementation is complex, the risk of doing "something wrong" is bigger. ;)
I really like how jails get stuff like their own IP address and more. Indeed it seems like the underlying tech is superior. But your point is on target too, the tooling has brought the dev experience to the front with docker.
https://blog.uidrafter.com/engineering/freebsd-jails-network...
Why VNET? "VNET jails give each jail its own isolated copy of the network stack" https://klarasystems.com/articles/virtualize-your-network-on...