
BSDs or Illumos may be good alternatives to Linux (2017) - O_H_E
https://www.adminbyaccident.com/politics/abandon-linux-move-freebsd-illumos/
======
craftyguy
So the main points of this rant are:

\- BSD has ZFS (but so do some Linux distros)

\- Linux bad because BTRFS, so it's bad (well, you don't have to use it)

\- BSD jails > anything on linux (I was hoping to see a clear outline of
pros/cons, but the author's reasoning is very hard to follow, and the tone is
very combative instead of constructive)

~~~
hacknat
At this point there really isn’t any security difference between zones, jails,
and hardened containers. That word “hardened” is the key. Zones are a
primitive in Illumos, containers are not a primitive in Linux, they are a
collection of security features. If you use them all then you can achieve
parity with Zones, but surprisingly few projects make use of all of them by
default (Docker certainly doesn’t).

The main difference then is usability, which is almost always the biggest
security flaw in any piece of software (that the security isn’t “on” by
default or has a prohibitive learning curve for the user).

~~~
posix_me_less
> _Docker certainly doesn’t_

Do you think LXD gets closer to Zones in terms of security?

~~~
hacknat
Honestly, I don’t know, but my guess is that it had some things turned off by
default. Actually K8s is the most encouraging project in this regard in that
it has opinions about how containers should work so they are slowly adding all
the secure options by default when containers are orchestrated to the various
runtimes. There are only a few features left that remain to be turned on (user
namespacing being the most notable).

~~~
technofiend
_I paraphrased you by cutting down your comment, hope that 's OK._

>Actually K8s [...] are slowly adding all the secure options by default [...]
(user namespacing being the most notable).

K8's RBAC is nice, and in particular auth-delegator really helps with odd duck
integration for the enterprise. But k8s is reliant on OS features to make user
namespaces happen and Redhat customers are still waiting for Redhat to
backport a newer version of shadow-utils [1] [2] to RHEL 7 so that user
namespacing works there.

Given enterprise certification and release cycles "Wait for RHEL 8" isn't
really a good answer for those kind of customers. I'm not disagreeing with
you, just saying some features of kubernetes aren't available everywhere.
Personally I'd _really like_ to see user namespaces come to RHEL 7 as it helps
with removing or mitigating privileged access.

[1]
[https://bugzilla.redhat.com/show_bug.cgi?id=1498628](https://bugzilla.redhat.com/show_bug.cgi?id=1498628)
[2]
[https://access.redhat.com/solutions/3657531](https://access.redhat.com/solutions/3657531)

------
setr
I'm naturally inclined towards BSD simply because of its extensive history
(though I've spent little effort towards it thus far), but even still this
article is wholly unconvincing... of anything. It really doesn't say anything
about ZFS, jails, or even the current state of affairs beyond a respectable
number of packages (and more importantly, linux compatibility); it just
assumes you already accept that they're better, and for some reason haven't
already switched?

It also badly requires editing

Hopefully someone has a better argument for FreeBSD than this... mess

~~~
AHTERIX5000
Yeah TFA reminds a bit of passionate blog posts and articles about switching
from Windows to Linux something like a decade or two ago. Something that would
fit better in BSD community such as Lobste.rs than HN ;-)

------
earenndil
> PS: I haven’t mentioned both softwares, FreeBSD and SmartOS do have a Linux
> translation layer. This means you can run Linux binaries on them and the
> program won’t cough at all

I don't know about SmartOS, but on freebsd, linux compat is at version 2.6.38,
and so most programs don't work.

~~~
RantyDave
I haven't run SmartOS for about a year, but I _was_ using it to run Linux
binaries. It was fundamentally great. Stable, efficient ... great. Joyent were
nailing compatibility bugs in a matter of days.

A couple of small things - one is that entire distributions "just as they are"
are unlikely to work, in much the same way as trying to boot a docker
container doesn't. Lots of things rely on /proc and cgroups and these are big,
ugly interfaces that are never going to see parity between the two OS's. /proc
is (amusingly) built as a r/o filesystem that has an htop-esque thing writing
state onto it constantly.

The big one is memory allocation. Linux will let memory allocations fail. It
will also just shoot apps in the head if it wants to. Solaris does not do such
shit and will return the malloc call when it has allocated some memory - even
if it's next week some time. On more than one occasion I have been completely
locked out of a SmartOS box because of this and one ends up having to be
really, really careful with allocation strategies as a result.

------
SpaceInvader
I have the longest beard (not yet white) in my company (1000+ plus employess),
I am primarily the AIX guy (800+ virtual machines) and I do use FreeBSD since
2002 iirc. I also do plenty Linux but AIX and FreeBSD, despite being totally
different, seems to be the rock solid option. Given AIX runs only on
freakishly expensive power platform FreeBSD can be a good alternative. ZFS is
great too.

~~~
posix_me_less
> _ZFS is great too._

Could you give some of your personal reasons for saying so? So much online
sources just say how great it is without any specific detail. It's not like
XFS on Linux has some obvious problems - it is rock solid too.

Let's say I want to run several different operating systems simultaneously
from a single big ass SSD. With Linux, XFS and Xen, I can try to set up LVM
and coerce Xen to use those logical volumes for VMs, but it is pain to get
there with the current state of XFS and Xen documentation. Is this
possible/easier with FreeBSD and ZFS?

~~~
reacharavindh
Just sharing my personal experience.

Context: I use XFS as the default file system on all HPC compute nodes, and
few other servers that run CentOS7. I built a FreeBSD+ZFS backup server
recently to mirror our primary NAS.

The experience with ZFS was so smooth and logical. All I needed was the ZFS
section from the FreeBSD handbook, and I actually understood what was going on
underneath to a reasonable extent. No magic - "run this command" without
explanation. Once spools are set, ZFS datasets provide the equivalent of
logical volumes you were asking for. You get a lot of tunables at that level,
and they come with sensible defaults. The whole process is simple, heck you
can even make a file act as a zpool for learning or testing purposes.

Once done, I was testing performance rudimentarily and saw that it was almost
saturating the network pipes beating my expectations without any time spent on
custom perf tuning.

And I get snapshots and online compression, SW RAID out of the box.

I may be able to achieve most of it with XFS, some specific hardware and lot
of hours of tuning, but it will never match the experience with ZFS.

ZFS blog post:
[https://aravindh.net/post/zfs_fileserver/](https://aravindh.net/post/zfs_fileserver/)

Performance:
[https://aravindh.net/post/zfs_performance/](https://aravindh.net/post/zfs_performance/)

~~~
posix_me_less
Thank you very much for this. I'm thinking about using ZFS on a single big
ssd, as a storage for several simultaneously running VMs. I'm aiming for
easier administration and advanced features like error detection/correction on
the drive. Would ZFS make sense in that case (single physical drive)? Or do I
have to put in more physical drives to get the actual benefits of ZFS, like
you did with NAS backup?

------
pmoriarty
BSD doesn't have systemd. That's a huge win right there.

~~~
bayindirh
I also despised having systemd for some time, but it's not giving big
headaches in our installations. So I adapted and got used to it.

Besides being a multi-role-somewhat-messy-blob, what are the grief points of
systemd in 2018?

Edit: carified the question.

~~~
jimpudar
I'm also wondering this. I've seen plenty of hate for systemd, but I've never
personally run into any issues with it.

~~~
bayindirh
administration differences, not being designed according to UNIX principles
and personal dislike towards developer(s) is coming to my mind.

administration differences is the most direct effect of systemd, but after
learning its various compatibility layers and watchdog functions, it can be
tamed to a great helper.

not being designed according to UNIX principles (composition of single-role
small applications, which are doing their jobs very very well) is also
bothering me, but we need to be pragmatic here. inter-process-communication
cannot replace in-memory-communication in all cases it seems.

The hate towards the developer(s) is stemming from his attitude, history (he's
also developer of pulseaudio IIRC) and somewhat from his coding style. It
seems like he's a kind of guy who can defend his work, do this in an
unorthodox way, make it work and got it accepted. At the end of the day, the
hate is a feeling and is personal. I think we need to be pragmatic here too.
Not that better communication from both parties will benefit everyone and
everything in the long run.

At the end of the day, we are using systemd, and it's working. If we want a
better system, we need to improve it. systemd is the state of the art right
now, and we should either write something better or improve it if we have any
beef with systemd itself.

OTOH, Beef with developers is something that should stay out of software and
community in general.

My original comment was trying to explore whether there are more reasons to
dislike it besides the original points that I stated above.

~~~
lloeki
> administration differences is the most direct effect of systemd, but after
> learning its various compatibility layers and watchdog functions, it can be
> tamed to a great helper.

It's both a negative and a positive. It's a change for sure but nowadays I can
log into an ArchLinux, Fedora, Ubuntu, Debian,... machine and just leverage
systemd instead of whatever local idiom (and its various assorted corner-
cases) the distro decided to use. It's becoming the happy path of the XKCD
comic about standards.

~~~
bayindirh
I also like the usage homogeneity, but not everyone thinks the same way.

I also like the loginctl and service auto-recovery functions TBH.

------
pierreprinetti
I'd love to switch to BSD, but

> Docker on FreeBSD is experimental.

[https://wiki.freebsd.org/Docker](https://wiki.freebsd.org/Docker)

I don't want to run yet another operating system just to have to run VMs on
top of it...

~~~
deathtrader666
Why not use FreeBSD jails?

~~~
bauerd
* Because my company runs OCI images (aka uses Docker), not jails.

* Because you can't orchestrate jails with Kubernetes, and that's what my company uses.

Docker won over the industry, jails and friends did not.

~~~
kev009
It really didn't. I can tell you with first hand knowledge the majority of the
internet by any measure, but especially high scale systems where I've worked
and know much of the small community, use neither Docker nor Kubernetes.
That's not any indictment on either technology it's just simple fact. Fad tech
appears a much bigger tiger in echo chambers like this website but the
majority of the companies aren't even using languages that are popular to this
crowd let alone tooling. Whether or not these tools have staying power or
contribute meaningfully to the industry the jury is still out on, and will
have to wait about 5 years to tell. In the past you "were an idiot" if you
weren't using C++ or CORBA or RPC or Java or SOAP or message queues or Ruby on
Rails or AWS or or Hadoop or OpenStack or blockchain or machine learning or..
all of these are fads and all that mattered then and now for consumers of the
tech is solving business problems cost effectively.

Software developers are an incredibly easy demographic to influence with
marketing. There is an industry of startups that have been minting money on
this for a long time. But it's important to know that a lot of software
development tools we use are just fashion statements.

~~~
bauerd
It's not about current market share but about adoption rates. Which container
technology/toolset/ecosystem is more likely to be adopted by the average
software shop today?

~~~
kev009
It's the wrong question unless you are in the business of profiting off of
container tech directly. What are you doing for the stakeholders delegating
your pay checks and the person authorized to fire you? If they are happy, then
it doesn't matter what the company uses. Most transactions and the world's
wealth are stored and processed on operating systems and frameworks most
people have never heard of which will outlast either of these by decades but
that doesn't mean you should or shouldn't use them necessarily..

~~~
bauerd
I'm well aware that there's tons of time proven software that serves us well
and will continue to do so for some decades. Yes, most of this is not written
in the PL/framework combo du jour, true. You seem to think I want everyone to
switch to Docker/Kubernetes. I don't.

~~~
kev009
Fair enough, and the parent topic about /switching/ between broadly similar
technology like operating systems is where departments usually go awry. My axe
to grind is about initial selection when starting from scratch (do you really
need fad tech? Statistically, likely not), or playing nicely in an existing
environment and being very careful with new tech (can the company internalize
and support fad tech? Statistically, likely not)

~~~
bauerd
OP asked what stops people from switching to BSD. I agreed that for me, too,
the missing Docker toolchain/runtime is a dealbreaker for me, as I _have_ to
work with these technologies on a daily basis. As Docker and friends see rapid
adoption, others will think twice whether they switch to a BSD and have to run
a Linux VM, too. I'm not sure what you're after here. You seem to think that
Docker and Kubernetes are "fad technology" and are inferior to some
alternative? Which orchestrator would you recommend? Which container tech?

~~~
kev009
Again it's not a judgement against the technology. What I'm getting at is the
talking that fad technologies won the industry is false and too early to call
like that; in the particular company you are working in docker and kube you
may have to use. In industrial terms, they are far from required and are not
used in the majority of deployments. Proprietary schedulers or other
deployment machinery are running most workloads. Distant second by cluster
size is Mesos, which has passed its fad phase.

I am currently using Nomad which works on multiple platforms and that is a
requirement for my particular setting. I don't currently use containers in
production environments and only see negatives to introducing them into this
particular setting. This is not general advice to use nomad, just a
counterpoint to any particular fad tech being an industrial requirement.

------
tmbsundar
I love BSD.

But, last time I checked NVIDIA didn't have the right CUDA drivers for BSD. Is
there any improvement on the status quo yet?

------
equalunique
Project-Fifo uses bhyve, zones, and jails to blend FreeBSD & SmartOS into your
own self-hosed IaaS. Worth checking out.

------
INTPenis
I came into IT on FreeBSD back in 2004. Was a real fanboy and couldn't stop
ranting about it. My first IT boss at the time got me into FreeBSD and
PostgreSQL without actually knowing why. I had no real experience to back my
opinion.

FreeBSD was my primary OS, both at work and at home.

To fast forward a bit I handed over my last BSD environment around 2012, last
Solaris environment around the same time. After that I only had Linux
responsibilities. Around 2014 I was running Debian as my primary OS and sent
on a RHEL course in Stockholm.

The course leader sold RedHat to me so well that the very first night at the
hotel I reinstalled my laptop from Debian to Fedora. Haven't looked back yet.

Been professionally building systems on Linux since before that. So I'd like
to say my perspective spans over both sides of this argument. I've tried to
summarize why I'd rather use Linux and why you might consider using BSD.

The only reason I can justify to use BSD is that the open source world needs
competition. It would be dangerous if we only had one OS.

But all my professional sense of getting things done and keeping them stable
tells me not to use it. Correction, I use OpenBSD on my router at home but
that's it. OpenBSD has a reputation of being very secure (perhaps earned in
part by having a relatively small user base) and I rarely need to login to
make changes. Because the user space tools are horrible compared to Linux.

And I believe most of that stems from the community being much larger and
therefore the software better tested and more mature.

I'm talking mainly about operational security and stability. I believe Linux
does this much better than any BSD.

Less critical bugs, less issues with package management, better maintained
packages and more binary packages available that don't require source
building. Pretty much all things very reliant on a strong community.

More programmers available to fine tune user space tools and make them hum. A
lot more docs available from hobbyists and officials.

Some of my friends today are hardcore BSD geeks and they often have to deal
with system breaking bugs, kernel panics even. I can't remember the last time
I saw a kernel panic in Linux. Almost every day I can see at least one issue
in our chats dealing with package management or kernel and system bugs. Bugs
in jails for example, or network management related to jails. They're often
following the latest releases to remedy these but that's even worse than using
Fedora on servers imo. Bleeding edge software just to avoid the bugs and be
able to use the latest so called virtualization technology available.

~~~
SpaceInvader
Are you sure about the security thing? Less critical bugs? Do you have any
data to back that up?

~~~
INTPenis
No I'm writing completely from my own experience running stable releases of
Linux distros such as Debian and CentOS.

Very rarely I've had to install Fedora on some intranet to get a specific
feature but I can count the times on one hand.

------
type0
About FreeBSD, I've heard that its' software is lagging behind many fast
updating Linux distros but is extremely reliable, how does Debian Testing
compare to something like TrueOS i.e FreeBSD-Current stuff?

P.S when I say lagging behind I don't mean the server things but rather on the
developer side.

~~~
hs86
The base OS and the installed 3rd party software (ports/packages) are on
different release models.

The base is either stable/LTS (~5 years support) or rolling if you track
-CURRENT.

For the 3rd party software it is either rolling or stable in short lived
quarterly branches.

Most server side software is usually pretty up to date but some desktop-
related tools can be outdated for a while. You might want to take a look at
[0] and especially at the commit history of some fairly popular desktops like
Gnome3 [1] or KDE5 [2]. They got updated recently but it took quite a while.

[0] [https://www.freshports.org/](https://www.freshports.org/)

[1]
[https://www.freshports.org/x11/gnome3/](https://www.freshports.org/x11/gnome3/)

[2]
[https://www.freshports.org/x11/kde5/](https://www.freshports.org/x11/kde5/)

------
wartakode
Recently i'm switchback to linux from openindiana, can't use eclipse (old) JEE
on illumos, as update site are not working well.

\-
[https://wiki.openindiana.org/oi/Eclipse](https://wiki.openindiana.org/oi/Eclipse)

------
2bitencryption
Curious, what's Darwin like? Is it much use outside of the Apple/macOS
ecosystem?

~~~
etaioinshrdlu
There used to be some projects using Darwin outside of macOS etc, but they are
long dead. People have apparently had a harder time even getting the kernel to
boot nowadays for various reasons.

------
pjmlp
Zero reasons why they would make a better 3D, UI/UX stack though.

------
writepub
The selection of software that simply works on Linux (especially Debian &
derivatives like Ubuntu) eclipses that on FreeBsd.

I've seen attempts at bringing the Linux ABI to FreeBsd, including the
familiar and popular apt package manager. If and when it's as easy to "sudo
apt-get install _package_ " on FreeBsd as it is on Ubuntu, it will see an
increase in adoption in both the sever-production setting, and as a
development environment.

~~~
ComputerGuru
> I've seen attempts at bringing the Linux ABI to FreeBsd, including the
> familiar and popular apt package manager.

That's ridiculous. `pkg` on FreeBSD is a thousand times more consistent and
user friendly. Perhaps you are talking about the days before pkg-ng (next
generation) became the default, but now installing anything is as easy as `pkg
install foo` and it just works. It even updates automatically so URLs aren't
out of date.

~~~
bayindirh
I have a deep respect for BSD family, and their way of doing things, but
regardless of OS and distro, apt is probably the most advanced package manager
currently.

When used with aptitude, it's very easy to use, extremely powerful and
flexible. Also, it has the most extensive knowledge on details of packages in
terms of dependency graphs and modification of files which belongs to
packages.

At least, that's what I saw in 14 years.

~~~
craigsmansion
> , apt is probably the most advanced package manager currently.

Apt is certainly the best package manager in my opinion. I haven't used rpm
based package managers in a while, but I doubt they don't have feature parity
at this point.

I think the magic ingredient of apt that makes it work so seamless is the
thousands of diligent maintainers behind it. That's the "feature" that none of
the other packaging systems have at the moment.

I also have a deep respect for the raw no-nonsense approach of the various
BSDs, but every time I try running it in a professional capacity, the apt eco-
system lures me back.

~~~
petre
I use both apt, pkg-ng and yast/zypper/rpm. Both apt and pkg-ng work well. Apt
has --really-long-switches that are easy to remember but not quite as many
shortcuts for them. Sometimes pkg-ng is annoyingly upgrading itself but that's
it. Rpm is rather annoying but zypper and yast make things easy to manage. I
remember the old RedHat 4.2 days when I had to work with rpm exclusively; it
has an inconsistent CLI with weird command line switches.

~~~
dmacvicar
You shouldn't compare rpm with apt. rpm is a different layer equivalent to
dpkg in the Debian world.

As you point out, apt layer/functionality is provided by zypper, yum, dnf,
etc. and the main difference is that it is not unified across the main rpm
distributions. Still, Fedora/Redhat yum replacement, dnf, is built on
SUSE/openSUSE zypper's solver library (libsolv).

~~~
petre
I know. Dpkg is more consistent anyway.

