
Debian developer prompted to revisit FreeBSD after 20 years - liotier
http://changelog.complete.org/archives/9317-has-linux-lost-its-way-comments-prompt-a-debian-developer-to-revisit-freebsd-after-20-years
======
penland
That's a good overall article - I've been switching back and forth between
FreeBSD and Debian for several years now, and I never spend more than a couple
of month without using one of them.

The one thing that surprised me was his take on FreeBSD's package management
system, pkg. PKG is relatively new, having been released in late summer 2012.
Prior to that, FreeBSD relied on the ports collection, which was (is) a vast
tree of Makefile's allowing you to create custom builds of virtually any
software imaginable.

While pkg is still raw on the edges, I VASTLY prefer it to Debian's hodgepodge
of package management tools that all do 90% of the other, though none of them
do it cleanly and none of them have straightforward interfaces. Am I using
dpkg here? What about aptitude? Or should I just roll w/ apt-get? All of these
tools combine the ease of use of git with the flexibility of a Maven build. If
you know how to use to descend deep into the depths of git or Maven, this
isn't an issue ( and for the author, it certainly is not ). Yet there are
numerous simple tasks ( like say, searching for a package remotely when you
aren't sure the exact name ) which still require hitting up google and
settling in for a Click-Your-Way-To-Adventure session.

PKG, in contrast, is both simplistic and flexible ( it reminds me of a
industrial grade version of Brew in some sense ). It's a tool that I find
myself integrating into my workflow beyond simple installs / updates,
particularly it's seamless integration with jails. Configuration of remote
repositories is vastly simplified as well.

PKG is not perfect by an means - it definitely has worts that need to be taken
care of. I'm also sure that PKG probably feels a tad inadequte to some hard
core sys admins - pkg has the look and feel of a tool designed by a programmer
looking to handle normal cases than a tool designed to provide a sys admin
with an atom bomb if necessary. Yet the system is still relatively new and as
the bugs continue to get ironed out it's a reminder to me of why I love
FreeBSD in a lot of ways - the abstraction point is flawless and it gets out
of my way.

~~~
weland
Do you know of a non-broken way to build ports with many dependencies? This
has always been my biggest annoyance and one of the things that constantly
made me pick the _cough_ other BSD. Whenever something that I need isn't in
the package set, all hell breaks loose: most ports seem to have most options
turned on by default (generating a lot of dependencies) and there seems to be
no easy way to select configuration options _prior_ to starting the whole
build process -- so I end up having to sit for the next two hours, pampering
the build process and tuning (actually, mostly disabling) options every time
it moves on to the next port. Needless to say, this isn't fun, especially
since there's no easy way to tell what dependencies an option is going to pull
(I remember being able to retrieve that information through a script I cobbled
up but...).

I could probably script it to just go ahead with the default dependencies, but
that gives terrible results (I try to install this little console tool and now
it's pulling gnome-something-something because the console tool can be
integrated with ImageMagick and that's compiled with support for this-weird-
format which is provided by this-weird-lib which is part of the Gnome project
and now I'm slowly compiling half the packages in Gnome even though _I don 't
use a single program from the Gnome project_).

~~~
Freaky
Add OPTIONS_UNSET=gnome to /etc/make.conf, plus whatever other options you
want excluded system-wide. Ditto OPTIONS_SET. You can override for specific
ports with ${PORTNAME}_SET/_UNSET too.

~~~
weland
Ha! Yes, this would make it a lot easier, since I often end up disabling the
same things anyway. Thank you!

------
Alupis
The article makes some good points, and I'm glad it didn't just rail on one
single part/component since linux distrobutions have so much variety. ZFS is
really a killer FS, and unfortunately Linux just doesn't have a native answer
just yet (although BtrFS does have some promising parts).

There's a lot of people either threatening to switch or are switching to one
of the BSD's as a result of being dissatisfied with the systemd debate outcome
-- apparently not realizing FreeBSD (and others) are in the early stages of
working on their own systemd/launchd-like init system. Everyone seems to
realize the need for a more modern init system...

~~~
vezzy-fnord
FreeBSD has been in the early stages of working on a launchd port for 10 years
now. It seems like progress with openlaunchd is going well, but there are
certain things related to Mach IPC that need to be ported to the upstream
FreeBSD kernel to get it working. I do recall some old version of launchd
entitled launchd_xml was ported to pfSense, but it's one that puts the XML
parser into PID1, so it's not desirable.

IMHO, I wouldn't count on any major init changes in FreeBSD any time soon.
It's been protracted enough already, and Jordan Hubbard giving a pep talk
doesn't mean all that much. I could very well be mistaken.

The differences in scope between launchd and systemd are pretty big, too.

~~~
throwabob214
> FreeBSD has been in the early stages of working on a launchd port for 10
> years now.

There were several half-serious individual efforts, but I don't think the
community / foundation were pursuing it seriously until quite recently. Kip
Macy has been working hard to get it in as an init replacement recently.

> It seems like progress with openlaunchd is going well, but there are certain
> things related to Mach IPC that need to be ported to the upstream FreeBSD
> kernel to get it working.

The mach stuff has been ported (recently), again by Kip Macy.

> I do recall some old version of launchd entitled launchd_xml was ported to
> pfSense, but it's one that puts the XML parser into PID1, so it's not
> desirable.

Right. I think the current plan is that the parsing is done in another process
and then IPC'd to launchd via Mach. Or the plist stuff could be replaced with
a 'libucl' parser that is safer in init? I'm fuzzy on these particular detail,
sorry.

> IMHO, I wouldn't count on any major init changes in FreeBSD any time soon.
> It's been protracted enough already, and Jordan Hubbard giving a pep talk
> doesn't mean all that much. I could very well be mistaken.

It's going to happen soon. :-)

~~~
gonzo
>> FreeBSD has been in the early stages of working on a launchd port for 10
years now.

> There were several half-serious individual efforts, but I don't think the
> community / foundation were pursuing it seriously until quite recently. Kip
> Macy has been working hard to get it in as an init replacement recently.

This is because iXsystems (Jordan Hubbard) hired Kip to implement it for
FreeNAS (and, presumably PC-BSD), not because the FreeBSD community thinks it
needs a new init system.

Jordan also has plans to bring other large pieces from Apple to FreeBSD.

~~~
throwabob214
> This is because iXsystems (Jordan Hubbard) hired Kip to implement it for
> FreeNAS (and, presumably PC-BSD),

I believe EMC Isilon has also sponsored the work.

> not because the FreeBSD community thinks it needs a new init system.

Right. Kip is working on it because someone hired him too. But practically,
this is how large-scale efforts get done. Some FreeBSD-using corporation hires
/ builds out a larger project in a way that is acceptable to the community and
it is incorporated into base.

I do think the community has come to more or less that consensus at this
point, though. The camp that is fine with /etc/rc hasn't started raising loud
complaints about the idea incorporating a backwards-compatible launchd to
replace rc.

~~~
digi_owl
And even now launchd is a more restrained projects than systemd.

------
vezzy-fnord
_Apparently this is because FreeBSD has nothing like Linux’s inotify._

It has kqueue, which is a generic event mechanism that can take the place of
inotify, signalfd, eventfd, timerfd and others on Linux systems. It has a bit
of a learning curve, but it's actually neat.

Concerning platform support, that has always been NetBSD's shining ground.

~~~
SpartanJ
This is true but not entirely accurate. The use case that he's mentioning, an
application like Dropbox, that relies on a good file system watcher wouldn't
be totally feasible with kqueue, since you're pretty much limited by the
number of open file descriptors the OS allows to handle, if i don't remember
wrong this number in FreeBSD is below 10000 by default, so it's limited for
that use case. And that's also why OS X have FSEvents, because kqueue falls
short for some use cases. I've worked on a multi-platform file system watcher
( [https://bitbucket.org/SpartanJ/efsw](https://bitbucket.org/SpartanJ/efsw)
), and i must say that the kqueue implementation was by far the worst to work
at. On the other side inotify is super simple and just works ( but sadly it's
not recursive like FSEvents and IO Completion Ports ).

~~~
throwabob214
> pretty much limited by the number of open file descriptors the OS allows to
> handle, if i don't remember wrong this number in FreeBSD is below 10000 by
> default

And on Linux inotify is bound by /proc/sys/fs/inotify/max_user_watches. Six of
one, half dozen the other. FreeBSD file descriptor limit is arbitrary and can
be raised to INT_MAX (2 billion). That should be plenty.

> i must say that the kqueue implementation was by far the worst to work at.

I agree that the kqueue API is a pain to work with. I haven't tried to do a
recursive file monitoring (dropbox-alike) application with it.

It seems easy to implement inotify as a thin wrapper around kqueue. Maybe
something to do for the FreeBSD linuxulator layer...

~~~
SpartanJ
> And on Linux inotify is bound by /proc/sys/fs/inotify/max_user_watches. Six
> of one, half dozen the other. FreeBSD file descriptor limit is arbitrary and
> can be raised to INT_MAX (2 billion). That should be plenty.

Yes, on Linux you're limited to max_user_watches, and you can edit that value,
just like you say with kqueue... but you need root access, so for end user
applications this is not really a solution. With inotify i use one watch per-
folder, with kqueue you need to do it by file ( of course you can watch the
folder changes, but you need to keep a copy of the file states in that folder,
and then re-stat that files to find out what file was changed, not even close
to an ideal solution ). So for example in my use case with inotify i can watch
65536 folders, and with kqueue less than 10000 files ( that could be a single
folder! ).

Edit: It seems that FreeBSD default a much bigger number for FDs (
[https://news.ycombinator.com/item?id=9063893](https://news.ycombinator.com/item?id=9063893)
), so that should suffice a lot of use cases, that's good news for me!

~~~
throwabob214
> With inotify i use one watch per-folder, with kqueue you need to do it by
> file ( of course you can watch the folder changes, but you need to keep a
> copy of the file states in that folder, and then re-stat that files to find
> out what file was changed, not even close to an ideal solution ). So for
> example in my use case with inotify i can watch 65536 folders, and with
> kqueue less than 10000 files ( that could be a single folder! ).

Ah, I looked at inotify API briefly and did not notice that directory watches
return file changes. That does make it easier to use than kevent vnode
watches. Mea culpa.

------
cplease
Those who switch operating systems out of ideological disillusionment are
liable to end up as disillusioned again as those who switch religions out of
religious disillusionment. You may think the new one is more pure, but it's
really just more novel to you. The old timers have been around for their own
iterations of in-fighting and may be happy to see fresh blood but probably
won't be happy to see rehashing of hardened partisan battles.

Linux hasn't lost its way, it has evolved over the past 20+ years as UNIX has
evolved over the past 40+ years. So have the BSDs--all of them.

------
xenophonf

        but there is no way with FreeBSD, like there is with
        Debian, to say “never even show me packages that aren’t
        DFSG-free.”
    

I would be very surprised to discover non-free software in FreeBSD's binary
package repository. However, if this is a concern, one needs to set
LICENSES_REJECTED in /etc/make.conf, at which point FreeBSD Ports will not
build anything with a license that matches. If a company's concerned about the
licenses of the binary packages provided by FreeBSD, it could simply set up
its own internal binary package repository using poudriere, and use poudriere
to filter out software with the unwanted licenses.

See /usr/ports/Mk/bsd.licenses.mk
([https://raw.githubusercontent.com/freebsd/freebsd-
ports/mast...](https://raw.githubusercontent.com/freebsd/freebsd-
ports/master/Mk/bsd.licenses.mk)) for the gory details.

~~~
emaste
A lot of non-Free software also disallows redistribution, so it's not
permissible to put it in the binary repository. It's possible that free but
not Free (e.g., zero-cost binary-only software) software could be included in
the repository.

pkg metadata does include license information -- see
[https://github.com/freebsd/pkg/issues/20](https://github.com/freebsd/pkg/issues/20),
but I am not sure of plans to provide user-facing control over acceptable
licenses at the moment. It's a good idea though and I'll see if we can make it
happen.

------
r3ddaem0n
"GEOM, however, supports only RAID0, RAID1, and RAID3. It does not support
RAID5 or RAID6 in software RAID configurations!"

Looks like the author is still learning FreeBSD and hasn't read all relevant
docs yet. 'man 8 gvinum' clearly says that raid5 _is_ supported:

"To create a raid5 array on disks /dev/ada1 /dev/ada2 and /dev/ada3, with
stripesize 493k you can use the raid5 command: gvinum raid5 -s 493k /dev/ada1
/dev/ada2 /dev/ada3"

I'd take this article with a big grain of salt.

~~~
teddyh
RAID5 is obsolete. In RAID5, if a disk breaks in an array and is replaced, the
system has to read all of the remaining disks to rebuild the array. If a
_single byte_ is unreadable on any of those remaining disks, your array can’t
be rebuilt and you have lost your data. With today’s disk sizes, the chance of
this happening is larger than is acceptable.

The interim solution is RAID6, which will work for some years to come.

~~~
Scramblejams
Not true with mdadm. Yes, your unreadable single byte will translate into a
corrupted file somewhere, but I've recovered plenty of mdadm RAID5 and RAID6
arrays that had lost one or both extra drives and suffered from UREs on the
remaining disks. Love good software RAID!

~~~
teddyh
I agree about software RAID – I use it exclusively myself. Also, nice to know
that an array can be recovered, even though it would seem to be a reasonable
thing to assume, it is good to know that it is possible.

But my point was that:

1\. You still have some lost data.

2\. An RAID5 array which breaks in this way must be _manually_ rebuilt,
possibly from a rescue boot media. Your server will no longer even _boot up_
by itself.

------
floatboth
"There are some other issues, too: FreeBSD ports make no distinction between
development and runtime files like Debian’s packages do. So, just by virtue of
wanting to run a graphical desktop, you get all of the static libraries,
include files, build scripts, etc for XOrg installed."

That's definitely a feature, not a bug! I hate how Linux distributions require
you to install a thousand " _-dev " or "_-devel" packages. All headers should
always be present on a Unix system!

Also, you don't need HAL/PolicyKit/dbus on FreeBSD. xorg-server recently
switched the default configuration option from HAL to devd. And you can
compile things like Firefox WITHOUT="DBUS" :-)

~~~
exacube
But not everyone that uses Linux compile X libs or Firefox themselves (a very
tiny fraction do). Why does it make sense to have include headers and source?

~~~
smutticus
Disk space is cheap.

If it really is a problem for you then work around it. But for most of us
installing dev files makes sense if we ever need to compile something against
those .h files.

~~~
cwyers
Even if it is, bandwidth isn't particularly cheap.

~~~
gaadd33
I think with compression most headers are probably tiny compared to even the
ads on a large widely read news site.

------
cthalupa
The article seems kind of odd on some of its logic:

>Its storage subsystem also has some surprising misses. Its rough version of
LVM, LUKS, and md-raid is called GEOM. GEOM, however, supports only RAID0,
RAID1, and RAID3. It does not support RAID5 or RAID6 in software RAID
configurations!

Well, actually, it does support raid5, but also... With ZFS being a truly
first class citizen, why not just use raid-z or zfs encryption instead of
raid5 or luks?

>ZFS also does not support some common operations, like adding a single disk
to an existing RAID5 group (which is possible with md-raid and many other
implementations.) This is a ZFS limitation on all platforms.

No block pointer rewrite for ZFS, so no expanding of existing vdevs by adding
disks, but you can still add more vdevs to increase the total pool size. You
generally want specific numbers of disks based on raidz level for performance
reasons anyway.

(In general) RAID isn't really a "consumer" thing, outside of maybe a basic
home NAS type environment. If you're doing this for a server environment where
RAID sees the majority of it's use, I don't think it's too onerous to have to
add more vdevs instead of a single disk.

> Apparently this is because FreeBSD has nothing like Linux’s inotify.

Nothing like? kqueue/kevent. It's not 'exactly' like inotify, but it's
definitely something like it. (It's also something like epoll, and others...
kqueue is fairly big and replaces multiple linux equivalents)

I find it odd that the network stack isn't mentioned either. It's still
significantly more performant than the linux network stack, and FreeBSD is
pulling away even farther in this arena.

Things like
[https://wiki.freebsd.org/NetworkRSS](https://wiki.freebsd.org/NetworkRSS) ,
netmap being a first class citizen, etc (Netflix blogs on why they chose
FreeBSD for their CDN devices are fascinating) are all pretty damn sweet.

Kind of nitpicky, I guess, especially since the author isn't claiming to be
intimately familiar with the state of every part of modern FreeBSD

------
_paulc
I think that a lot of this depends on whether you want to run a desktop or
server - a lot of comparisons focus on how Linux/FreeBSD compare as a user
workstation. To be honest I wouldn't recommend anyone use FreeBSD as a
desktop/laptop OS but if you want a 'production' server there is a lot to be
said for simplicity and avoiding too many 'clever' features.

------
jorgecastillo
If you happen to try FreeBSD don't let that be your only impression of BSD
based systems. Try OpenBSD too, if your hardware is supported and all the
software you use is available you won't be disappointed. OpenBSD more or less
just works provided the previous conditions are true.

------
hobarrera
It's a shame FreeBSD was the chosen BSD to do this comparison. It always
seemed to me that FreeBSD felt almost-like-linux.

It also seems to me that _most_ of the issues mentioned would be non-issues
for OpenBSD (though it does not support ZFS, which the op wants).

------
dorfsmay
I saw some @HNstatus tweet about ZFS issue that shows that it runs on FreeBSD.
Are they running the storage layer on FreeBSD only, or everything?

------
gtrubetskoy
I too started out on FreeBSD. Another feature that I don't think you
mentioned, and don't know whether it's supported with ZFS was _union mounts_ ,
where you can mount one filesystem atop another (as many times as you'd like)
and thereby get yourself a COW filesystem. I think there was a method to run
FreeBSD from a CD-ROM using this?

------
mavhc
Package management was my main problem, all I wanted to do was keep my home
server up to date, but it became a nightmare. Moved to debian/kfreebsd (where
a lot of stuff just didn't work) and then debian/linux when ZOL became stable.

------
davidgerard
Yeah. I was an ardent FreeBSD fan for years - it pleased me in all the ways
Linux didn't - but then I tried Ubuntu 5.04, discovered Synaptic and went
"HOLY CRAP THIS IS SO OBVIOUSLY THE RIGHT WAY TO DO THIS."

------
smegel
> and solid memory management

What does he mean by this?

~~~
feld
You'll often find FreeBSD using less memory to run the same software stack. I
also tend to see more intelligent use of swap. I can't comment on the
comparison of the filesystem cache performance.

------
eleitl
He seems to think in his comparison Debian comes on top.

However, the points which he thinks are where Debian is stronger illustrate
precisely where Linux has lost its way. Laptop support is lacking -- assuming
you care about laptops, and care about suspend working out of the box. Guess
what, suspend didn't work out of the box under Debian, either.

Storage, he's forgetting that zfs is both a filesystem, and a volume manager.

Support of alien file systems, really? Irrelevant for servers, for everything
else mount it via NFS.

Virtualization -- between jails and Virtual Box (on desktops) what is the use
case for a different product here?

Nothing like Dropbox -- that's not a bug, it's a feature.

Desktop environments need configuration -- why I am migrating from Linux, you
think? If I already have to use alternative window managers and desktops on
Linux, where is the big difference?

Package managers, now that is a good point. XML config, that too. Nigh
everything else is missing the point.

~~~
lutoma
> Nothing like Dropbox -- that's not a bug, it's a feature.

What a horribly arrogant and conceited thing to say.

~~~
vezzy-fnord
It's also wrong. FreeBSD supports the best there is: Tarsnap.

~~~
smcgivern
I use Tarsnap and Dropbox. They aren't the same at all:

* You run Tarsnap whenever you want to push changes; Dropbox automatically pushes changes.

* Tarsnap is for 'real' computers only; Dropbox runs on my phone (which is very useful for photos, among other things).

* Tarsnap is billed by usage (at a very small amount); Dropbox has a free tier.

* Tarsnap doesn't have a natural sharing usecase; Dropbox does.

* Fundamentally, Tarsnap is backup, and Dropbox is sync. They are two different things. Not to mention that Tarsnap is also supported just fine on Linux.

