

Performance comparison of FreeBSD 7.0 to FreeBSD 4.11 and Dragonfly BSD 1.12 - gongfudoi
http://people.freebsd.org/~kris/scaling/dfly.html

======
cperciva
A bit of background information here about FreeBSD: For pretty much all of
2001 through 2006, FreeBSD kernel development was centered around making SMP
work -- getting rid of the "Giant" lock, and thereby making it possible for
multiple CPUs to be operating in the kernel at any given point.

Starting about a year ago, several prominent FreeBSD developers started
looking very seriously at performance (how fast FreeBSD is) and scalability
(how much faster FreeBSD gets when you have more than one CPU). A key part of
this was putting in place the necessary test infrastructure to allow FreeBSD
and Linux to be compared -- most of the dramatic performance improvements over
the past year have resulted from noticing that Linux does better than FreeBSD
on a particular benchmark, and tracking down why.

At this point, FreeBSD scales linearly (or until it hits hardware or
application performance limits) up to 8 CPUs on pretty much every benchmark
available, but on some benchmarks doesn't scale very well with 16 CPUs.

~~~
SwellJoe
Colin is too polite to point out that Dragonfly was an "I'm taking my ball and
going home" moment in FreeBSD history. Matt Dillon, a long-time FreeBSD
developer (and all-around impressive guy, though this episode in history isn't
his finest hour), had some disagreements with some other long-time FreeBSD
developers, and one of those disagreements, perhaps the most important one,
was over SMP and the direction FreeBSD was moving with regard to SMP-at-the-
expense-of-UP performance.

So, the irony lies in the fact that current FreeBSD is faster than current
Dragonfly in both UP and SMP environments. If one were keeping score, it would
be:

FreeBSD core developers: 1

Matt and his "screw this, I'm leaving" crew: 0

To be fair, Matt has been willing to talk about some major issues in FreeBSD
that are considered verboten by others. Package management in FreeBSD, for
example, is a joke, and ports is waved around as a solution rather than the
very serious problem that it really is. (No fault of cperciva's, as he's done
the only really good thing in FreeBSD package management in years in the form
of freebsd-update.)

Not that I have any strongly held opinions, or anything like that.

~~~
cperciva
_SMP-at-the-expense-of-UP_

This is an interesting topic, actually. When the process of improving FreeBSD
SMP got underway seven years ago, the initial work _did_ hurt UP performance
by 10-20% -- enough that in FreeBSD 5.x and FreeBSD 6.x, there are two kernels
shipped with the release: the GENERIC (single-processor) kernel and the SMP
kernel. In FreeBSD 7.x, however, we're back to only shipping a single kernel
-- one with SMP enabled -- because the performance penalty on UP systems for
having SMP enabled is practically nil. FreeBSD SMP development started out
needing a lot of faith to say "we know that we're making things slower every
day, but it's going to pay off in the end" -- and while it turns out that the
team which decided on a fine-grained sleeps-and-spins locking structure were
correct, I can't fault Matt for not having as much faith.

 _ports is waved around as a solution rather than the very serious problem
that it really is_

The FreeBSD ports system works very well -- as long as you don't mind
compiling everything from the source code. Binary packages... yeah, there are
issues there.

~~~
SwellJoe
"while it turns out that the team which decided on a fine-grained sleeps-and-
spins locking structure were correct, I can't fault Matt for not having as
much faith"

I agree, and I'd find it very hard to have anything but the utmost respect for
him. His C compiler for the Amiga, DICE, was where I began to learn about real
software development and the UNIX tool chain, so he holds a place nearly
comparable to K&R in my history as a hacker.

It's just unfortunate that some personality conflicts have led to a great
developer bitterly spinning his wheels for several years on something that is
clearly going the wrong way. (Then again, I can't say that of OpenBSD, as it's
spawned a lot of great stuff, though it may have looked like the same kind of
situation a few years into the project. NetBSD, I'm a little iffy on....)

"The FreeBSD ports system works very well"

Ah, you have to say that, or you'll wake up with a horse head in your bed.
It's the official party line. ;-)

Actually, to be more clear about my intent, the ports system does work very
well for the problem that it solves.

The problem is that it's solving the wrong problem, and no one will admit it
because it looks so pretty solving the problem that it does solve very well.
Because ports exists, and works really well for its problem domain, it has
prevented any real solutions from being put into place for package management
and has also caused a lot of weird and incompatible tools to spring up to deal
with various meta-problems like managing software on many machines (for
example). Meanwhile, over in Linux-land, not just one, but two, really good
package management systems (yum+rpm and apt-get+dpkg) have sprung up and made
it possible for many people to produce nicely bundled software (and provide
for the dependencies in an automated fashion) for Linux...while distributing
software for FreeBSD is a choice of many poor options, mostly ending up with a
tarball and a shell script.

Anyway, I think freebsd-update is a great improvement, and the right
direction. (I'm really not just blowing smoke...it makes me feel a lot better
about our customers using FreeBSD. Now, if only you could do something about
the silly secondary groups limit of 16, and fix the default username length,
I'd have nothing left to complain about...well, except for the other package
management issues.)

~~~
cperciva
_The problem is that [the ports tree is] solving the wrong problem_

I'm not sure from what you've written what you consider to be the right
problem -- can you clarify? Would you be happy if the FreeBSD ports tree had a
magical "make installbinaries" command which would download and install
binaries instead of building from the source code?

~~~
SwellJoe
Before continuing my rant, I will mention that I am a FreeBSD amateur and
tinkerer, and I know the Linux tools far better than the FreeBSD tools. I
could be entirely wrong in some of my complaints. I have spent the past two
weeks knee deep in FreeBSD package management and scripted installation of a
very complex system. These are the things that have caused me great pain, and
I've been unable to overcome them with repeated readings of the documentation
--that's not to say they can't be overcome. I just haven't been able to.

"I'm not sure from what you've written what you consider to be the right
problem -- can you clarify?"

There are actually many problems that yum and apt solve very well, that ports
doesn't even consider. Yes, the fact that ports builds everything from source
is, frankly, stupid (OK, not stupid...just solving the wrong problem...it
should be a tool used for building the distribution by the developers of the
OS, not for installing applications on every freakin' FreeBSD box on the
planet while we all grow old waiting on the damned build to finish).

I generally use pkgadd -rI to install things, and it works reasonably well
(still needs a lot of work--the functionality of pkg* is pretty dismal
compared to the rich capabilities of rpm and dpkg, particularly with regards
to upgrades and getting information back about what is already installed). But
the lack of binaries isn't really all that big of a concern to me. It's slow
and pointless, but some people like that sort of thing--heck, one nutjob even
built a ports style system for Linux and built a whole distro around it,
apparently because yum and apt-get were just too damned quick and easy for
him. ;-)

Among the issues that ports doesn't solve (and maybe even shouldn't--this
might be a job for extensions to the pkg* tools):

ports doesn't provide any tools for distribution of software other than that
provided by FreeBSD. As a third party software developer, I can't (easily) add
my own ports tree path to a configuration file and from then on my customers
will be able to use ports for my software, as well as the FreeBSD stock stuff.
Both yum and apt repos are available for dozens of projects and packages
outside of the "official" repositories, and it makes using Linux a whole lot
more fun. One can grab newer or custom versions of things, commercial software
that doesn't meet the license requirements of the OS repos, etc. And once
installed one can keep them up to date with a simple command ( "yum update" or
"apt-get update; apt-get upgrade" ).

Updating with ports is complicated and slow, so even if it were possible to
add third party repos, it would still be the wrong tool for the job. freebsd-
update actually manages this better than ports (but again, only for OS-
provided packages, and in a quite limited fashion). Distributing software to
FreeBSD users is generally done via a tarball with maybe a shell script...and
that's a valid option for some software. But we've got dozens of packages and
they are updated sometimes weekly (much of it is security related--for example
we distribute 80 or so installable web applications of wildly varying
quality...when Wordpress has a security bug, we have to roll out an update to
our installer for that script quickly and easily).

Installing software with ports is not scriptable in any sane way. It asks
questions using an ncurses interface, which, as far as I can tell, cannot be
disabled. This is abomination. pkgadd has the -I option, which means no
configuration is done, but at least it is non-interactive. It even seems to
handle dependencies reasonably well, though I may just not be hitting it hard
enough yet. Another few days and I'll begin the heavy lifting and I'll know
for sure.

So, I think those are the biggies, and being from source isn't really the
biggest of them. If our customers want to wait four hours for our tools to
install, that's not really my concern--it's their time, not mine. But, the
other stuff is serious: non-scriptable, no good tools for querying the
existing state of the system (versions, locations, etc.) and updating it, and
no support for third party software.

Anyway, those are the things that spring to mind. I'm sure I'll think of other
stuff to bitch about later, particularly when I'm back to work on the FreeBSD
installer for our product.

~~~
cperciva
_I can't (easily) add my own ports tree path to a configuration file..._

Good point. In most cases the solution is to have your application added to
the ports tree (unlike certain ungulate/avian operating systems, FreeBSD
doesn't mix up religion and computing, so this isn't only an option for free
software), but there are certainly cases where that isn't an option.

 _Distributing software to FreeBSD users is generally done via a tarball with
maybe a shell script_

I'd say that a more common method is to distribute a package (which can then
be installed via pkg_add).

 _[The ports tree] asks questions using an ncurses interface, which, as far as
I can tell, cannot be disabled._

This needs to be documented better: Add BATCH=YES into your environment, and
the ports infrastructure will run without pestering you. Of course, there are
a few ports which absolutely _need_ to interact with the user (e.g., to have
the user accept an EULA) -- if you set BATCH=YES these ports will print an
error message rather than installing.

~~~
SwellJoe
"I'd say that a more common method is to distribute a package (which can then
be installed via pkg_add)."

Same problems, though, if I need to distribute a dozen or so packages. No
repo-based software distribution means my user either has to download and
install them all individually. And, when it comes time to upgrade, they have
to do it all over again. So, we've had to write our own package management
stuff to deal with the frequent updates we roll out (not just
FreeBSD...Solaris also has crap package management).

"Add BATCH=YES into your environment, and the ports infrastructure will run
without pestering you."

Lovely. The squeaky wheel does indeed get the grease. I've gotta air my
grievances in public more often.

~~~
cperciva
_Same problems, though, if I need to distribute a dozen or so packages._

True. This is something which deserves more thought than it has been given by
FreeBSD developers.

 _The squeaky wheel does indeed get the grease._

No, the squeaky wheel gets told where the grease has been sitting for several
years. :-)

More seriously, BATCH=YES and DISABLE_VULNERABILITIES=YES (which is needed to
install ports with known security vulnerabilities if you have portaudit
installed) are two options which lots of people trip over -- they should
probably be better documented.

------
kingnothing
Looks like it's about time for me to consider making an upgrade from 4.11.

~~~
SwellJoe
I dunno...you wouldn't want to rush into anything...

~~~
kingnothing
I use my BSD box as a personal file server and never saw any reason to
upgrade. Why fix something if it isn't broken?

~~~
cperciva
For one thing, because FreeBSD 4.11 is no longer supported by the FreeBSD
Security Team (and hasn't been for over a year). Any security issues
discovered won't get fixed.

