

PostgreSQL benchmarks under giant lock free DragonflyBSD - unix-junkie
http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00008.html

======
JoachimSchipper
Summary/background: DragonflyBSD split from FreeBSD (partly) over multicore
strategy, FreeBSD going with threads and fine-grained kernel locks and
DragonflyBSD going with a microkernel and multiple processes (not entirely
dissimilar to GNU Hurd.)

This post shows a standard PostgreSQL benchmark on a big machine (2 cpus * 6
cores/cpu * 2 hardware threads/core, 96GB RAM) and the news is that Dragonfly
is approaching FreeBSD's performance (while hopefully being more
flexible/secure due to the microkernel design).

As an aside, Scientific Linux handily destroys both FreeBSD and DragonflyBSD
in localhost benchmarks, but crashes rather a lot. (This seems odd, but the
focus is on the DragonflyBSD / FreeBSD comparison anyway.)

~~~
einhverfr
I don't know that the issues reported can be called "crashes." Rather it is
locking up, probably because there are not enough resources reserved for the
terminal and therefore the terminal becomes unresponsive while waiting for an
opportunity to run.

~~~
m_eiman
_It was also the less stable. I discovered more than once the machine was
completely frozen (blank screen, no keyboard or network response). Even a NMI
couldn't change anything._

Sounds like a rather hard crash to me.

~~~
seclorum
Were database transactions continuing while the terminal was non-responsive?
Yes. Thus, its not really crashed.

~~~
gaius
Hmm. I remember back in the day, when the display driver could crash on an NT
3.51 box, but SQL Server would still be up. We treated that as a crash.

A machine that was not responding on the network is getting its database
transactions from where exactly? It's crashed, even if the disks are still
spinning.

------
foobarbazetc
When you're "crashing" Scientific Linux (which I presume using a non-modified
RHEL 6 kernel) then you're doing something wrong.

We run far bigger PostgreSQL DBs than this, and we've _never_ had a RHEL
crash. Ever.

Maybe there's something wrong with the hardware?

Additionally, if fsync = on in this test, then you're mixing a file system
test with a database test.

~~~
fdr
> Additionally, if fsync = on in this test, then you're mixing a file system
> test with a database test.

Which is reasonable, given that file systems of any reasonable performance
level (read: not user-space file systems) still tend to be bundled with one or
at most a handful of reasonable kernel implementations. Turning fsync off
would highlight some interesting bottlenecks, but I'm not sure if that makes
for the most sensible comparison either. Not only is the fsync/checkpointing
period of a database's operation important to model in a broad-spectrum
benchmark, but some amount of file system coupling is unavoidable. One example
is metadata syscalls to the file system, which have been a point of poor
concurrency before[0]

I agree the fsync = off benchmark would be _interesting_ , but I don't think
it's fair to say that the information of an fsync = on test is a) less
valuable/clearly improperly performed b) necessarily indicative of the most
important bottlenecks for database workloads. A kernel that takes a huge lock
during fsync(), for example, should be penalized for it.

Finally, fsync = off doesn't accomplish the stated goal of "independence" from
the file system. And as seen in Robert Haas's post, non-fsync file system
calls can be the problem in high concurrency situations.

[0] [http://rhaas.blogspot.com/2011/08/linux-and-glibc-
scalabilit...](http://rhaas.blogspot.com/2011/08/linux-and-glibc-
scalability.html)

