Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Do you use FreeBSD as web server? Why or why not?
69 points by kiloreux on May 31, 2016 | hide | past | web | favorite | 65 comments

Yes, because it is a great platform to learn system administration. First, it's so simple that eg. it takes only few lines to automate installation or create a customised internal repository. Second, the documentation.

edit. And oh yes, the memory usage of the default installation is so low, that you can virtualize dozens of them on your laptop. This helps when you want to locally reproduce the stack you run in cloud.

Yes, I use it for my personal web servers. I completely rid myself of Linux when Debian switched to systemd. I run nginx for static and proxy services. I wrote some Scala/Play apps that I run in jails on OpenJDK8. I run some open source PHP, Python, Ruby and C and perl CGI scripts for various admin things and "cloud" things but all on my own hardware. I use iocage to manage the jails. I replicate the jails between two physical servers asynchronously, and also send them to my house for an off site. I run PostgreSQL on the two servers in a failover setup with WAL shipping. It takes about 30 minutes a month to keep security patched. I use saltstack to manage as much as possible. It took about 2-3 weeks to set everything up using just evenings and weekends, it's a lot more hacky than stuff I do for dayjob because I'm the only one that needs to understand the setup so I can elide some robustness I'd use in a team environment (i.e. no test suite, no branches, no CMDB just simple text files assigning roles, no multi-platform or abstractions everything is decisivly coded for my tech stack of choice like FreeBSD, Postfix, Dovecot, PostgreSQL)

I also use it at work. The scale is much different as one of the largest CDNs, the network interface stats are eye popping. This is proprietary caching software and nginx and apache for certain types of origins or admin services.


I have a high tolerance for trawling through documentation and reading the code of open source libraries in pursuit application logic and the fiddly bits of related services. I have low tolerance for the same work in pursuit of infrastructure configuration and systems administration.

I run various flavors of Debian and Ubuntu across a desktop, 2 laptops and a home server (various audio/video projects). Thus I tend to default to the familiarity of these systems, and get back to the part of the stack I enjoy working in. Another word for this might be "laziness", but that doesn't quite tell the full story.

At first, I thought you were being satirical in making a backhanded jab at the poor quality of documentation on Linux as compared to the excellent documentation in FreeBSD. But it seems that since you've already been through the pain on Linux, its poorer documentation is no longer a hurdle for you so you just go with what's familiar.

As work on Debian/kFreeBSD progress, hopefully you can have both!

Is work on Debian/kFreeBSD going to survive Debian's systemd transition? (I see articles from late 2014, but not sure the current status)

As a FreeBSD user and a Debian user, I was excited to try Debian/kFreeBSD around 2012ish? but it was kind of painful -- a lot of important FreeBSD userland tools weren't available in packages, so you had to do a lot of messing about to get things you might need. It might be interesting to run a Debian/kFreeBSD jail inside a regular FreeBSD install, but at that point, you may as well just bite the bullet and do things in regular FreeBSD.

From what I can see from keeping half an eye on the Debian/kFreeBSD Devel list[1], work is progressing nicely.

It should be noted that a) FreeBSD is doing work to keep software working without SystemD (an obvious point, I know, but it might not be obvious that this means that work also benefits Debian) - so long as we don't see any crazy dependency on SystemD from Apt or the installer itself (hopefully unlikely) - there shouldn't be any issues. b) Debian/GNU Linux still supports more than one init-system[2] -- at least for now.

While the long fingers of SystemD reaches for more and more of Linux, as long as developers are supporting other systems at all, there should be a way forward for distributions that doesn't want to incorporate SystemD. So this includes stuff like native Windows support, building on OS X, and packaging for the various BSDs -- all that work that goes into the big packages should continue to benefit Linux users that want to avoid SystemD for one reason or another.

> I was excited to try Debian/kFreeBSD around 2012ish? but it was kind of painful -- a lot of important FreeBSD userland tools weren't available in packages

As far as I know, this should've improved quite a bit. The biggest bar for entry (last I checked) was that the nightly installer builds might not work out of the box (one possible workaround would be to install Wheezy/7.0 and then dist-upgrade).

To be clear, I've yet to play with Debian/kFreeBSD, so this mostly from impressions from the list (and I'm not quite caught up with the latest). AFAIK the installer should now support installing on zfs rootfs, for example.

[1] https://lists.debian.org/debian-bsd/

[2] https://wiki.debian.org/systemd#Installing_without_systemd

Thanks for the detailed response (and link to installing without systemd!)

HN does, because pg and rtm liked it from way back, we inherited it, and it works well.

By the way, when are you going to come to a BSD conference and talk about running HN? We love "view from the trenches" talks, and I'm sure Arc makes things interesting in unusual ways.

Back when HN was still running with millions of files in one directory on UFS, maybe there was something to talk about (not much more than sysctl tweaks). Oh, and UFS snapshots (for tarsnap to reference) sometimes panic'd the box.

When I left we'd added nginx in a jail and switched to ZFS, which solved most of the problems we had been having. There were a few ZFS deadlocks, but I never got to the bottom of it and FreeBSD 10 seemed to fix them. I'd imagine things are even less notable now.

UFS snapshots (for tarsnap to reference) sometimes panic'd the box.

Ick. Yeah, UFS snapshots had issues for a while... had I known you were using them I would have encouraged you to switch to ZFS sooner.

There were a few ZFS deadlocks, but I never got to the bottom of it and FreeBSD 10 seemed to fix them.

Yes, FreeBSD 10 definitely fixed a bunch of ZFS deadlocks.

It would be a boring talk. It just works and that's about it.

Arc does make things interesting in unusual ways, but not on an OS level.

If you don't mind, can you list HNs entire stack: OS, external web server, app language, db?

The OS is BSD. Everything else is Arc IIRC.

I've seen nginx failures recently, so there might be a nginx reverse proxy hanging out in front of Arc (the whole stack noticeably behind CloudFlare, too, but that's sort of beyond the original question).

That'd be how I'd build it, if I'm right. Doing webby things without nginx is like forgetting pants, to me.

Excuse my ignorance, but what is "Arc"?

Arc is a lisp variant: https://en.m.wikipedia.org/wiki/Arc_(programming_language)

The current[1] distribution comes with a hn clone, that as far as I know, is mostly equivalent to the code that used to power hn at some point in the not too distant past.

[1] current might not be the best name for it, but as far as I can tell it a) works, b) has been ported to Racket (as opposed to being trapped on some ancient version of PLT Scheme), and c) is maintained (ish): https://github.com/arclanguage/anarki

A Lisp. (The nerdy kind. Not the verbal kind.)


Cool, thanks. for some reason I always thought HN was using github.com/reddit/reddit

Isn't CloudFlare the one using nginx?

The nginx errors I recall being unstyled and I think CloudFlare hides or styles theirs. I consider CF tossing an unstyled nginx error fairly unlikely anyway, but that's a gut feeling. Might be wrong.

I started out using fbsd back when 4.4 was a thing, and the performance was much better than anything GNU/Linux could do, for running Apache.

Back then fbsd had something called accept filters, which basically made the accept call only return when a http request was received in the network buffers.

Systems running fbsd loaded up different than Linux, kept being responsive. On same hardware, could do 3-5x more rps with lower load iirc.

As my applications at the time became more cpu bound, I switched to Debian around 4.11.

I've recently been using 10.x, coming back to fbsd from 10 years of Debian. I love what you've done while I was gone. Especially binary packages and easy upgrades, but especially jails and zfs.

Yes, on a small fleet of node.js (and some other services) VPS' on DigitalOcean in production, after a year or so playing with it for personal playthings.

Default system uses less memory than Ubuntu (~50MB last time I checked which means something on .5, 1GB RAM instances). I could swear memory consumption in general has fallen down, but haven't done any measurements.

I don't have to jump all over the web to figure out how something works - I just go to handbook. Which means one gets tendency to study topics instead of copy-paste snippets from around.

Since base system (maybe things were different before) does not abruptly change or pull the rug under your feet, combined with handbook studies, this has the effect of compounding knowledge that'll be effective in years to come.

Firewall (pf) is a thing of beauty. I've barely scratched the surface of it.

Base/userland abstraction split, as well as consistency throughout the system helps one maintain a solid mental map of the system. After a short while, you just know where stuff is, how to configure it, and where its defaults are even when you're in completely new territories.

Bonus: You can really go the distance without compiling anything, but if you're keen on maintaining a low memory profile by dropping various features from your packages, this is a great new build system I use to have all my pkgs up to date - https://github.com/jrmarino/synth

Yes. Because it is simpler. Less moving parts. This means more predictable interaction between components. No systemd. Slightly more efficient and faster. Quicker startup. And ZFS is useful for snapshots.

Absolutely. It's the only OS where I can be certain that I know where everything is and how everything works. It has few deviations from the Unix architecture and the ports repository enforces adherence to the filesystem layout.

I don't buy into the "increased performance" though. Every well done benchmark I've seen is at least several yaers out of date, and my personal benchmarks usually put linux ahead on any sort of operations that include networking. Not to say FreeBSD doesn't perform well, it does.

Simplicity is key.

Yes because it has lower response latency and performs about 12-15% better than Ubuntu on top of much lower CPU and RAM usage on AWS for hosting the same nodejs apps.

Quantify what you mean by "better," please. I'd assume you're still talking about latency (quantify that, too; TTFB? TTLB? Lower median? p95?) but there's an "and" in there, so I'm not sure. RPS? Throughput? Utilization? Fan0 RPM? Revenue?

I might have been kidding at the end there.

Pardon the questions -- I'm suspicious (seems high) but willing to be surprised.

Using for 16 years because I want little hassle from my machines. FreeBSD tends not to blow up everything for new features, yes major technologies have been introduced but rearly at the detriment to others.

I use it. I use a pretty standard stack - nginx, postgres, php and a few other things so I don't have the issues some of the other posters have mentioned here about needing to hunt down source and patches and everything is installed via the standard package manager pkg.

One thing I really like about it besides the things others have mentioned is that by default it sends emails every day/week/month regarding security and system usage statistics, so it tells me when a security vulnerability has been found for a package I have installed and that I should upgrade it.

I've found this makes me much more likely to keep things up to date.

Yes, because Jails and ZFS are cool. FreeBSD has superb documentation and a giant software repository.

Yes, all the other servers where I work are FreeBSD, why would we run something else for www?

It works fine, but Linux is probably ok too. There's some things you need to tune if you're high traffic -- most of them are sysctls you can tune at run time, but there's a couple boot time values; basically if you have enough ram to do webserving at 10gbps, some of the auto tuning for network buffers are going to be oversized, you'll likely actually allocate that much during peak, and FreeBSD won't return free'd network buffers to the overall pool; so you can't just drop the per socket sysctls, you need to reboot with lower network buffer caps (and per socket buffers, probably too). Depends on what kind of traffic you're serving though, and I'm guessing a lot of people won't hit 10Gbps out anyway. There's some other minor tweaking of that nature required too; and I would suggest running 10.3-RELEASE, if possible, there's some useful bug fixes in there.

Yes. Because it just works.

No[t yet].

Migrating to FreeBSD for both dev and server work is on my personal roadmap for around June / July this year, once 11.0 is released and my laptop chipset is supported.

laptop chipset

You're talking about Intel video driver issues, I assume? If you can, I'd suggest trying 11.0-ALPHA1 (https://lists.freebsd.org/pipermail/freebsd-snapshots/2016-M...) in case there are bugs you can report. It's always difficult with new hardware to get enough people testing before the release to get everything fixed.

Yup. I hadn't realised there was an alpha out; last I checked 11.0 was still on nightlies. I'll definitely give the alpha a look, and try it out if all seems good. Thanks for the heads-up :)

The release cycle just started. It's basically a weekly snapshot with a different name, but we've moved into "code slush" (prelude to "code freeze") so from now on there should theoretically be more bug fixes and fewer new features.

I gave it a whirl - both in FreeBSD and PC-BSD forum - but couldn't get anything other than VESA drivers running.

Yes, because pf is how a firewall should work.

Yes, because I rather enjoy using FreeBSD.

Yes, because I wanted to reduce response latencies, and wanted something efficient to serve more people with minimal hardware. (I might go unikernel later, once I build a custom HTTP server.)

Linux also seemed to have changed quite a bit since the last time I used it in the 90's/early 2000s. With systemd, it doesn't seem like Unix anymore.

I also looked up recent Linux distributions to use, and was largely overwhelmed at the choices. I looked up FreeBSD, and it was just one distribution.

I use FreeBSD for hosting personal web sites. I've used it on and off since version I like BSD in general and have hosted small web sites using NetBSD and OpenBSD. I came back to FreeBSD, mostly because of the broader ports support and binary updates. The packages do most of the work for setting things up. With NetBSD and OpenBSD, there is more integration work to do at setup time. I'll add the caveat that this is for small all-in-one web sites.

What reliable and reasonably inexpensive hosting services allow you to use FreeBSD?

I've been using Vultr (https://vultr.com) for a couple of years. Good, reliable service and responsive, helpful support. Very documentation too. Cost is lower than most competitors, especially for FBSD instances, though I haven't checked around for a while.

DigitalOcean does as a droplet. Linode does offer it as well, but currently afaik it's not fully supported so their automated backups don't work, which can be solved with zfs snapshots and sending them to your own backup instance.

No, because the JVM is more supported, tested, used, and developed for linux.

This is no longer really valid with OpenJDK 8. I run a production Hadoop and HBase cluster on FreeBSD at work. I run Scala applications on my personal FreeBSD servers.

What about performance, have you benchmarked this? Java 7 was in my experience 50% slower on FreeBSD.

Something sounds suspicious there, I would use DTrace to see what is going on with locking. I have no performance complaints, including the HBase cluster that is very demanding.

Yes, because of it's superior networking stack

I see this a lot. Do you know of any good benchmarks comparing FreeBSD to Linux with similar hardware and whatnot?

As a user of FreeBSD and Linux, int's not the benchmarks of FreeBSD that draws me to it - it's the reliability. My FreeBSD firewalls just work, while my Linux ones need a reboot now and then.

Here's a job posting by Facebook that claims the same. http://www.theregister.co.uk/2014/08/07/facebook_wants_linux...

Yes, because it performs well.

Yes .... and I have for years, both locally hosted on services like Digital ocean.

Contrary to what others here have written, FBSD is no more 'fiddly' than any other server class OS. It is faster, smaller, and better conceived than linux - which is a achieving Microsoft-levels of accretion and cruft.

Moreover, FBSD is not constrained by the execrable GPL and its variants, uses a far more modern compiler chain, has better release control, etc.

I do use Linux as well - both professionally and for my desktops. Linux is a worthy replacement for the desktop, but Berserkely is an overall more solid server platform.

Did for years, prior to that I started on BSDi. Flirted with Linux a few times, but ran into dependency hell. Came back a few years later, the package manager finally worked, so I moved most things to Centos.

Yes, because upgrades can be done without fear and because of ports in general. To say nothing of better TTFB I've observed in my personal experience.

I am away from FreeBSD because today is OpenSource is mostly Linux-only development, no interest from software author to get application on FreeBSD, no support. Nothing ;-( If demand technology appear in FreeBSD it occurs many years after Linux. All companies invest money and develop pwoer only in Linux.

I use FreeBSD as my VPS OS. I use nginx as my web server. FreeBSD because I am used to it (used BSDi a long time ago, and stuck to BSD ever since), and is an excellent OS. Nginx because it is rock solid.

I'm doing a coreos + containers thing at the moment.

No, because it is not popular and i want to be popular and use linux.

Yes and I've upvoted all the posts with opinions I agree with as well.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact