So far I've narrowed down the list to Debian, Gentoo and FreeBSD. Am I missing any other? What are your experiences? By web applications I mean really any application, be it in Perl, Python, Ruby or even C, that needs to attain awesome uptimes with great stability and reliability, specially if I need to run several copies of it in parallel.
Debian and Ubuntu are both rock-solid, excellent choices for a server of any kind. I have all of the servers I work with running Ubuntu, with my own personal screw-off slice running Debian Etch.
I have a great deal of experience with the older versions of Fedora (the consumer Redhat distro) and slight experience with CentOS. I'm fairly certain that majority of Linux webservers out there are running some Redhat variant. I personally prefer the Debian/Ubuntu approach and stick with that.
Off the top of my head I know that all of the Mediatemple servers are running CentOS, and Digg runs Debian.
I think like SwellJoe said, if you're thinking about FreeBSD for the sake of going with FreeBSD and not really knowing why, then you don't want to be choosing FreeBSD. The same goes for Gentoo. Gentoo is a great distribution, but it's not for the mainstream plug-n-play let's-build-a-webapp! type of person. Gentoo, although featuring a pretty nice 'emerge' package management system, is going to require more work from you in the beginning to get it to a desirable state.
I guess there is no real answer to your question, Linux is fantastic in general as a web application server. Any distro with the base binary packages for a typical LAMP stack or even a (Rails|Django|Etc...)/Nginx configuration will treat you very nice until you reach the proverbial land of scalability. Once you see problems with your app, fix them. Don't worry too much about that right now. Also keep in mind that 99.99% of the time your application is going to be "distribution agnostic" so later on down the line if you decide you'd rather go with CentOS over Debian, or a BSD variant over Linux, you won't have any issue with that.
I'd focus more on writing a solid application and getting a solid product out the door on any of the distributions that were mentioned by the wise hackers here on HN, learn what you like as you go. As you learn more about the way Linux works and you configure things to suit your needs, the various unique characteristics of each will become more clear, and your "distro of choice" will become more evident.
Bottom Line: Go with CentOS or Ubuntu, and worry about these things as they become worries.
I agree with the CentOS or Ubuntu sentiment. The only piece of wisdom I'd like to add is to pick a distro that you know well. If and when the shit hits the fan, you'll be glad you know how to quickly and properly assess your bottleneck.
This has been asked repeatedly. Do a search. No, seriously. Search first.
I universally recommend Debian or CentOS for our customers (and we deal with thousands of folks building web applications and websites--I have a huge swath of data in my head on how much trouble each choice causes). Ignore this advice at your peril. CentOS is the one I recommend for newbies...just because the documentation out there is so much better, the support channels are so much more populous, and so much software is tested first on CentOS. I really like Debian, personally, and the package selection is vast and wonderful, but for a newbie CentOS is just a simpler choice.
FreeBSD is a fine choice, if you know you want FreeBSD and you know why you want FreeBSD. Since you've asked a bunch of strangers for advice, this is not you. So, FreeBSD is not a good choice for you.
Finally, and perhaps most importantly, you'd have to be retarded, or hate yourself, to choose Gentoo for a server.
Oh, yeah, hosting providers almost all support CentOS and Debian. Gentoo and FreeBSD, not as much. I don't know of any hosting providers that offer Gentoo, actually, and if I saw one that did, I'd pretty much write them off as amateurs. People who are passionate about FreeBSD are often extraordinarily savvy, and there are a few FreeBSD-focused hosts.
OK, I should have said, "that support Gentoo". Linode and Slice, both of whom I've talked to and know are extremely savvy people, let you pick your own poison...but they don't support what's inside the "slice" or "node".
EngineYard, I'm unfamiliar with. Maybe they just really know what they're doing. It's possible to make Windows run reliably on servers, if you really know what you're doing...that don't make it a good idea.
But, it changes nothing. Gentoo is, for most users, a horror show on servers. Probably less than 1% of our users use Gentoo, and about 50% of the batshit crazy problems (the ones of the "my Apache won't start and says there's a link error with blah blah blah, and when I try to run a PHP script it kills one of my goldfish...why does your software keep breaking my server?" variety...our software always get the blame, of course, since ours is the UI they see) happen on Gentoo.
I'm just trying to help, and trying to be emphatic that Gentoo on a server is begging to suffer. If you don't really know what you're doing (and often, even if you do), it is so easy to shoot yourself in the foot (or your whole leg, or nuke Idaho), it just doesn't make sense to run it on a production server.
I figure I have a lot more data than most people here, since we make systems management software for web servers, and we have millions of people worldwide using it. I don't intend to piss off the raving Gentoo fans, of which I know there are many, I'm just pointing out that Gentoo is a dramatically more complex system than any other distro, because of basic design decisions...and the amount of testing that any one Gentoo configuration ever gets is miniscule--practically nil--because there are so many possible combinations of "Gentoo system". If you enjoy Gentoo, that's fine. But, I know what happens when Linux newbies are set loose on a Gentoo system: disaster. And it happens fast, and in dramatic ways that leave them wondering what happened, and why they didn't keep better backups. I'll also point out that we support Gentoo with our products (what does that say about us? nothing good, except that we listen to what our users ask for). We try to accommodate them and help them, but the marketing for Gentoo has been so effective, and the quality control so ineffective, that it leads a whole lot of newbies down really painful learning paths. And the culture around Gentoo is one of "embrace the learning experience"...no matter how painful it is.
That's a fair way of putting it for the most part. Much better than saying hosts with it are retards or amateurish.
I don't think it is any secret that Gentoo is not meant for the inexperienced. On that same point, it is clear that by its design most Gentoo configurations are not officially tested. From this, it becomes clearer how to properly manage Gentoo servers: the same way any diligent admin would - stage, test, stabilize, deploy. That is the beauty of Gentoo as a meta-distribution.
It is far from point-and-click and I hope it stays that way.
Much better than saying hosts with it are retards or amateurish.
You're right. I'm not entirely politic sometimes. At least folks won't think I'm ambivalent about the issue. And, I pretty much do only intend my comments for the kind of folks who are asking strangers for help in their selection--obviously, people who've been using Linux for 10 years, or whatever, can use anything that suits them, and they'll be able to make it work really well. They also need a lot less help (so popularity doesn't matter), and have far better grasp of how things actually work (so they can see through the differences between distros and not having a standard initscript or rc script for some app doesn't matter because you can write it yourself).
"Finally, and perhaps most importantly, you'd have to be retarded, or hate yourself, to choose Gentoo for a server."
Oh my hosting provider is retarted :) They use gentoo only, no other stuff.
Maybe it's they the same thing as with FreeBSD, the know they want Gentoo and they know why.
Thanks for all the responses so far, guys! HN has helped me immensely already.
I should add that I am, by no means, a "newbie". I've coded and deployed many production web applications in Debian that are still running today, and although as a programmer my focus is in programming itself, not server administration, I've been sufficiently well educated to configure and launch web apps with Debian a few times already.
As of late, however, I've been looking forward to expand my knowledge and grasp a deeper understanding of Unix in general, how everything works beneath the nicer high-level tools. So when I ask what distro is the best, please note that I'm not asking what distro is the easiest. I want to know, from a engineering point of view, why so many spectacularly smart people choose, for instance, Gentoo and FreeBSD. Can any of the few who know why they want these systems name a few of the advantages they have over Debian, for instance?
Its derivative, Ubuntu, is pretty good too. I use it, and really enjoy the fact that I run the same thing on my server and on my laptop. Wikipedia is switching to Ubuntu, apparently.
I'd recommend Ubuntu too - especially if most of your work is "vanilla" - e.g. Apache, MySQL, etc. The assistance around this is both high quality and accessible (not that it's not with Debian... I just find it better for Ubuntu).
Once you get running you might consider a switch to Debian, but I've never reached that point.
the last poll on this showed that the vast majority (90%+) of posters here are running Debian on their servers.
That would be a self-selecting poll and highly unscientific. I don't remember it. But, I'm confident it was skewed, even amongst the technorati found here at HN. CentOS is dramatically more popular for web hosting (ask any web host you know that offers both).
I think Debian is awesome, and you definitely won't go wrong choosing it for a server (or any system, really). I'm not at all arguing that Debian is a poor choice--it's a great choice. I just wanted to point out that 90% seems ridiculously high for Debian market share on web servers.
You should give us reasons why you have narrowed it down to those 3 and what you level of experience is.
I personally use Gentoo or FreeBSD whenever I have a choice. A significant reason for that is years of experience tuning these for my needs; it has become second nature.
For some environments I have used CentOS with good success, but I still end up modifying and trimming things so it feels more like Gentoo. In the end, you would likely be happy with CentOS, Debian, or Ubuntu as far as binary distros go.
I would choose whichever one you are most comfortable with, or the one that you want to learn. If you don't know any, just choose one that sounds the most interesting. This is one of those "religious" choices between computer geeks and is of little consequence in the long run. If you hack with other people, you'll get experience with several flavors anyway.
I personally use Debian, and highly recommend it. It's very easy to get started out-of-the-box and there are tons of tutorials on how to get your typical LAMP setup or Rails or anything else set up. Ubuntu's also great, and probably has an even more supportive community than Debian.
I'd say pick the distribution you know best, with a few provisos:
- It's one of the major distributions
- It's not Gentoo
I used to use SuSE on servers because I knew SuSE best, and before that Redhat for the same reason. Now I use Ubuntu. All have been fine. Worst case is getting something on a server that you don't know how to configure.
SuSE makes me cry. I mean, it's a great desktop distro... but I'm not sure how stable it is for servers. I could be completely wrong, but I'm completely biased from the dep-hell I had to go through with SuSE a while ago.
I worked in the SAP LinuxLab for four years where our two supported distros were Redhat (RHEL) and SuSE (SLES). Both are fine for servers.
Usually when people have problems with SuSE's package manager it's because they added non-standard repositories to YaST and the packages are broken in strange and mischievous ways. That's not to say that I've not seen stuff broken with SuSE, but not with particularly higher frequency than other distros.
Debian also has the advantage that you can run Ubuntu on your desktop and use the same package manager as the server. It's a huge advantage to have the same deployment & configuration procedures on your dev environment as in production.
I've used Centos a lot with no major difficulties. I've also had great luck with SuSE, where I'm a big fan of AppArmor to help prevent zombification (assuming you're running any standard/open source web apps).
let's look at requirements first:
solid, stable, secure, KISS, small, best of breed applications instead of trying to include everything, vanilla kernel, not a patched one!
Use FreeBSD. It is based on Unix and powers the internet. In fact, its slogan is "The Power to Serve". It's unified, current, faster than Linux in some areas, excellent man pages and other documentation that's easy to find and can do anything Linux can do.
I have a great deal of experience with the older versions of Fedora (the consumer Redhat distro) and slight experience with CentOS. I'm fairly certain that majority of Linux webservers out there are running some Redhat variant. I personally prefer the Debian/Ubuntu approach and stick with that.
Off the top of my head I know that all of the Mediatemple servers are running CentOS, and Digg runs Debian.
I think like SwellJoe said, if you're thinking about FreeBSD for the sake of going with FreeBSD and not really knowing why, then you don't want to be choosing FreeBSD. The same goes for Gentoo. Gentoo is a great distribution, but it's not for the mainstream plug-n-play let's-build-a-webapp! type of person. Gentoo, although featuring a pretty nice 'emerge' package management system, is going to require more work from you in the beginning to get it to a desirable state.
I guess there is no real answer to your question, Linux is fantastic in general as a web application server. Any distro with the base binary packages for a typical LAMP stack or even a (Rails|Django|Etc...)/Nginx configuration will treat you very nice until you reach the proverbial land of scalability. Once you see problems with your app, fix them. Don't worry too much about that right now. Also keep in mind that 99.99% of the time your application is going to be "distribution agnostic" so later on down the line if you decide you'd rather go with CentOS over Debian, or a BSD variant over Linux, you won't have any issue with that.
I'd focus more on writing a solid application and getting a solid product out the door on any of the distributions that were mentioned by the wise hackers here on HN, learn what you like as you go. As you learn more about the way Linux works and you configure things to suit your needs, the various unique characteristics of each will become more clear, and your "distro of choice" will become more evident.
Bottom Line: Go with CentOS or Ubuntu, and worry about these things as they become worries.