Weirdly, that's the reason I switched from OS X to Debian. OS X's hardware support is fine, but I do a bunch of hobby and open-source programming, wrestling with getting stuff to compile under OS X is just not my idea of a fun afternoon. MacPorts is a joke compared to the size and reliability of Debian's package archive, and Debian doesn't make you compile everything yourself, or leave multiple, uncleaned build-trees lying around for every package you install.
That's a good reason to prefer Linux. If OS X didn't have pretty stable and easy system upgrades and migrations I would miss Linux package managers. Copying the list of installed packages to a new machine and then running aptitude update is just awesome.
I have pretty moderate package manager needs so homebrew is good enough, and when homebrew doesn't have it I just install from source.