Setting up email servers, especially ensuring that they don't get blacklisted by other servers, is a huge pain and definitely not worth the trouble for an individual or a small team. If you're concerned about Google, there are lots of other providers out there, and whatever they're charging is probably a bargain relative to rolling your own.
Can you elaborate on the blacklisting part? I've had my mailserver for a little over a year now and never had any issues with my mail going to spam, but my uses for that mailserver only revolve around 2-3 email accounts.
Sorry, I was a little unclear - the setup is easy, it's the maintenance that's annoying. (I've run mail servers in one way or another for ~15 years. I use Google Apps for my personal email now but still maintain some servers that do about 10,000 emails a day.)
Even if you have DKM/SPF setup, someone can decide to start spoofing email from your domain to send spam, especially if your domain has been around a while and is relatively clean. Since you don't send a lot of real mail, all of a sudden 99% of the email marked as coming from your domain is spam.
It's not always even obvious that you've been blacklisted; mails may simply be significantly delayed (AOL does this, I guess so they can check for similar messages to other users), etc.
This is an issue for email providers as well, but they have people on staff paid to deal with it.
Even if you only have to deal with this once every 2-3 years, that's still generally going to be worse than simply paying someone $5 a month for a managed email service, not to mention to you have to deal with backups, etc.
You may be right on this ... spoofing is still an aggravation because of the bouncing. Most of the time I haven't been able to find out exactly _why_ a particular provider has an issue (not blacklists), only ask for them to change their mind.
AOL, as far as I'm concerned, is a lost cause. They will block on rDNS alone, and no amount of calling or emailing will convince them otherwise. Recently, Verizon has gotten pissy about domain names matching the sending server, without even checking SPF.
In my 10+ years of running an SMTP server for SOHO use off of first business DSL (where Verizon refused to change rDNS), then cable modem, AOL was the sole server I encountered rejecting on mismatching rDNS alone.
Managing black lists is definitely a pain in the ass. If you can do outgoing spamassassin it definitely helps. I also use MXToolbox's free plan to monitor my two outgoing mail servers for blacklist. Alternatively, you can use Nagios to monitor blacklists.
You've been lucky so far; I have been running my own personal SMTP server(s) for quite some time, and while it's usually setup and forget, it can be an all-nighter headache.
First and foremost, static IPs are a must. Second, make sure those static IPs haven't been misclassified as dynamic or dialup. Then get reverse DNS setup properly (some providers won't do this). Then setup SPF. On top of all this, make absolutely 100% sure you don't run an open proxy. Then keep an eye on blacklists and your logs, and be ready to call and deal with obstinate tech support for hours on end. DKIM, and having some way to reject emails at the envelope stage (to save bandwidth, avoid double bounces, and punish the guilty rather than bouncing to innocent parties being Joe-jobbed) are also good ideas.
It's not horrible, IMHO, but it's not necessarily a cake walk either. You might need to have an understanding employer so that you can take off for half a day to deal with things.
Can you make a recommendation? I want something in the middle of a cheap hosting like Dreamhost and Google/Yahoo/Rackspace.
It was difficult to find it. Dreamhost was great because I can create a lot of e-mail accounts for my company but in the last year there were a lot of outage that complicated my business. On the other spectrum Google/Yahoo/Rackspace could be great but they are expensive when there is a linear relationship between e-mail accounts and price, and we create new e-mail accounts all the time not only for employees but for testing specific pieces of software.
Honestly, I use Google Apps for Domains myself, behind a domain I own (so I can leave down the road if I want.) I don't think folks moving off Gmail to leave Google want to do that but it's worked well for me. It's been long enough that I can't remember what I used before that for my email.