The one that got me was spam filtering. Google does a pretty good job at that, and I was really drowning in it.
I think they deserve a lot of credit for creating a product that both my mother (definitely not a 'power user') and I (previous email client: Emacs' Gnus) can both be reasonably productive with.
It's still a service which you have to maintain and there's constant config & upgrading required since some spammers are smart enough to test their messages with it first.
I've run SA professionally for modest (low hundreds) of users. It works reasonably well but it's a job and I'm not paid to do it personally.
One could always mooch off of Gmail's spam filtering... Use SpamAssassin in conjunction with a gmail account that is only used as an attempt to attract as much spam as possible. Then just download the spam folder over IMAP and use it to teach SpamAssassin...
The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system.
One example is when the Hotmail team first enabled keyword filtering. When the spammers found out experimentally, they began injecting HTML comment tags into high-weighted words like "free", breaking the model. When the Hotmail team took steps to combat this problem, the amount of spam that employed this technique dropped from 5% to close to 0%[1] in a matter of days.
Spam detection is complicated and hard.
[1] Hulten, G., Penta, A., Seshadrinathan, G., and Mishra, M. Trends in
spam products and methods.
"The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system."
Finding holes on sites like gmail and hotmail is actually much easier than on private sites, because the spammers no doubt have accounts on gmail and hotmail and can test what gets through and what fails and tweak their algorithms until their spam gets through.
Spammers don't have that luxury on private servers, so they have to spam blindly. So in this respect private servers have an advantage over gmail and hotmail.
On the other hand (as has been pointed out many times in this thread), gmail and hotmail have the advantage of advanced spam detection algorithms and virtually instantaneous feedback from millions of users.
That's a perfect example of how spam detection is harder for a centralized server operator like Hotmail or Gmail than for somebody who's running their own server: if I were to add a delete-HTML-comment-tag preprocessing phase to my own copy of SpamAssassin, spammers would have a difficult time detecting that, and so would not be able to adapt. (Because none of the dozen or so people who can get mail on the server are spammers.)
SpamAssassin takes care of this for you by automatically downloading new filtering rules. You host the spam filtering software, but the actual rules are out-sourced by default. It even includes configuration for multiple DNSBLs, DNSWLs and RHSBLs by default. You can add your own local rules if absolutely necessary too.
I'm not saying it's perfect. Obviously having a network effect allows you to do things like flag messages that are exact duplicates over multiple sent-from and sent-to addresses.
I've seen this happen where 1 or 2 messages hit my inbox, but the next 20 or so are in my spam folder.
Gmail's spam filters are so good because they have a network effect going though, I doubt one spam folder would achieve much, you need a much bigger sample of all the spam than one account could ever attract
Pretty sure you are making some unwarranted assumptions here – other commenters mention that SpamAssassin can download updates to itself regularly, so you aren’t starting from scratch with a completely untrained Bayesian algorithm.
I use SpamAssassin server side, and Thunderbird's own filter polishes off the rest. I see perhaps 1 spam mail per week (from the literally tens of thousands that hit my server).
I think they deserve a lot of credit for creating a product that both my mother (definitely not a 'power user') and I (previous email client: Emacs' Gnus) can both be reasonably productive with.