
A Survival Guide for the Small Mail Server - Sami_Lehtinen
http://www.spamhaus.org/news/article/719/a-survival-guide-for-the-small-mail-server
======
Someone1234
I think it is absolutely hilarious (and infuriating) that in their opening
paragraph they forgot to list spam blacklists, like themselves, as being
perhaps the biggest hurdle for small companies and individuals running mail
servers.

I ran a mail server for almost five years, and 9/10 of the issues I had to
resolve were various anti-spam blacklists or individual hosts blocking our
emails and then having customers complaining because we never emailed them.

There was never any reason given for the blocks, and our logs didn't show any
outgoing spam. However all of these spam blacklists would accept a "donation"
if you didn't want to wait the 24-72 hours for them to unblock you.
Essentially it is a shakedown.

I've just given up self-hosting and I recommend others do the same. The issue
isn't the spam, it is how much time you'll waste fighting the anti-spam
blacklists and large email hosts. A job better left to e.g. SendGrid or SES.

PS - We sent tens of thousands of emails a day.

~~~
saurik
Yeah :/. I have tons of operational knowledge of email and have been running
my own email setup since 1998. My email servers were ever _only_ used for
email I typed manually by hand (due to being dumb I never even used shortcuts
to auto type repetitive responses to customer messages). Then, one day a
couple years ago, I realize no one @yahoo.com is getting my email anymore.
Talking to Yahoo, I get barely any response and certainly no explanation. I
can easily believe that my entire IP block was cut for being a hosting
provider.

I now pay some company to deliver my personal mail, which is just dumb, and
clearly isn't caused by them being better about spam (as I sent no spam): it
is caused by a militarized anti-spam-crazed group of people who have managed
to build a system that essentially requires an oligopoly for email to
function. A lot of these people don't even consider it a problem to use
"report as spam" as a ways to punish companies for things that aren't even
sending spam (which is something I only bring up as it demonstrates the
"militant" comment).

The reality is that an open internet where people can talk to each other
without going through third parties requires a world in which you, heaven
forbid, receive some spam. It is actually a similar construction as the one
for why we should have network neutrality: large internet carriers in the
world of email should, if you really believe in open access, not be allowed to
say "these people can't send mail because they are too small and not paying
someone". We all need to accept at least a little spam to guarantee everyone
can send mail.

If this means that you have to get better at dealing with spam, as some of the
heavy-handed techniques people have put in place for dealing with spam are no
longer possible, well: so be it; and if that means you need to turn off the
notifications for new email on your phone to keep from getting angry when you
receive some spam (which tends to come up as a reason to hate spam), I hope
people realize that they caused their own reason to be upset, and that getting
notifications for anything is probably unhealthy, at least for you, anyway.

Sadly, the large companies who have slowly come to "own email" have no
incentive to break this cycle, whether in action or in words, as the current
state of affairs is what is giving them their power: whether they build
solutions that can solve spam in a way that is both fair and federated, or
they just try to educate users about how to better deal with spam and decrease
the stress in their lives (including telling people to stop letting email
interrupt them), it will just harm them. This doesn't sound like something the
private sector can solve.

~~~
JoshTriplett
> A lot of these people don't even consider it a problem to use "report as
> spam" as a ways to punish companies for things that aren't even sending spam

I get a fair many spams that have "unsubscribe" links, or even that look like
newsletters (for companies or organizations of varying legitimacy). I'm always
pleased when they come through one of the major mailing list providers that
provides a separate "report abuse" link, with which I can report that no, I
don't just want to unsubscribe, I want to report that I never subscribed in
the first place so that the list itself gets terminated.

If it's possible for someone to be subscribed to your "newsletter" without
having explicitly consented to doing so (and in the process proving ownership
of the subscribing email address), you are sending spam.

~~~
kijin
> _If it 's possible for someone to be subscribed to your "newsletter" without
> having explicitly consented to doing so (and in the process proving
> ownership of the subscribing email address), you are sending spam._

A problem with this attitude is that most people have no idea whether or not
they consented. Perhaps they consented in 2003 but don't remember. Perhaps
they consented yesterday but want to revoke consent because they're too busy
right now to read your newsletter. There's a reason why Gmail silently
converts your spam report into an unsubscribe if the correct headers are set.
Too many PEBCAK false positives.

~~~
dredmorbius
It may be a problem, but that problem _is for the sender_.

If anyone indicates in any way, shape, or form, whatsoever, that they don't
want your crap, stop sending it to them.

Email _is_ all but useless these days. Postal mail's hardly any better.

~~~
kijin
No, the problem is figuring out whether or not to _punish_ the sender.

If it's obviously spam, the sender's IP needs to be blacklisted, his hosting
account terminated, etc. in order to protect other people.

If the recipient just changed his mind about whether he wants to receive a
newsletter _that he explicitly signed up for less than a week ago_ , there is
no need to punish the sender. The sender just needs to be notified that the
recipient unsubscribed.

I don't know about you, but if all the social networks, instant messengers,
and "we're gonna replace email" startups in the world went dark for 24 hours,
I probably wouldn't even notice. They're all but useless to me, after all. But
if my email went down for 24 hours, I'd definitely make a big fuss about it.
Ditto if someone makes the wrong decision about which senders to block.

~~~
DanBC
If the sender is sending mail that causes significant numbers of people to
flag as spam, even if those people signed up and confirmed just a week ago,
they need some reminder that their behaviour is not acceptable.

Very many senders stretch the boundaries of what they send.

------
exratione
Part of the problem talks about the solution.

The real problem for a small company mail server isn't setting up a
respectably secure operation: set up SPF and DKIM and you're good to go. The
problem is maintaining deliverability for anything more than very casual usage
of mail into a very complicated ecosystem. At any point in time a random
switch can flip on some heuristic somewhere and suddenly a large mail provider
will reject you, or you're on a blacklist that provides no explanation or
ability to remove your mails, and that isn't isolated: it will spread
throughout the ecosystem, and leave a stain that can linger for a long time.

I've run a few-thousand person newsletter mail list for more than ten years,
and fairly recently switched to SES because it's simply not possible to
guarantee deliverability even at that modest scale running it yourself from
your own servers. If you're not hooked into the ecosystem then you have to
fight ongoing battles that only a professional in the mail deliverability
business has time and connections to understand and effectively win.

Even then, you have to be very careful: don't mention sums of money (e.g.
censor all discussion of research funding), don't trigger dumb spamassassin
rules (e.g. sentences with lots of long words in a scientific paper), don't
talk about known trigger words (e.g. no scientific articles on the role of
growth hormone in mouse aging) and so forth.

It is crazy. The absolutely rational response to all of this is to outsource
mail delivery.

------
geoka9
I set up my own Postfix server about a year ago, because I wanted to send
newsletters to my users and have an "official" mail box for support requests
and such. I used a guide[1] to secure it against spammers. I don't use spam
assassin. In the year that followed I received exactly zero spam messages.
There have been plenty of attempts to send spam using my server, but the
postfix filters intercepted all of them.

To make sure that my messages are not treated as spam by the major email
providers, I checked that my VPS IP is not in any of the spam blacklists and
configured SPF and DKIM records. Gmail spam-flagged the messages for a while,
but after a couple of months it learned they're not spam. Surprisingly, other
email providers (yahoo, ms, aol) never batted an eye.

I'm quite happy with my own server.

[1] [https://honeypot.net/filtering-spam-with-
postfix/](https://honeypot.net/filtering-spam-with-postfix/)

~~~
sliverstorm
I got flagged as spam pretty reliably by GMail, but I eventually discovered it
was because my mail server was delivering mail to GMail over IPv6, and I had
never set up proper records for my IPv6 addresses.

GMail was the only one I had that problem with because few mailservers are
using IPv6.

~~~
mike-cardwell
GMail has been strict about reverse DNS on IPv6 since the beginning.

~~~
sliverstorm
Well, I know that _now_ :)

------
tristor
I have a work-in-progress guide for folks wanting to host their own email
servers which provides a step-by-step process for taking a Debian (or Ubuntu)
server and building out a secure install of Postfix and Dovecot to provide TLS
1.2/PFS enabled SMTP, IMAP, and webmail services along with competent spam
filtering with low false-positive rate and all the required DNS and other
settings to ensure your mail is accepted by GMail and the like.

I've never really posted it up, but I feel like if you've read this article
and are wanting more, this might help you. It's work-in-progress because I've
been sidetracked from finishing the Client piece and also creating a
Dockerfile that generates a group of self-contained images configured as
recommended, but all the steps for the server are complete and tested.

You can find it at
[http://securemail.tristor.ro/#!index.md](http://securemail.tristor.ro/#!index.md)
It's all written in Markdown and I accept pull requests at
[https://github.com/Tristor/securemail.tristor.ro](https://github.com/Tristor/securemail.tristor.ro)

Hope that anyone in the process of setting up a mail server finds this
helpful.

~~~
gog
Hi!

I am the author of [http://gogs.info/books/debian-
mail/chunked/](http://gogs.info/books/debian-mail/chunked/) and I am working
on something similar (I have switched to Dovecot and wrote a nice GUI for
managing users and domain) but in the end it will be a set of Ansible scripts.

The things I did different:

* Amavisd-new instead of dspam and OpenDKIM

* SQLGrey instead of Postgrey

* unbound instead of BIND as a caching DNS server

I have been thinking on adding PolicyD for rate limiting accounts but then I
get a lot of overlap with other services. How are you satisfied with postfwd?

~~~
tristor
Nice, I hadn't seen that book before but am glad to see there's some more
detailed documentation out there on this subject. Setting up an email server
can be pretty confusing even for someone who's experienced just because
there's so many moving parts and its very finicky.

I haven't really worked with Amavisd, SQLGrey, or PolicyD before. I've now
switched to using unbound instead of BIND myself, but this hasn't yet been
reflected in my instructions since I kind of stalled on updating them.

I like postfwd quite a bit for how I'm using it, which is to do hybrid
greylisting based on DNSBL weighting. It allows me to reduce latency for
inbound messages that are free and clear rather than greylisting everything. I
don't really do any rate limiting or anything like that with it though, so I
doubt I'm using enough of it to really form a clear opinion.

------
jgroszko
I spent a few weeks this past fall trying to configure spamassassin to be
useful, but I was getting up to 30 pieces of spam a day and it was getting
obnoxious. I configured my setup to use greylisting and it wound up being
significantly more effective. I get maybe one piece of junk a week.

[https://help.ubuntu.com/community/PostfixGreylisting](https://help.ubuntu.com/community/PostfixGreylisting)

~~~
blfr
I only greylist messages which get at least one point from SpamAssasin. That
way obvious ham is delivered immediately.

~~~
jlgaddis
I'd like to do that, but greylisting is typically done at the SMTP level --
before the messages are anywhere close to SpamAssassin. Can I ask how you're
accomplishing this?

~~~
dmoo
Have a look at assp, complex set-up but very powerful.

[http://sourceforge.net/projects/assp/](http://sourceforge.net/projects/assp/)

~~~
jlgaddis
Thanks, the web interface for that looks pretty nice but it'd take too much to
move our mail systems to that.

------
2bluesc
I'm currently using Fastmail and might be too lazy/busy to switch as I'm
currently a trial user. The Android + Web app is better then K9-Mail and
Roundcube IMHO.

If I did switch back to running my own show, I'd do Digital Ocean + Mailinabox
via Docker.

Mailinabox aims to do all the things right out of the box:

[https://mailinabox.email/](https://mailinabox.email/)

~~~
Karunamon
What do you think of Fastmail? I'm considering migrating from Gmail (Google
Apps) to them.

~~~
rando3826
Great. The thing I like is that you get maximum flexibility, and good web
interface. For example: I run a mail server which handles mail for some
subdomains, and the root domain mail I route through them. Every time I give
an email out to a company, I name it company-name@one-of-many-short-fastmail-
domains.com. I never have to worry about spam from those email address, as I
can filter and cycle those at will.

~~~
tem5050
I do something very similar, but instead of using the company name (or
something close to it) I use a long random string instead. It mainly prevents
someone from 'company hopping' (i.e. changing 'facebook' to 'google'). The
chance of that happening might be small, but I figure I may as well as it
isn't really any more effort

------
ForHackernews
I'll throw in a plug here for
[http://www.iredmail.org/](http://www.iredmail.org/) which is a great way to
set up and configure a personal mail server.

Edit: I just noticed iredmail quotes _this_ article on their front page.

------
kefka
I just set up my new VPS, which included getting
Apache/Wordpress/Postfix/Dovecot set up and secured, along with integration of
StartSSL free cert.

site: [https://crankylinuxuser.net](https://crankylinuxuser.net)

And I secured it down appropriately whilst also using multitude of external
scanners to verify what I thought I did. It was somewhat problematic, mainly
with the AUTH between dovecot and postfix.

And then I do a RBL search. Some idiot spammer did the dirty deed back in
September. 3 RBLs, all lesser known ones. So, I responded as they requested,
with a justification of why I should be removed. I did that today around noon.

I'm now on none of them. So the whole "shakedown" issue, at least for me,
seems not to be the case.

------
tobbyb
We[1] have a multi-domain mail server container with imap, pop, GUI admin,
webmail, and spam lists that is basically ready to go, based on LXC.

You can launch the container in seconds and should be the fastest and perhaps
easiest way to have a fully functioning mail server. There is a starter guide
[2] and even a [3] video to get you going.

However a mail server unlike most other apps is not only complex to install -
which we try to address - but also complex to run.

For most if not all users mail just can't fail and this needs a fair bit of
knowledge of dns, smtp, spam prevention, security. I think in all but the most
committed cases it can prove a bit overwhelming for the average user.

[1][http://www.flockport.com/containers](http://www.flockport.com/containers)
[2][http://www.flockport.com/using-the-flockport-
mailserver/](http://www.flockport.com/using-the-flockport-mailserver/)
[3][https://www.youtube.com/watch?v=ysUswy8rGwM](https://www.youtube.com/watch?v=ysUswy8rGwM)

------
jrapdx3
Like others have commented, I've been running a _small_ email service for
several years. The key point here is "small" meaning serving a few users for
mainly personal mail. It's a very low volume affair, no "mass mailings",
newletters or anything of that kind.

I'm using Exim only because it was the MTA I knew how to configure. It's a
simple setup, allows no relays, uses TLS when possible, a few bad actors are
RCPT blacklisted, that's about it. I use a webmail program I wrote to retrieve
and send mail. It's all quite basic.

Have had very little trouble with spam or or being blocked, which may be a
benefit of keeping such a low profile. All in all, I've enjoyed the freedom
and privacy of running my own email server. I suspect the secret of success is
keeping it small, personal and inconspicuous.

------
zaroth
I run a small server with a few dozen users, the only problem I've had is
verizon.net bounces all messages claiming spam from my IP, and steadfastly
refuses to whitelist me despite numerous attempts. Always just a form-letter
response, despite having checked all the boxes (SPF, DKIM, rDNS). Luckily I'm
happy to live without being able to email people on @verizon.net.

I'm just not willing to store all my personal and business communication with
a 3rd party. It goes against the entire purpose of the standard, and I figure
the day it becomes impossible to run my own server is the day I no longer use
email.

------
Spooky23
Guide for small server operators is to put a bullet in the server. Buy
O365/Google/etc.

I ran a smallish email system for a year and a largish email operation for
several years. It's a lot of work to do things right and the benefits are
really small. The security related issues presented are very niche -- if you
are concerned about data leakage for confidential discussions leaving your
datacenter perimeter, you probably need to look at alternate solutions anyway.

~~~
bsder
That's grand until ... gee ... Google is thinking about buying you and they
have _all your internal email_.

Yeah, no.

------
leni536
> The organization no longer has control of its own email security.

Well it's not entirely true. They could have a strict client side encryption
policy. However it could be even harder to manage than mail server.

