
Your own Debian Mail Server (part II): how to prove you are not a spammer - tastalian
https://scaron.info/blog/debian-mail-spf-dkim.html
======
jfaucett
My number one product I wish existed is this: A complete email server package
that is easy to configure, setup, manage, is secure, and is accepted by other
email service providers (gmail, yahoo, etc) out of the box. And with easy I
mean as easy as apt-get install or just downloading a binary.

For anyone that has configured email servers, you know it is a headache, this
tutorial makes it look easy but its only adressing a a tiny portion of the
problem (albiet a important one) - email spoofing. (EDIT: it does mention spam
assassin at the end so there's a little bit of info about spam filtering)

~~~
Nux
The tricky bit here is "accepted by other email service providers". This
depends a lot on the IP you are using, on the reverse DNS, DKIM/SPF settings,
your ISP and "neighbours" reputations, RBL listings etc.

It's not just a case of distributing postfix and a nice UI on top. That's what
makes email difficult nowadays.

~~~
lewisl9029
I tried doing this a long time ago, and eventually gave up exactly because of
this. I self host a lot of things, but self hosted email has always taken up a
disproportionate amount of my time. At some point it's just not worth the
headache any more.

I ended up ditching the self hosting route and went with an Exchange Online
hosted email subscription at $5 per month and never looked back.

[https://products.office.com/en-ca/exchange/exchange-
online](https://products.office.com/en-ca/exchange/exchange-online)

~~~
Nux
Well, it's not THAT hard. Find a good host with clean IPs, check the IP
against RBLs before embarking on running your own email, if it's listed demand
a clean IP instead.

It's not the easiest thing in the world, but it's far from brain surgery.

------
dfvgskdfjghs
To me, the main obstacle is not the software (I can install a server in VM),
it's the implication that I have to rent a VPS, or buy a DNS record, or
subscribe to a different kind of ("business") Internet plan, when I already
have packets flowing, just so that the email giants and everyone else believe
I'm a legitimate participant in humanity. I feel stymied by email giants and
ISPs that seem to collaborate to prevent me from doing something that even I
agree is simple, to the point that I don't bother. That's 100% a social
problem.

I think the desire to run one's own email server today mostly reflects a
longing to re-discover the highly-accessible anarchy of the early Internet.
Unfortunately, if that is ever to be found, it likely won't be in the form of
complying with the highly-burdensome mostly- _social_ requirements of our
modern, well-centralized, email system. More likely, it will come from
painting over it.

~~~
spotman
It's not just corporations that would not think your legitimate but everyone
really.

These sort of unspoken rules you mention are very largely due to combat spam.
If anyone with port 25 open was trusted the amount of spam would be
intolerable.

Seems sort of silly to want to use email without taking these steps to make it
official as possible.

This is not some manipulative plot by google to get you to spend an extra 60
dollars a year, sorry.

~~~
mjn
Yes, treating dialup sources as likely spam sources has been done since long
before Google became a significant player in email. In earlier years the main
approach was to try to convince ISPs to filter outgoing port 25 by default on
their dialup IP ranges. Later, people started compiling lists of dialup IP
ranges (later expanded to DSL/cable/etc.) to block them at the recipient side,
since there were too many ISPs who weren't filtering. Recipients disliked
email from dialup IPs because ISPs seemed unwilling or unable to police their
customers and respond to individual abuse reports, and so little legitimate
email originated there anyway that it was easier to just cut them off.

I don't think the bigger end-user providers have been very involved in
developing those kinds of policies. The NANAE crowd was/is mostly
administrators of smaller and university servers, not Yahoo/AOL/Hotmail/Gmail
administrators.

------
jvehent
Been doing it for 10 years, and it's still fun :)
[https://jve.linuxwall.info/blog/index.php?post/2015/03/11/10...](https://jve.linuxwall.info/blog/index.php?post/2015/03/11/10-years-
of-self-hosting-Linuxwall.info)

------
devereaux
I have a mail server hosted at Linode on:

\- a clean IP, not on RBL, not blacklisted elsewhere to the best of my
knowledge (outlook.com tells you when you IP is bad)

\- also accessible on IPv6

\- both IPs having a proper rDNS on my domain

\- supporting SSL on port 487 and 565, with a certificate from a known
authority

\- with DKIM and SPF both passing according to gmail

Yet it ends up in gmail spam folder. And I'm only sending email to myself and
2 other persons, so it's not even mass mailing.

I think there are other factors at play.

~~~
sn
Google wants you to do special things for them. Have you gone to
[https://postmaster.google.com](https://postmaster.google.com) and set up your
domain there?

~~~
pflanze
This was first mentioned on HN here:
[https://news.ycombinator.com/item?id=9905767](https://news.ycombinator.com/item?id=9905767)

(I'm hoping I won't have to partake in this.)

~~~
sn
Yes, and even if the tools are not available perhaps the domain verification
would help.

~~~
pflanze
I really would have hoped that Google would try to live without asking people
to link domains to Google accounts. At least that's what I understand they are
doing here. If so, this is a step towards their interest, not ours, and their
slogan "don't be evil" pales more every day. Yes, of course some newer
competitors of theirs are leading this course by attempting to win over people
to their own closed messaging worlds, and users at large don't care, so all of
this is sad. (The language on their blog post about this (linked from the
mentioned post) is laden with a disappointing amount of weasel words, too.)

So what I'm saying is, no, I don't want to link my domains to my Google
account just so that I can send mails to them. And I'm going to hold out
hoping that their systems are collecting enough trust in my domains and/or IP
addresses in other ways so that it won't be necessary.

~~~
sn
Depending on who it is, maybe you can give the people you want to email
mailboxes on your own domain?

------
kierkegaard9
Avoiding spam filters is unfortunately not as easy as setting up rDNS, SPF,
DKIM and DMARC. Reputation is also becoming a big issue:
[http://liminality.xyz/the-hostile-email-
landscape/](http://liminality.xyz/the-hostile-email-landscape/)

------
fensipens
Again: SPF, DKIM and DMARC are no indicators of spamminess of a source. These
systems have a completely different purpose.

~~~
dchest
You have to configure them in order for your mail to be successfully delivered
and not put into Spam folder on major email providers.

~~~
wampus
I've run mail servers for decades without configuring them and have never had
issues. Reputation is probably the most important (note that my domains and
even some of my servers were online before these technologies existed) and
it's extremely important to get your DNS right, especially Forward-confirmed
reverse DNS (FCrDNS). Strictly enforce authentication on submission port 587
and segregate user submissions from application generated submissions so you
can tweak each configuration appropriately. Keep in mind that marking messages
as spam involves a complex chain of weighting, so if a minor adjustment gets
your messages accepted, you could still be straddling a line and would benefit
from fixing the basics. And never launch a server on an IP without first
checking it against blacklists (demand a new one if it's listed anywhere).

~~~
dchest
Reputation is everything, but when you need to setup a new server on a new
blacklist-checked IP for (non-spammy) mass mailing, without SPF and DKIM your
emails will most likely go to the Spam folder, in 2015.

Of course, those things are not guaranteeing delivery, but they play an
important role.

------
joering2
Slight OT/marketing, but I'm constantly having deliverability issues with
Yahoo and spent weeks figuring out whats wrong, when spam-tester shows score
10/10, SenderScore 97, and yahoo keeps automatically marking my messages as
spam, even when many times I contact client via phone and they swear they
never clicked "mark as spam", which I have no reason no to believe.

The spam complaint rate keeps being broken (around 0.3%) because of yahoo, and
utilizing Sendgrid as an EPS, I'm afraid of losing ability to send.

At this point I would love to hear an opinion of a e-mail deliveribility
expert/veteran, or someone who can help me get off the cloud and host own
email server that will be well-configured and maintained. I'm aware this
service might come with a hefty $ bill. Please contact me via my email in my
profile.

</shameless plug for help>

------
exratione
A different and less comprehensive recipe to compare with for Ubuntu:

[https://www.exratione.com/2014/07/setting-up-spf-and-dkim-
fo...](https://www.exratione.com/2014/07/setting-up-spf-and-dkim-for-an-
ubuntu-1404-mail-server/)

------
efesak
If you like Docker you can try Poste.io
[https://hub.docker.com/r/analogic/poste.io/](https://hub.docker.com/r/analogic/poste.io/)

------
INTPenis
Is anyone else peeved at the fact that SpamAssassin is still the de-facto
standard for spam filtering?

Works pretty well for post-queue but as soon as you try to pre-queue filter
anything you're in deep trouble.

Has anyone tried compiling SpamAssassin or writing a faster version of it in
another language? It was a long time since I played with perllibs but I seem
to remember being able to load perl code into c programs.

~~~
technion

        >Has anyone tried compiling SpamAssassin or writing a faster version of it in another language?
    

How fast to you want it? I've been pumping roughly 20,000 emails per day
through Spamassassin via maia[0], on a relatively moderate VPS. greylisting in
front of it handles a huge portion of the load.

[0]
[https://github.com/technion/maia_mailguard](https://github.com/technion/maia_mailguard)

Edit: Been a while since I checked. Actually yesterday's number was 98,000.

~~~
INTPenis
It's been made clear to me now that I need to look closer at greylisting in
postfix before mails are passed to the proxy_filter.

But if you're interested I'm talking around 62k mails a day. My experience
with this amount has led me to use 64G RAM on each MX but that's only to
handle a certain incident with very high load. Usually RAM usage is much lower
than 64G and there's plenty of IO cache available.

------
soneil
I'm not sure I'd put much stock in the linked mail-tester. It has a very
incomplete SPF implementation that'll heavily penalise perfectly valid
records.

(eg, my record is simply "mx -all". So if a host is listed as an MX for my
domain, it's also a valid sender. mail-tester doesn't appear to understand 'a'
or 'mx' entries, so fails this entirely.)

------
aroch
This is more or less a problem that you're never going to solve as a normal
person running their own mailserver. Hell, apparently Google has problems with
it. They mark emails I send from gmail.com to others _within_ my own GApps org
as spam because the headers don't match. What??

------
mpnordland
All this is moot if your server is in a blacklisted ip range. I found it out
the hard way. Not just any VPS provider will do, make sure you get one from a
company with a clean ip range.

------
alexkavon
Lots of people think it's a great idea to make packaged all-in-one mail
systems, but what you're really doing is making it easier for spammers to get
up and running.

------
hit8run
Been there done that. But in the end I went with
[http://mandrill.com/](http://mandrill.com/)

------
xbeta
Kinda off-topic, but I was wondering if anyone can recommend some setup for a
CLI based mail client that works with IMAP and Gmail?

I'm on a Mac now.

~~~
jlgaddis
I used _mutt_ on a Mac and was quite satisfied with it, although I'm kinda
"GUI-averse" anyways.

 _mutt_ itself is kinda slow if you actually use its internal IMAP and SMTP
features over the Internet (such as with Gmail). Instead, I used _offlineimap_
to pull down all of my mail to the Mac and pointed _mutt_ at the (Maildir)
directory where it was stored (which was incredibly fast, as you might guess).

In addition, instead of having _mutt_ send outbound mail directly to Gmail
itself, I fed messages to _msmtp_ (locally) and let that take care of sending
them off to Gmail in the background.

That setup isn't for everybody, of course, but I was very happy with it. In
particular, {processing/catching up on} all of the mailing lists I subscribe
to was much, much quicker.

------
illuminated
I'm using kolab.org for few years, great product overall.

------
plg
what about os x server?

~~~
josho
OS X server mail setup is easy. But, if you need to customize anything it
starts to get scary, i.e. If you don't change the settings the OS X way then
you risk an update blowing away your customizations. The OS X way is a
minimally documented serveradmin tool from terminal.

Oh and OS X server doesn't support anything newer than tls1.0.

So for these reasons my next mail server will be something more mainstream.

~~~
plg
Yes---you're absolutely right. Thinking back, I have run into similar problems
with the os x server apache implementation, where (a) anything slightly non-
standard is not possible from Apple's GUI interface, but (b) editing config
files by hand works UNTIL an update wipes it all away, etc.

