Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Is it possible to run your own mail server for personal use?
648 points by jdmoreira on Aug 13, 2016 | hide | past | favorite | 299 comments
Recently I decided that I wanted to run my own MTA. Downloaded qmail, applied a couple of patches and it was done. The problem is making sure my mail is not marked as spam by the major MTAs out there, gmail and hotmail both mark my mails as spam. So far I've:

- made sure I'm not running an open relay (obviously)

- made sure a reverse lookup to my IP matches my host

- made sure my IP wasn't already in any spam IP blacklist

- added SPF

- added DKIM

- run a lot of tests at glockapps.com

It's still not enough. I don't know what else to do, is it possible to run your own MTA, for personal use in 2016? Who, here, is doing it successfully?




It's absolutely possible to run an email server in 2016 and I encourage anyone capable to do so!

Email is one of the bastions of the decentralised Internet and we should hang on to it.

Every day more and more people are moving to Gmail/Hotmail/Outlook and while I do understand the reasons, it also puts more and more power into the hands of these providers and the little guy (us) gets more screwed (like marked as junk by default by them :< )

Having said that, here's my check list for successfully delivering email:

- make sure your IP (IPv6) is clean and not listed in any RBL, use e.g. http://multirbl.valli.org/ to check

- make sure you have a correct reverse dns (ptr) entry for said IP and that ptr/hostname's A record is also valid

- make sure your MTA does not append to the message headers your client's IP (ie x-originating-ip), messages can be blocked based only on "dodgy" x-originating-ip (see eg https://major.io/2013/04/14/remove-sensitive-information-fro... )

- set up SSL properly in your MTA, there are so many providers giving away free certs nowadays

- SPF, DKIM, DMARC - set them up, properly, this site can come in handy for checking yourself https://www.mail-tester.com/

- do not share the IP of your email server with a web server running any sort of scripting engine - if it gets exploited in any way usually sending spam is what the abusers will do

- last but not least - and while I loved qmail and vpopmail - use Postfix or Exim, they are both more fit for 2016, more configurable and with much, much larger user bases and as such bigger community and documentation.

HTH


> Email is one of the bastions of the decentralised Internet and we should hang on to it.

This.

I hope someone will eventually create an "E-mail server in a box" package for Ubuntu LTS, so that more people can run their own E-mail. I'm not saying it has to be super-easy for everyone, just that it should avoid unnecessary chore work (like configuring postfix to always use TLS, or plugging postgrey into postfix).


As long as you know Docker, this is pretty much what you are asking: https://github.com/tomav/docker-mailserver. I use it in combination with the rainloop webmail client: https://github.com/jprjr/docker-rainloop.

Everything is configured with a single docker-compose, read about it here: https://news.ycombinator.com/item?id=11748036


What is it if anything that happens if my internet cuts out at my home, should I instead use some sort of hosting provider for my mail server?


My ISP has a nice service, called tertiary MX. It's a MX server to add at the lowest priority that will accept emails into a queue that attempts to delivery to the primary MX every 15 minutes for a month.


    > What is it if anything that happens if my internet cuts
    > out at my home,
You bounce any incoming emails in that period, and (obviously) can't send any.

    > should I instead use some sort of hosting provider
    > for my mail server?
I suppose that's a trade-off against how reliable you consider your ISP and electricity supply, and the volume or importance of email you'd be likely to receive in such a period.


Bounce? No. The servers trying to send mail to your server will retry for up to, typically, four or five days, before they bounce the mail back to the sender.

Mail was written to be resilient against network downtime.


That's why you soft bounce. It tells the sender that you can't receive the email _at this time_.


Pretty sure you can't do anything when you're not connected to the internet.


A connection timeout is a soft bounce.

https://tools.ietf.org/html/rfc5321#section-4.5.4.1

   The sender MUST delay retrying a particular destination after one
   attempt has failed.  In general, the retry interval SHOULD be at
   least 30 minutes; however, more sophisticated and variable strategies
   will be beneficial when the SMTP client can determine the reason for
   non-delivery.

   Retries continue until the message is transmitted or the sender gives
   up; the give-up time generally needs to be at least 4-5 days.  It MAY
   be appropriate to set a shorter maximum number of retries for non-
   delivery notifications and equivalent error messages than for
   standard messages.  The parameters to the retry algorithm MUST be
   configurable.


That's why sending mta's should resend. Think sendmail does this every few hours or so


Ha, I wrote that so cautiously, thinking "I'm sure this terminology's not right, I'll probably be called out on it"!

Thanks though, I learned something :)

Regardless of what we call it - the sender would get a copy back with a message not containing the word 'bounce' that says it failed to send, right?


No, no message will be sent until after 4-5 days have passed without connection.


It's usually a bad idea to run email from a home internet connection because it massively increases the chances of your mail being marked as spam. The hardest part of running a mail server is making sure your mail gets through spam filters, and the IP is critical to this.


I forgot to mention that I use the abovementioned setup on a dedicated server, while also setting up a secondary MX dns record for bouncing the mail in case of malfunctioning as sister comments suggested.


I've been doing a similar thing but with Poste.io. https://poste.io/

It has a decent web admin panel built-in to the box and also a copy of RoundCube Webmail. It'll even automatically set-up LetsEncrypt SSL for you.

You might want to set-up rainloop manually though because RoundCube isn't the best.


that's the difference between rainloop and roundcube?


There is something you can do to install that : https://mailinabox.email/


I've been using it for a year. Works great. One of the most thoughtfully designed and complete bits of software I've ever run into.

I'm especially delighted with it's self monitoring and the status change email messages it sends me.

That it includes webmail and gives me a nicely featured DNS server only makes it a touch more awesome.

I run it at Digital Ocean for ten bucks a month. I support family and friends. My email does not pass through google or microsoft servers and that makes me very happy.


DigitalOcean is just as susceptible to surveillance as Google or Microsoft.


If you aren't sending encrypted email, it makes no difference where your email is hosted.


I don't think the point is to avoid surveillance (although self-hosting may make it slightly more inconvenient to snoop). The point is to own your own data, not depend on Microsoft/Google's benevolence.


That attempt at the problem looks pretty nice. Thanks for sharing it.


Mail in a box works great... replaced my old sendmail setup - using an external DNS is a bit funky, but a great package.. support multiple domains easily, pop3/imap and web client. Recommend.


It prefers to use its own DNS server because email DNS is tough. However, it provides the information to do it. While it probably took me a couple of hours to figure it out, I now have a DNS template at my registrar that makes it nearly automatic. That said, where I don't need the performance of a hosted DNS system, I let mailinabox take care of it.


a recommendation for mailinabox, I've been using it for about 6 months, very nice


The problem is that setting up an MTA that can deliver to gmail etc can't e.g. Be done (typically) on a dynamic IP like you'd have at home and the scripts running would have to interact with your registrar and/or DNS provider to properly configure DNS (unless it also sets up a nameserver, which would make assumptions about how the domain is / could be used).

There are a lot of moving parts, and being an email admin requires some maintenance like being vigilant about not allowing spam. An automated script to set it up for people who wouldn't know how to run it would be doing those people a disservice.


The problem there is most of the hard work the OP mentions has nothing to do with the mail server itself, but instead it's around DNS, networking, certificates, and monitoring 3rd party services.

Some of this may be able to be automated, but that's the hard part; not installing and configuring a few software packages.


This is definitely an issue. The emails server is easy. The tedious part is the infrastructure setup.


The real problem is that the unencrypted nature of email makes most solutions non-starters. If end-to-end were the default, then you could imagine having a process whereby you ran a P2P service which kept your MX records pointed to a cluster of network members, all of whom agreed to act as your MX fallback if you went down and which collectively handled monitoring and distributing blacklist notifications.


You're implying that this is not advisable to do because any such fallback could snoop the data?

I'm sorry I'm not sure I quite understand what you say the problem is and how it relates to the unencrypted nature of email.


The "email server in a box" is called a Mac Mini with the Server package.

It's such an easy-to-use email system. Been working great on a home static-IP address for me for a couple years now.


Maybe when (if?) Apple releases a new one. Right now the hardware on the Mac Mini is so pathetically underpowered for the price, and Apple has shown such blatant disdain for both the Mini and the Pro that it almost feels like they don't want customers for those product lines.


How on Earth MacMini with 8GB of RAM and i7 CPU is underpowered for the use? It is underpowered in only one way - energy consumption is so low that there are even server farms built on MacMinis, and colocation services targeting this brave little server.


Not for the use, for the price.


You get a custom operating system that literally no one else offers. It is a bit hard to put a price on that, but to me, it's absolutely worth the money they ask for the mini. I would even risk saying it's very cheap for what it offers.


Like the others noted here - it is well worth the price, since it saves a lot of time for admin, is easy to use and provides extra value for money.


You pay extra for Macs so you don't have to spend as much time to set it up.

Your time is valuable, right?


Thank you for mentioning https://www.mail-tester.com/ - I wish I knew this site existed back when I was setting up my mail server. I struggled setting up SPF and DKIM, having a site that validates your config is very helpful.


I would suggest running OpenSMTPD over any of the alternatives, it's much easier to configure and should be/remain secure as it comes from the OpenBSD people (the audit from Qualys attests to that).


Yeah I'll have to disagree here. I beat my head against this horribly documented MTA just trying to get virtual users to work with a relay host. I gave up after three days of slitting my wrists trying to get it to work. It's horribly documented, getting help out of the obsd folks is like getting blood from a turnip, but if it works for you great! But I would never agree it's easier to configure or has better documentation than postfix.


I'm genuinely curious here since opensmtpd has been by far the easiest to configure MTA I've ever seen. What problem(s) did you have? Virtual users through a relay host is literally one simple line:

    accept for domain "example.com" virtual <users> relay via relay.example.com
I don't see how anyone could conclude that is more difficult than postfix.


And have you tried it? Because when I tried it it did not work as documented.


Yes, it works as expected. Again, what problem(s) did you have? Vague "its hard" replaced with an even more vague "it didn't work" comes off as trolling rather than a legit problem.


There was http://seclists.org/oss-sec/2015/q4/11

PS. maybe that's one of the vulnerabilities mentioned in the report (1). Anyway it's an argument for me to run Qmail/qpsmtpd instead.

(1) https://www.qualys.com/2015/10/02/opensmtpd-audit-report.txt


Yeah those issues were found in the Qualys audit. They have been fixed so the codebase should only be stronger now.

I just wish we had something that wasn't written in C.


Try running any such program through Softbound + CETS or SAFEcode (already in LLVM). They turn C programs memory-safe. Should knock out most of your risk immediately with an acceptable performance hit unless your volume is really, really high. Code-Pointer Integrity at least protects control flow with max of around 10% penalty. Given they're all alpha by few developers, they need more people using them on various software and doing error reports if they fail.


SPF, DKIM, DMARC could be dropped if you actually use aws ses to send your mails, you can actually forward the mails there. So it's pretty impossible that your messages will be in spam. The only thing you need to manage is incoming mail now and not being a open relay.


Hence defeating one of the main purposes of running your own mail server in the first place: decentralization.


Unfortunately SES servers seem to be regularly blacklisted. A quick google confirms. I've been using SES for 3-4 months now and had a few hard bounces with specific mention to the IP being banned.


The last spam that got through GMail's filter was from amazonses.com. It wasn't clear how to report it back to Amazon.


actually I never had blacklisting problems on aws ses. My guess would be that the big providers ignoring the blaklists for themself. I mean even Google sometimes gets on a blacklist or Microsoft on still no email happened to be in Spam filter.


And if you still want them, AWS supports enabling these for your outgoing emails.


Burn how do you ensure your mails aren't tossed in the spam box of the big ones?


Each of the points mentioned in the post you replied to contribute to the decision if big players mark your mails as spam or not.


It's also recommended to not have any "vacation" autoreply mails - you will get spam with blacklist's honeypot email addresses as apparent sender and end up on blacklists.


How realistic is it to be able to do all this from a Digital Ocean or other VPS, given that those IP blocks may have already been used by spammers or may be used periodically by spammers? I.e. it's trivial for them to set up VPSes on the fly.

I'd feel nervous about doing this at home. For one thing, my ISP is a residential cable internet provider, and not 100% guaranteed uptime as a result. I'd hate to miss mail or worry whether I did.


>- make sure your IP (IPv6) is clean and not listed in any RBL, use e.g. http://multirbl.valli.org/ to check


I recently ran into issues with the PTR on a mail server and did the lazy/unsustainable move of just disabling IPv6. Wish I had found this post before!


If you can't control your PTR on IPv6, in postfix I recommend using: "smtp_address_preference = ipv4", instead of fully disabling IPv6.

This way you can still receive email from IPv6 servers.

I run an smtp server from home with a static IPv4 address (for which the ISP lets me configure the PTR), but their IPv6 is "beta" and they don't yet support delegation, which is rather frustrating, but different topic.


Plus make sure that your whois is not registered behind proxy registrars. In other words use your real name and real email address.


> Email is one of the bastions of the decentralised Internet and we should hang on to it.

ITYM federated Internet, agreed though.


>use Postfix or Exim

Better yet, opensmtpd.


Any open source mail servers taht set all this up? Last I checked in 2011 there weren't any.


Mail-in-a-Box


I disagree with you. I consider email legacy technology because of its reliance upon DNS and static IP addresses. These two requirements make hosting email unnecessarily costly.


What replaces email?


For most of the people I interact with, and most of the usecases: Viber


i consider viber legacy technology because of its reliance on a centralized system and which will stop working as soon as vc money runs out


Moreover, Viber's adoption is highly concentrated in some specific countries so, depending on where your friends are based, it might not be a viable solution.


I uninstalled Viber when the last friend still using it started using WhatsApp.

Btw, everybody still has email and SMS like in the 90s. For me SMS are a fallback when IM won't work for a given person; email is my primary mean of communication for business. Furthermore I can archive and search email much more than IMs.


One little trick that I rarely see mentioned for working around the negative or neutral reputation your MTA's IP might have is that you can route your outgoing emails through another MTA that has a higher reputation. For example route them through smtp.gmail.com (or for other options see https://support.google.com/a/answer/176600?hl=en). It does not mean you have to use Gmail. It does not mean you have to change your MX records. It does not mean you have to use a @gmail.com address. None of that. Your recipients will not even notice you are routing through smtp.gmail.com (unless they inspect the detailed headers). All you need is a Google account and password to authenticate against smtp.gmail.com, and Google will happily route your email to wherever, to any external domains, etc.

Doing this makes you retain all the advantages of running your own MTA: none of your emails are hosted at a third party provider, no scanning of your emails to personalize ads, no government agency can knock at the door of an email provider and ask them for the content of your inbox, etc.

The only downside is that in theory Google can scan and block your outgoing emails (not incoming emails since these hit your MTA directly). But if you don't send spam, this should never happen.

Another option is to route your mail through your ISP's MTA. Yes ISPs usually offer SMTP relay service accessible only from their customer's IP addresses (eg. for Comcast it is "smtp.comcast.net" IIRC.) However the reputation factor of an ISP's MTA might be worse than Google's MTA.


Sending email this way is typically known as smart hosting. https://en.wikipedia.org/wiki/Smart_host. It's a pretty reliable way of sending out email if your IP has a less than perfect reputation. In this instance less than perfect could be something completely out of your hands.

Are you an EC2 user, well there is a black list for those IP's. Are you on a residential connection or a dynamic IP... there is a black list for that also. So you may have a perfectly configured MTA with a pristine IP and still have delivery issues. Using a smart host is perfect in these situations.

It's been mentioned before but part of running your own mail server is the process of learning and tweaking things to your liking. I remember the days when you could run a server on your own home connection on port 25 without any issues - but those days are long gone. A crucial part of every sys admins journey historically has been setting up and configuring a personal use mail server.

You can always run a mail server at home on a port other than 25 and have an upstream email service deliver it to your alternate port.

So if you'd rather not deal with filtering your own spam, or would like a excellent smart host with SPF and DKIM - let me know.

I'll provide commercial spam filtering with backup MX or smart hosting services to any HN user for 50% off from duocircle.com - just message me... running your own is sort of a right of passage.


I use MailGun for this, they have a very easy guide which checks that you've done everything right and they're free for a few thousand messages per month. It's been great for the few years I've been using them (though it's just for light personal use).


I didn't realize this had an actual term.

I've smart-hosted using postfix, direct for incoming and gmail/sendgrid/mandrill (before it was neutered by mail chimp) for outgoing.

It was fairly cost effective - generous free tier (upto 12k emails) for outgoing, and a small vps to host postfix (~$5/month).


I'd like to try this out. Not sure how to PM you, though…


This completely defeats the original purpose of this HN. Why would anyone invest effort in hosting an independent email server to have all its content routed clear-text through Google's servers/databases?


Because your mail will end up at Google anyway? Even if you don't use Gmail, all your friends do.


https://mako.cc/copyrighteous/google-has-most-of-my-email-be...


PGP solves that problem, and yours. Running your own MTA comes at a price. You need a domain, certificates, your ISP needs to not block port 25, possibility of abuse (so separate IP, server, domain, certificates), server which costs money needs to be maintained and draws power and requires space, and another responsibility to maintain. Disclaimer: It is possible containers mitigate some of the above problems. It is also a legacy protocol. Just run PGP on top of it and use any free provider out there. If you don't use that you can assume Google's oversight (the NSA) sees your mail.


I would like to agree with you (it sure is much easier than running a mail server), but isn't the problem that your friends also need to use PGP?


They are talking about MTA output. Mail input and archiving are still important factors.

I personally want to control my input addresses.

Hosting your own you can get funky wildcard postboxen.

Hosting your own you can archive all the mail without 3rd party approval.

I can see the advantage of the small amount of outbound being via someone else.


I have a VPS I use as a smart host.


There is no way that Google is not recording messages sent through smtp.gmail.com.


The only issue with this is Google's sending limit (which I believe is ~500 messages per day, perhaps it's even less through SMTP). That could be bad if you wish send an opt-in email newsletter, for example. Also, Google passes the X-originating-IP header on all mail sent through its SMTP servers (at least it did the last time I tried this), which can easily be a one way ticket to the spam box for all of your outgoing emails.


Huh..never thought of this. I run my own e-mail server on a Linode. I have constant trouble not being classified as spam. I did a whole post on it:

http://penguindreams.org/blog/how-google-and-microsoft-made-...

The sad reality is, I'm thinking of moving my e-mail back to someone else. It might be Fastmail, might be Amazon. Running my own e-mail server is a pain and I hate having to send people a message on Facebook or Reddit saying, "I sent you an e-mail. Check your spam folder."

If I route my e-mail through Gmail's SMTP server as a relay, that shouldn't affect my DKIM/SPF stuff right? I'd just have to change SPF/DMARC to say gmail is allowed to relay messages for me, correct?


I did a similar thing 2 years ago and chose Fastmail. Their web-app is best of breed ex google -- unfortunately, the honest truth is nothing is as good as gmail web + gmail apps + google cal.

Fastmail does have more issues -- lack of fit and finish, their web app is sometimes buggy, their spam detection is buggy, it's imap, the lack of gmail style conversations being the fundamental unit of work in the app shines through sometimes, the gmail app on android does not interact well with fastmail, etc. However, they have recently fixed my biggest complaints: charging 11 cents per sms message to do 2fa and their rules interface looking like it was written as a my-first-javascript project. So it's actively improving. I like it enough that I recently re-upped for another two years.

They do enjoy one big advantage over google besides privacy: tech support! Like a human, that reads and responds to issues submitted through their site! I've contacted them twice with questions about setting up my domain and one other thing and I got replies, from someone who knew what she/he was doing, within an hour each time!

Also, they will do a catch-all address (eg xoxo@xoxo.com) which receives email from every address not specifically defined. Thus you can implement the best anti-spam technique: every email you give out is domain@xoxo.com which forwards to you@x0x0.com so you can tell exactly who sold or leaked your email address. eg latimes@x0x0.com was sold all over the place.


I love fastmail. Really I do not like gmail style conversations so I prefer fastmail over gmail.


I too love the FastMail web interface. It rocks! On Android, unfortunately, FastMail comes up short. I have yet to find a satisfactory solution


I've had my mail servers at Linode for about 8 years now, no problems at all. If anything, I am more restricted on mail processing rules than anyone else.


Yes in theory all you need to do is update your SPF to whitelist Google's SMTP relay.


Silly question, but have you tried sending an email to someone running spam-assassin and check the headers of the received mail to see if there's anything obvious flagging you? For example, on Linode make sure that your reverse DNS for IPv6 are also set correctly.

I find it hard to believe that you get classified as spam for no good reason, unless perhaps that Linode / Digital Ocean are known for hosting so much garbage and people can be tempted to outright block them, but I doubt Gmail does that?


But doesn't this also mean that google will read it and use the info in targeted ads, analytics, etc. If so, it might make this approach less desirable if your goal is a decentralized comm channel not spied on by big guys.


Having a perfect smtpd that speaks TLS 1.2, has properly set up dkim, sfp and dmarc records, working reverse DNS, etc is sadly not enough these days if you use a commodity vps/VM host. IP block reputation matters as well. Sadly, some other customers in your same /24 have been less clueful than you within the recent memory of major SMTP operators(gmail, office365/Microsoft, etc) and your IP space probably had a bad reputation.

Reputation perception by opaque large SMTP operators will not show up in RBLs and other ways to check for blacklists. You cannot query your IP block's status unless you happen to personally know a senior sysadmin on their mail operations teams. They don't share this information because it would help spammers choose new "clean" places to spam from.

One solution is to Colo your own 1u system with an ISP that is known to have very stringent zero tolerance abuse policies. Typically not one that is a commodity hoster.


I don't understand. Why this over-reliance on IP based reputation and blacklist? Aren't Bayesian filters enough? Or maybe people don't know how "mark as junk" works?


This is at a level before 'mark as junk', where a mail provider will refuse to deliver your message at all - or potentially even refuse to accept it. The earlier the mail providers can detect/block spam the less resources they have to dedicate to storing and transmitting the spam.


Ah, so cost is the reason why the won't let the email through at all… All those people who mistakenly think email is free… (even gmail is not free: you pay for it by letting them spy on you and sending you targeted ads).


It's actually worse than that. Gmail, msn, yahoo, et al. are far from free. Anyone that wants to email all their customers that use those mail providers have to pay into whitelists and maintain UCE response policies with them. Those costs are passed on to you, the consumer.


Most people in fact know this and assume they are being tracked everywhere and they also assume that the service providers (Google, Facebook) can read everything.

The problem is not that alternatives are too expensive, but that there are no good alternatives. I would be glad to pay $5/month to Google to get an ad-free version.


There was a recent blog post about migrating from gmail to FastMail on here earlier this week.

In my personal experience, it's been excellent. I've but using it for about a year and a half now, receiving from custom domains and sending from their servers.

Their webapp is better than Google's and it's actually the main reason why I've stayed with them.

It's also cheaper than $5/mo.


I disagree that most people know this. I imagine most reasonably technical people do, but I'd be amazed if most people on the street understand any of this.


Heh. You might be overestimating your own importance.


Huh? What makes you say that? This is not about me, this is about everyone getting spied on, and about anyone wanting to keep emails decentralised —which by the way has some import for everyone, not just computer geeks.


I don't buy into the whole 'everyone is getting spied on'. While the minority of individuals have valuable information, everything about the masses is completely predictable and mundane. There is nothing interesting about an average person, their lives are identical to others and there is no point in spying to get the information already known.


> I don't buy into the whole 'everyone is getting spied on'.

They are. At least the NSA is doing it at a global scale —thanks Mr Snowden for turning this "conspiracy theory" into an established fact.

> There is nothing interesting about an average person

Until there is. At that point, all they have to do is a nice search query into their ludicrously deep archive, so they can dig up whatever wasn't exactly average about that particular person. There has to be a trigger of course, but if you set one off, you'll be put on a watch list for higher scrutiny —at best.


Absolutely possible but its a battlefield on the Internet so you have to understand the players. Two things I haven't seen mentioned in all the excellent advice:

1) Does your ISP let you send email? Some ISP's will not allow any outbound traffic to port 25 from a non "business" port. They force their users to send their email to their server, and then they forward it on to the Internet.

They do this with nominally good intentions (it is easier to control spam generated from their networks), but they also are financially motivated to do so.

2) Don't try to send mail from a dynamic IP address, you should have (and would probably pay extra for) a fixed static IP address (V4 and V6).

Dynamic IPs have two problems, one they change and mail receivers don't like that. Two, they carry with them the abuses people who had the IP address before committed. So your email may get delivered one day, and then poof you renew the lease on your IP and get one that is on a black list somewhere.


> mail receivers don't like [Dynamic IPs]

That's an understatement. Reading Hotmail's policy, I saw a blanket ban on dynamic IP address. Your mail won't even attain the spam box of your recipient. It will bounce right away. Other big providers probably do the same.

In many cases to successfully sending a mail from a dynamic IP is flat out impossible.


I'm running small scale mail servers. It's not nearly as difficult as the common folklore makes you believe.

Make sure you don't send spam and your server is properly configured. If you are sending mails to people that don't want it then it is spam. "They silently agreed to get our newsletter because it was listed in our ToS on page 357" is not acceptable. No other excuse for sending spam is acceptable. Whenever you send any automated mail there must be an easy way to unsubscribe.

A few more tips:

* Check your mail server on http://multirbl.valli.org/ - if it's in any blacklist try to find out why (there are a few rogue blacklists, ignore them).

* Hotmail allows you to receive a report for every mail that a hotmail user thinks is spam. Use that. Act on it.

* Check your logs for messages that indicate that others think you're spamming.

* E-Mail forwarding is a tricky business these days. Avoid it.

I occasionally get dubious spam rejections, but they don't come from the large hosts. They usually come from some small ISP using a proprietary antispam solution that gives you no insight what's going on.

My suspicion would be that qmail is your problem. There are a great many details that a mail software has to get right, qmail often doesn't do what the email ecosystem expects.


I've been running my own MTA for about 15 years now, so it's definitely possible without spending the majority of one's waking hours to do so.

Even when I switched mail server IPs twice over the last few years I didn't run into the issues you ran into. A large part of it depends on where you run it - if you, say, run it on your home Internet connection that's usually an immediate strike against you because of the insane number of spammers using backdoor'd PCs to do exactly that.

The only time I ran an MTA out of my home was when I was on a commercial ISP with a fixed IP address, that seemed to be good enough for most services including gmail and hotmail.

These days I run my MTA on a VPS with a reputable hosting provider and don't seem to have that many issues with outgoing mails marked as spam.

SPF and DKIM are pretty much a must these days, so that's a good starting point, as are the rest of the precautions you already too. I assume you're using your own domain, how "old" is that domain? That might also have an impact giving how many phishers and spammers register odd domains and use them for a short amount of time. I've used the same domain since about 1999 so that could make a difference.

I use postfix instead of qmail, but I've used qmail in the past. Both work well and are easier to configure than sendmail or exim IMHO. On top of that I do run amavisd/spamassassin/clamav for the incoming emails as well.

One more thing I've got set up that I didn't see in your list is that I've got TLS set up with a non-self signed certificate for both incoming and outgoing email. I suspect that this also makes a difference even if the other email server won't request a client certificate (most, if not all, won't). Certainly shows up when I send an email over to gmail.

My biggest issues these days are more with incoming email:

- You'll never get to the level of spam filtering that, say, gmail offers. To me, that's OK

- I use greylisting to weed out a lot of the spam that would normally make it through spamassassin, but unfortunately that's when you find out how many people have misconfigured servers that bounce emails when they encounter temporary failures


You are me, almost. I run a small amount of mail out of an OVH box with few issues; before that I hosted from my house on a static IP address.

I don't actually greylist any more, though -- I've found that postfix's protocol violation detection (postscreen[0]) is just as good, if not better, and I like my mail to come through immediately :).

Amavis is set up to do pre-queue filtering, which means that mail is either rejected or delivered (possibly to the spam folder), my server shouldn't be generating its own bounces or dropping messages on the floor.

[0]: http://www.postfix.org/POSTSCREEN_README.html


OVH boxes aren't blacklisted everywhere?


I've been running my own mail server off of a residential connection since 1998, so... 18 years. I do pay for a static IP, and I switched ISPs away from Comcast (who had inherited me as a customer) when they abruptly started to filter inbound port 25 and claimed I had a malware infestation.

For years it was qmail, but when I wanted to use SMTP/SSL as much as possible, switching to Postfix was easier than maintaining all the qmail patches.

I switched over to Let's Encrypt certs several months ago, and those have been working out quite well for me.


FYI they don't filter inbound OR outbound port 25 if you're a Comcast Business customer; in fact I think the only ports they filter for business customers are a couple of ports only used for remote attacks on MS Windows boxes.


You know how Comcast gets the Worst Company in America award every year or two? I really don't feel any desire to go back to them.

As far as I can tell Comcast employs a number of very competent network engineers and an astounding number of horrendous customer service and executive managers.


> when they abruptly started to filter inbound port 25

Didn't you mean "outbound"?


No, I didn't.


I don't understand. Many providers block the outbound 25 port because of their distrust of the infected computers of their end losers. What could possibly be their rationale for blocking the inbound 25 port?


A lot of ISPs say you are not allowed to run 'servers' on your home internet, and hence block inbound low ports like SMTP, IMAP, http. It's just extortion.


Incompetence. At least that was the reason why my ISP at the time blocked both outbound and inbound port 25. It took a couple of weeks to convince them that they where wrong and open for inbound again.


> if you, say, run it on your home Internet connection that's usually an immediate strike against you

While true, some IP addresses were not previously used, either by home connections or by anyone, so you may be in luck there. There seems to be a correlation with my latest IP switch (three years ago or so) and less mail server trouble.

Most services are also smart enough to learn to trust IP addresses after a little while. Even when SPF and DKIM is ignored, this usually solves it in a few emails.


Some ISP turn over their Residential IP Blocks to Blacklist providers to add automatically to the blacklists.

ISP state in their TOS you are not to run "servers" from your home connection, as such they simply blacklist all Residential IP;s from the start, because if you are running a email server on a Residential Connection it is either a Botnet, or you are violating the TOS of your plan.


> ISP state in their TOS you are not to run "servers" from your home connection

I don't know what evil ISP that is but running your own server is 1) encouraged by my ISP since forever (XS4ALL, the Netherlands) and 2) these days it's illegal to tell people not to run servers because of net neutrality laws.


Comcast, ATT, you name it. Any of the Big US ISP's

Current Comcast AUP (http://www.xfinity.com/corporate/Customers/Policies/HighSpee...)

>>use or run dedicated, stand-alone equipment or servers from the Premises that provide network content or any other services to anyone outside of your Premises local area network (“Premises LAN”), also commonly referred to as public services or servers. Examples of prohibited equipment and servers include, but are not limited to, email, web hosting, file sharing, and proxy services and servers;


Being able to use a previously unused IP address is usually a good thing, pity that there are so few of those left.

ISPs definitely seem to have an impact as well, but some spamfilters at least used to automatically assign a negative score to anything originating from an IP block used by an ISP for domestic Internet connections.


Well, there's plenty of IPv6 around. My ISP gives out /48 blocks to all customers together with a single, static v4 address. The difference in number of public addresses is incredible.


Good point, although that's a different rant after I found out that my ISP has an IPV6 enabled backbone but won't allegedly roll out IPV6 to their clients until late this year.


Of course it is. I've been doing it since 2001 or so. It isn't as easy as it should be, but it isn't that hard, either.

I had problems with mail acceptance only once, when one of my ISPs got me an IP address that was either used by a spammer in the past, or was in the same subnet that the spammers used. Other than that, no problems over the past 15 years, and I switched providers and systems at least three times over that time.

I'd encourage everyone to go ahead and do it. It isn't very hard, cost on the order of several dollars/euros a month, and you finally own your E-mail. I find it appalling that most people either use company E-mail (it isn't yours, anyone can read it, and if you part ways with the company you have a problem) or Google Gmail (Google does read it, trains its algorithms on it, and targets advertising based on that).

Don't worry too much about DKIM. It is no longer a good signal anyway, most spam gets it right.

So, if you're capable of it, go ahead and run your own mail server. I wish more people did it, so that we could avoid the "big guys" restricting E-mail. If more individuals ran their own servers, we could democratize E-mail again: it wouldn't be that easy to just reject E-mail for no good reason.

For the reference, the software I use right now is: Ubuntu LTS, postfix, postgrey, amavis, dovecot. I rent a virtual server at Hetzner.de.


I will not repeat what everyone else has already said, but I can add one thing. You need to 'warm up' your IP address. You need to send a lot of non-spam email. MTAs will A/B test it: mark some as spam, mark others as not spam. Then they see if they get user spam reports or non-junk reports. They don't know you. The more you send (successfully, if everything is marked as spam by receivers it won't work), the more they start trusting you.

SparkPost allows the use of dedicated IPs and it has a warm up time. They tell everything about it in [0].

[0] https://support.sparkpost.com/customer/portal/articles/19722...


I'm doing it, also using Qmail. I've felt the same pains as you (even started to suspect that providers might detect mail was being sent by Qmail and scoring that lower (perhaps (only) spammers are using Qmail today?), but more probably my network block (Hetzner.de) is the biggest reason for my difficulties).

Here's what I've done on top of your list:

- backscatter prevention (using my own https://github.com/pflanze/better-qmail-remote)

- do the Google domain verification dance (postmaster tools, configuring their entry in the DNS); still didn't prevent mails ending up in spam, but who knows whether it might still have helped.

- started running mailing lists on it anyway, in spite of me knowing that mails end up in people's spam folders, and simply tell all new subscribers that mails first end up in their spam folders and that once they mark them as non-spam the problem goes away. This seems to be working (people haven't complained), and will over time hopefully give my server the reputation I need.

(PS. I'm also still using DJBDNS, with a config generator written in Scheme, look out for tinydns-scm on my Github)


Why would other MTAs have a buff against Qmail?

I'm using maradns for DNS but I respect djb's software a lot. I'm using his publicfile as an httpd.

I have already added my domains in google's postmaster tools. So far it hasn't helped much


It's not the MTAs that don't like Qmail, it would (my hypothesis) be the scoring systems run by the ISPs that would (hopefully automatically, i.e. based on statistics) give senders using some MTAs a lower score than others.

Small users generally don't use Qmail anymore, so if you're a small sender and use Qmail, then perhaps that's odd, and iff spammers (perhaps for Qmail's efficiency (1)) are using Qmail often, then...

(1) although you can attain higher efficiency with a custom MTA that's not safe, i.e. big spammers generally don't even retry for efficiency (that's why greylisting works).

A friend of mine runs several mail servers on OpenBSD, using OpenSMTPD, and says he's got almost zero problems and scratches his head on what I'm doing. So there's 3 hypothetical reasons for that for me:

- basically no spammers will be using OpenBSD (why should they?), - and hence also none be using OpenSMTPD, - and hence also none of them will be using a OpenBSD hoster (netblock!).

Relatively many spammers (perhaps not the biggest ones, but still spammy senders) will be using services like Hetzner, and will most likely be running Linux. Now if Gmail etc. do check the MTA you're running, and perhaps do OS fingerprinting, they'll "know" how to judge you.

Anyway, I don't have evidence that Qmail is a big reason, it seems likely the netblock is much more important (or at least that's what I'm inclined to think because the alternative would indeed be rather frustrating).


I think the parent's hypothesis was that MTAs would negatively weight any MTA that isn't the custom ones running on Gmail/Outlook/Yahoo servers.


Thanks. Well... that's just terrible.


One thing I learned from my spam filter's logs is that sending email in a slightly weird way is great way to marked as spam. I used to receive a few thousand emails a month that didn't have a Date header - all spam - which is strictly required by RFC2822 but sometimes allowed to be absent. amavis-new easily and correctly detected them.


Just to prevent misunderstandings, I don't think my Qmail setup does anything "weird" at all. Just that if you consciously want to score according to the MTA used, that you can. The only weird thing about a Qmail installation adapted to today (backscatter protection, DKIM, standard Qmail patches) is that not many people are using it.


Linode + Postfix successfully for years.

Reverse DNS very important and the SPF

Linode have excellent setup documentation ( https://www.linode.com/docs/email/postfix/email-with-postfix... )


I have been running this setup for a few years now and it's great. I used to have a problem with GMail rejecting a lot of forwarded emails as spam but I set up graylisting (http://postgrey.schweikert.ch/) and since then I haven't had a single email rejected. It probably also matters how long you've been running your server so I would expect (assuming you're not actually sending or forwarding spam) to see the number of rejected emails decrease over time.


I ran a setup like this for several years but rejected/spam foldered emails were always a pain, plus a well-appointed mail host uses a fair bit of memory. I use Google mail now. It's not very expensive, and instant unconditional delivery to other Google mail users is a pretty nice perk. The nicest thing is not having the worry in the back of your head that just maybe something has gone awry with your setup and you are losing mail right now.


Yes, it's absolutely possible. I wrote an extensive set of step-by-step instructions on how to deploy secure email services on top of Debian 7[0]. They still work but are no longer maintained because my current position is that services like Proton Mail [1] make running your own email services unnecessary. You're welcome to review and use them, and if anybody wants to update them to work under Debian 8, PRs are welcome on the Github repo [2]

[0] http://securemail.tristor.ro/#!index.md

[1] https://protonmail.com/

[2] https://github.com/Tristor/securemail.tristor.ro


I am running my own mailserver for several years now and it is quite possible. The problem is, that your IP is new to the other MTAs and you need a couple of months to build up the reputation. Services like the one from Microsoft have forms, where you can delist yourself from their blacklists. Even though you are not blacklisted, it helps for the reputation if you fill out the forms with the MTAs you have problems with. Nevertheless it is a constant work (not much..1 hour per week) to keep up the reputation. Just make sure you are not loosing the IP when it starts to work out ;)


Yes, I think this is a great part of the problem. My IP doesn't have any reputation. But how do I build one? I send a very small volume of emails, I'm just one person.


Send legitimate e-mails to people you know on those services. Ask them to mark those e-mails as trusted / not spam. Make sure they reply to at least a few.

I've been running my own e-mail server for years. I have DKIM, SPF, PTR, and everything else set up and my e-mail domains and IP addresses are many years old. Every so often, Gmail still decides to spam-bin me for a week or so.


It takes some time, but even with small amounts it will eventually work out. I started to send requests for whitelisting my ip to the technical support emails as well and some MTAs even have forms for that. Keep running it! The net was made for decentralised services. I think it will even be in 2020 worth it.


Same for me, I'm just one person. After a while you've mailed enough hotmail users that hotmail will accept it as non-spam. Google is trickier, they can get away with a lot more because (even when I explain why it happens) people assume I'm doing something wrong when it's google that configured their spamfilter.

Eventually though, gmail will typically accept email from home IP addresses. Google apps for business seems to be even trickier than that: they have an opt-in strict setting which blacklists (outright rejects) anything remotely suspicious. It'll tell you where you stand through delivery failures, but it also means you'll end up having to call whomever you're trying to reach.


If you go on the blacklist by default then it isn't really a "blacklist", is it? It's a whitelist, and you aren't on it.


Yes. I'm not on the blacklists but I'm also not on the whitelists. That's the problem. It's ridiculous!


You can do this, and I do this (although not for my personal email, currently, although I have in the past -- I do it for a club though and it works fine).

If you are not being blacklisted (check the common ones plus AOL, they run their own), and are using SPF and DKIM, you shouldn't be having problems with messages getting blocked. That's pretty unusual.

What could be happening is that you might be in an IP range that's residential; there are some operators who blackhole all messages originating from "residential" IPs, even if they are not specifically being blacklisted for bad conduct, and even if they have valid SPF/DKIM records. I think this is a pretty bullshitty thing to do, and completely out of the spirit of Postel's Law, to the point where I think anyone who configures a server this way ought to be forced into a lifetime of Windows XP helpdesk duty. But it's a thing that happens.

One solution that's worked for me is to get a cheap VPS and run my mailserver there. It's in an IP block that traces back to a big datacenter, and it seems to be much more acceptable to various overzealous spam filters than my home IP.


It's worth checking whether you're running IPv6. It does become very relevant. e.g., SPF records need to include it, it must also have a good rDNS, etc.

In particular, I know gmail hold IPv6 to higher standards. Some things (e.g. rDNS) that we traditionally treat as 'should', gmail will treat as 'must' over IPv6 - it's being treated as a chance to drop a lot of legacy leeway.

I do run my own MTA. It's not high-maintenance, at all. Understand the pitfalls, iron them out, and then stick with it to build your reputation. There is no magic bullet - the big providers won't tell us how they measure us - the best we can do is be well-behaved, stay well-behaved, and adopt modern standards (TLS, SPF, DKIM, etc) as they're thrown at us.

My best advice is to choose a reputable host. There's a lot of race-to-the-bottom in the web hosting market, and VPS are turning out no different. Keeping a clean house is good for your reputation - but so is living in a nice neighbourhood. It's well worth a couple of bucks extra to find such a neighbourhood.


> My best advice is to choose a reputable host.

This sucks big time. We should be able to send email from fucking home, not beg an external, bigger guy to either do it for us (one's ISP, google…), or lend us a machine so we can do it from there (Amazon, OVH…).

Email providing is so concentrated right now it is starting so show signs of network effects —just like more recent (anti)social networks. This is not right.


We should be able to send spam from home.

But if we can send email from random dynamic IP addresses, so can spammers.

And spammers do. I run an MTA (postfix) on a couple of Linodes and I get more hacky spam-wannabe traffic than I get actual mail.

Postfix is configured to kill it all, so I never see any of it. But the logs are spam carnage.

My domains are hardly famous, so if you're running a high profile site I can imagine the problem would be a lot worse.


What I mean is, the originating IP address should not be such an overwhelming criterion of spamminess. When I receive spam, my most important criterion for filtering it is pure content.

I have a hard time believing spam is such a huge problem we have to use simplistic filters such as IP bans to deal with them. My email address is written without obfuscation in the open web (my own website), and my email clients' filters (Thunderbird or Evolution, depending) are more than enough once I have trained them a little.

(My domain is probably even less famous, but I do receive cold, non-spammy email from time to time regarding my blog articles.)


I tried to think of network effects in email and could not think of any. To what do you refer?


I am referring to how the big players (Google with Gmail, Microsoft with Hotmail, and I suspect many others) tend to flag as spam, or even outright reject your email, as soon as the IP comes from a "little player", such as residential IP, or even virtual private hosting IPs.

Sending an email to a user of such networks has become difficult —at least, more difficult than it should be. Unless of course you come from one of those huge networks. And of course, the users themselves don't see any problem with this: they assume that if you can't send an email to them, the problem must be on your end —not, say, Gmail's spam filter.

On the other hand, if everyone had a little box that send email directly from home, nobody would ever dare to block them.


What is MTA?


"what he said". The Mail Transfer Agent (e.g. postfix, exim, etc) is the part of a mailserver that accepts mail, hands it off to a delivery agent if it's local, or hands it off to the recipient's MTA if it's not.

Basically it's the component that handles SMTP. Local delivery, pop3, imap, etc are almost always handed off to another component.

It's usually the difficult component because it's the service that other hosts will interact with. So if my imap service misbehaves, it annoys me until its fixed. If my MTA misbehaves, it's a lasting black mark against my host's reputation.


Message Transfer Agent. Compare with MUA, MDA, etc. Lots of WTF TLAs when trying to build SMTP over TCP/IP, etc. :-)


Send a mail from your address to mailtest@unlocktheinbox.com. They send back an extensive "lint for smtp" report within minutes. I found it indispensable for debugging a DMARC issue recently.


I tried this service right now and unfortunately, large chunks of their report require you to upgrade to a paid version to read it.

mail-tester is free: https://www.mail-tester.com/

(I'm not affiliated with either of these tools apart from using them.)


Was a mail admin for quite a number of years, here is some tips. Check that the ip address you are running your own mail server from is not black listed. Nor can the IP hosting the mail server be in a home ip range. This is because there is spam black lists that explicitly mark home user ip ranges as possibly spammy.

Check the repuation in the various anti spam blacklists out there * Check IP in multiple blacklist http://multirbl.valli.org/ * Check server IP on mxtoolbox http://mxtoolbox.com/whatismyip/ * If you are using home server IP please consider using a VPS with a good IP reputation. * Consider lookup up ip at Cisco senderbase reputation and check its score, make sure its consider good. https://www.senderbase.org * Lookup ip in Barracuda repuation http://www.barracudacentral.org/lookups (Cisco and Barracuda is because these are somewhat common antispam services at edges).


PSA: If you run your own mail server and use that email address for password resets, please use a reputable hosting provider and dns provider, and turn on 2FA.

Don't let that often overlooked weak point be the way every one of your accounts gets compromised. Once they have your email, they have everything that resets via that email domain.


Also make sure to use a different email address as your password reset email for your internet provider. In retrospect it sounds obvious but that totally bit me a while back.


Yes. Double yes. Came here to leave that exact comment. Here is a pretty bad war story: https://medium.com/@N/how-i-lost-my-50-000-twitter-username-...


Any recommendations for a reputable name server that has 2FA?


I used to just run bind (and then later tinydns) on the same machine that had the SMTP server, but finding secondaries got really hard (the last really good one got bought and shut down by dyndns). A couple years ago I switched my DNS to Amazon AWS's route53 which has been working great, and AWS has 2 factor authentication (at least for the web console).


1. Creating a good IP reputation takes time, get a static IP from your ISP and gradually increase number of sent out emails.

2. Having multiple IP addresses and throttling helps if you send bulk emails.

3. Check email headers of spammed emails, they usually contain valuable info about the reason of being detected (SPF/DKIM ,...)

4. Check the contents of your emails and find out spam score of the content.

5. You can look into commercial solutions like: www.own-mailbox.com

6. Hillary, if it's you, don't do it again!


It's possible but it's not worth it unless you have a lot of knowledge about it or a lot of time and enjoy it. That's why most small to medium companies outsource this to services like sendgrid, mailchimp, etc.

You can't do everything so you have to pick your battles ;)


> That's why most small to medium companies outsource this to services like...

Indeed big companies outsource mail servers too, that was very surprising at first but reduce [effective and hidden] costs significantly and with a good SLA they can receive an excellent service.


I've been running my own mail server on AWS for about 3 years now. Postfix, Dovecot, and a Rails app I wrote for webmail. At first I had 0 deliverability but over time it's improved to near 100%. Just setting up SPF, DKIM, not being on a blacklist, and building up a reputation of good mail seems to have worked wonders. I've been wanting to move to DigitalOcean but I don't yet know if there will be a significant hit on my IP reputation.

Postfix occasionally drops a legitimate incoming email due to a misconfigured sender, usually from a domain that doesn't resolve to anything, but I just log those in case I miss something important.


If you change IP addresses when moving to DigitalOcean it will reset your reputation and deliverability. Speaking from recent experience.


Can confirm IP address change will reset 97% of reputation. Domain reputation is almost negligible and content is also much less important.


Receiving email is easy. Sending it is much harder.

One of the things that is the most frustrating is that if you end up on a blacklist, or a large provider decides independently not to trust you, it's often completely silent when it blackholes all your outbound emails to that service.

I've just moved over to a hybrid of hosting my own MX servers for incoming email, and forwarding all my outgoing emails to an email-as-a-service provider for outgoing messages. Their trusted IPs usually help delivery, and they're actively paid by their users to have employees making sure that their IP ranges are whitelisted.


I might go down that route. What service are you using? Thanks


Personally, I use MailGun. They have a generous free tier (10,000 messages a month) that I've never exceeded for my personal use.


If you are having trouble with Google putting your emails into spam folders, email to yourself at Gmail. Then examine the original headers ("show original") and check the authentication header Google adds. It begins:

Authentication-Results: mx.google.com;

and details what it did and did not like about your message. For example, it let me know my mail server had suddenly started sending over IPv6 (actually, Google started accepting there and IPv6 had priority) and I only had SPF records for the IPv4 address. Google's authentication results are the friend of everyone with a personal mail server.


It's been many years since I've run my own mail server, but especially if you are running your mail server on a public cloud, you should make sure you aren't on any blacklists like Spamhaus: https://www.spamhaus.org/lookup/

EDIT: Looks like this is one of the things https://glockapps.com/ checks.


I already did check my IP. It's not in any known list. But thanks :)


Also check the IPs of your neighbours (in the same subnet or network block). I suspect the big ISPs also look at this, and on Hetzner at least I've found many neighbours with somewhat spammy IPs.


See this discussion: https://news.ycombinator.com/item?id=12107688


In particular this comment:

https://news.ycombinator.com/item?id=12109727

about some hard-to-find tools that help with Microsoft mail (Live.com / outlook.com / hotmail.com etc). There may have been some similar strange link for Gmail - i only have access to my phone right now, so I can't check my notes.

I believe that was the thing I was missing from my setup (and it was frustrating to debug as mail simply disappeared).

Between a (for now) cacert-certificate (which I think is treated as "self-signed" for most purposes) and SPF - both Google and MS appear to accept my mail. I've turned off ipv6 - if you want it, it might take some extra work.

I'm planning to move to opensmtp "when I have time" and maybe dbmail for the store - now I'm on exim and dovecot.

I get little spam - I do have greylisting set up. What I get is generally to emails leaked in the linked-in and Adobe attacks. I should change those addresses, and reject/black hole the old ones. But for now it's manageable - and is filtered out in my "misc" folder by simple address-basrd filtering (exim .forward file)

(I generally give each site a different mail, se eg my hn profile).


If you look at the aggregate of the comments here, I think you get the idea. Its possible, but you have to constantly work at keeping it going with reputation, whitelisting, etc. and you'll never get to 100% deliverability ingoing or outgoing. Probably more like 70%-80%.

The big guys whitelist each other.

There are (very expensive) services for the medium guys, such as [Return Path](https://returnpath.com/) , that help you keep good relationship with the various ISPs. With many ISP's they even have a very specific whitelisting deal where you literally pay to be whitelisted...I think thats mostly the second tier ones like AOL, Comcast, Yahoo. Gmail for example doesn't play ball with that - but they will whitelist you if you follow all the rules and have a good reputation, which someone like Return Path can help with.

Is the situation shady? Yes. But it rose out of a need for dealing with a real problem (spam), which you have to admit has gotten under control over the last 10 years.

Bottom lime, I would use a 3rd party service.


Yes, absolutely. I do this & have done for a decade or more.

But that decade probably helps - my mail server has kept the same static IP the entire time, so has a pristine spam reputation.

I added SPF a couple of years ago as otherwise Google was started to look askance at some of the emails sent from my server but I haven’t bothered with DKIM (apart from added a DKIM policy that says “I don’t do DKIM” that is). No problems so far.

Where are you hosting your mailserver? If it’s on a dynamic IP on your home internet, then you’re on a hiding to nothing. Static IP on a home internet might be OK, if you can fix your reverse DNS to be something sane rather than the more usual adsl123455.isp.net or something. Google generally hates reverse DNS entries that look like consumer internet connections.

If you’re hosting with AWS or another cloud provider, then I believe the only way to get an server on an IP address that doesn’t come with a terrible reputation for spam is to cycle through IP addresses until you find one that works - this is what the big mail delivery companies do I believe.


> is it possible to run your own MTA, for personal use in 2016? Who, here, is doing it successfully?

Hello, I run my own mail server, though admittedly with an attitude of "your spam filter thinks I'm spam? That filter is broken; have fun reading your spambox."

Company email addresses are actually never any trouble anyway, only personal ones (the free ones like yahoo, gmail and hotmail) are the ones where people have trouble with broken filters, so I don't think I'm missing out on anything. And even those filters usually learn (after a few emails to different accounts on the service) that my IP address is not to fear.

I add SPF records but don't sign with DKIM (too much trouble; I set this up years ago when I didn't have much experience yet).

The last time I had trouble sending email was with (of course) google apps. Some company, whose product we were required to use by school, had no privacy policy so I wanted to ask after it. Sending them an email, google's mail server outright refused my IP address to deliver a message (this is extremely rare, usually it goes into a spambox). Google is the only one that can get away with this, given the near monopoly, without people thinking it's an issue on google's side. In the end I just didn't send them an email. Also quite ironic that google, of all companies, is the one standing in the way of my email trying to ask after a company's privacy policy.

This was half a year ago. Before that I can't remember having issues with anyone or anything for another year or so. Given how much I use email and how little spam I receive (catch-all with a blacklist), owning a mail server is totally worth it. Also because I don't have to accept anyone else's privacy policy or consider how many people have access to my inbox (I host at home, not colocated nor VPS).


Regarding spam filtering, you actually can get as good if not better than Google's. I use bogofilter, the main issue though is that when you first start using it will be quite bad at filtering, you will have to continue to train it.

I actually use sendmail with bogofilter-milter.pl script (there is also bogom program with does similar thing) as opposed to usual way of running it with procmail, it essentially does the filtering at the moment it is being sent to you and rejects spam right on the spot. The advantage of it is that sender will get notification that spam was not delivered, also many spammers might drop you from their lists when they can't deliver spam.


Using another method (catch-all with a blacklist) I also get better filtering than gmail can offer. Much less sophisticated and a little more trouble, but much more effective.


I've been doing it for ages, but I do it a very lazy way. It's so lazy that I highly recommend it.

Before I explain what I do, I'll just mention that if you run the server that is the destination of your MX record, then you probably also want to run a back-up spooler at a remote site for when the connection to the primary server, or the server itself, is down. Down for security patching, for instance. Running a rock-solid mail service is kind of a pain, because it never fails at a convenient time.

So... I let my ISPs handle the high up-time requirement stuff, and let them be the mail spool as far as the outside world is concerned. Then I pop it down and requeue the mail on a machine that sits behind my firewall and isn't even on the front lines of the internet. I run an IMAP server on that. If it goes down, pfffft, the mail spools up a the ISP for a while and it gets popped down when my sever comes back up. It actually all works pretty well, but since I use my ISP's SMTP server for outgoing, all of my e-mail clients have a rather funky asymmetric set-up. The e-mail setup wizards just don't handle it. At. All. As long as you remember how to do old-school e-mail config settings, and can convince the new-fangled e-mail client to let you do a manual config, the asymmetric server is not much of an issue.

For remote access, I port-forward IMAP in my firewall.

So I should probably modernize this whole kit, but.... I think I mentioned above that I am lazy.


The problem is that your ISP has named your home connection with a DNS name that probably doesn't match with the domain name that you're trying to recieve and send mail for. There's a workaround by adding a vsp text record to your DNS. But that's no guarantee that all mayor mail providers will accept mail from your mailserver as unmarked (rather they'll end up as spam in your spam folder)


I run my own mailserver and have done since ~2012. I don't have DKIM or DMARC set up but I am using postfix and not qmail. I'm sorry I can't help you with your delivery problem - except that to say that if you didn't have eg rDNS set up to begin with gmail might have a negative cache of it. I haven't had problems with delivery of outgoing mail except to a couple of poorly administered exchange hosts run by recruitment agencies - which complain about my mail but confusingly do still deliver it.

Two real problems I have faced...I once (embarrassingly) created a unix user with test/test credentials for messing about and forgot that my postfix setup at the time reused unix credentials (ssh was locked down to only allow specific user to log in). I sent a few million spams a hour for a couple of weeks, getting my host into all the DNS blocklists. This took some time to fix (you have to apply to have your host removed from the blacklist). While I'm sure sending all that spam was annoying to many other people it didn't actually affect delivery for my mail...so it seems other administrators aren't using DNS blocklists?

Second, after a while I started to get a lot of spam. Maybe 10 per day. I tried various things to handle this, including setting up a proper bayesian spam filter (amavis-new) and using DNS blocklists myself. None of this worked for me. Greylisting however worked great.

So my suggestions to you: use defence in depth for mail as well as ssh. That means, fail2ban, different creds for both, unusual ports, user whitelists, high patchlevels (auto-patch and restart is great for a personal mail server) maybe client side TLS certs...etc. If you're relying on a single layer of defence eventually you'll make a mistake with it and then you're in trouble. I guess that really applies to anything you're trying to secure.


One idea I just had: inspect the headers of your emails as they are received in a gmail account. These will often contain diagnostic information that will help you debug any problems (with SPF/DKIM/etc)


That's a very good suggestion. I pretty much do that every time I make setup changes.


"gmail and hotmail both mark my mails as spam."

And they will continue to do so for as long as it takes for your mail server to earn a positive reputation. I recently had this problem with gmail after moving mail servers. there's really nobody that you can contact, gmail doesn't whitelist mail servers.

You're going to run into this problem with proofpoint, barracuda, postini, etc...


Using Postfix & Ubuntu on a Linode server, they have a very good how-to[0]. The main problem I have is filtering incoming spam on spam-assassin.

[0] https://www.linode.com/docs/email/postfix/email-with-postfix...


Using postgrey[0] fixed this problem for me, a few months back I started getting hammered with spam and now get virtually zero.

[0] https://wiki.centos.org/HowTos/postgrey


I'm running my own mail server for a couple of years now. I'm using postfix and dovecot. I have SPF and DKIM set up and I'm using spamassassin, roundcube.

Setting up a working mail server is one of the biggest challenges because so many components are involved these days and you need to make them play perfectly together. I'm against using out of the box VM images / installers because then you don't understand whats going on under the hood. Mail servers are these kinds of beasts I would suggest to understand as best as possible before letting it loose on the world.

Its like with security. There is no "Just press this button/Just install this software and you are secure" solution - There is no easy and convenient way to run your own mail server without getting really involved with it :)

Just saying this as a warning for everybody who has this thought. I've been through it and my mail set up is running for 5 years like a charm now. But it was a steep way to get there :)


Yes, it is possible. Here's a blog entry [1] I wrote some time ago how to set up your own email server that will accepted by major MTAs. I use my own mail server since 2004.

[1]http://draganmatic.com/init/default/show_page/1


You also need a DMARC record (https://dmarc.org) along with your DKIM and SPF records.

Be careful testing your configuration when sending emails to the large providers, you can inadvertently score negative marks against your own reputation, which is hard to recover from.


And setup reporting and forensic reporting. I have a domain that seems to be the default for a botnet, so my daily reports from GMail and Yahoo! always include at least a few IP addresses attempting to submit as admin@[domain].

The reporting sets my mind at ease that those big guys are blocking it and that the (low) legitimate volume of mail to those guys is reasonable.

It's also interesting to note that with DNSSEC, DKIM, SPF and DMARC, the pattern seems to be that some large Chinese mail providers drop DNS responses to try to overcome the "-all" token in the SPF record and "p=reject" token in DMARC. At least the reports show that the authentication (by SPF/DKIM) failed, so that makes me sleep a little better.

Par for the course, I guess. :P

*edit: grammar


I had been running mail servers for a few years but didn't bother with my own until about 12 years ago. Back then all I needed was an old box in a cupboard, static IP, mx record and not have an open relay. As you have discovered things have changed a lot.

I have seen some people do cool things with qmail but unless you have a long history with it I think you are better off with postfix. Qmail requires a lot of patches to catch up with the way things are done these days.

It sounds like you have done all the best practice things. A valid PTR record, valid ssl certificate, SPF, DKIM, DMARC. You generally can't host from home anymore because your IP will likely be blacklisted so grab a vps and check your IP is clean. You will want to add ridiculous rate limiting so you don't get sin binned when someone mails to a group. Adding your IP to dnswl probably does not hurt.

Even with you doing everything else right it won't get you instant acceptance somewhere like hotmail. You are going to have to establish a reputation over time with them. Get an email address at all the free services, send emails, make as not spam, check their help websites. View full source of received emails if they let you to see what headers their mail system attached to your messages.

And ofcourse protect your reputation. If you need to send legitimate bulk email use someone like mailchimp and let them deal with any damage.

And this is only the outgoing part. You still have to worry about incoming emails, submission, retrieval, filtering etc.

I do email setups for websites where I need to ensure spam filter free communication with clients and most business mail system I encounter don't seem to bother with half the stuff I do. They will deliver all their emails plain text with no spf or dkim. And they will do brain dead things like having fake blackhole mx records as some sort of homeopathic spam remedy. And they all seem to stay in business so perhaps we just overthink this stuff sometimes.


You can deploy end-to-end solution like zimbra (community version is open source and free). Has all necessary features (DMARC, SSL, clamav, spamassasin, and s-mime), really good webmail interface, some "cloud" file storage features (briefcase!) and more. As more and more people depend on big providers like google and microsoft with they mail we should advertise as much as possible, and convert as many as we can to self hosted mail solutions. Mail is last bastion of free communication. And should last "neutral" as long as is possible. Big providers are bulling smaller ones marking mail as spam/junk or blocking entire address spaces on "we thing that we should, and we do as we thing" policy.


Adding in my 2p's worth of advice here as well. The big thing I think is your IP. Even if it is 'static', if you are running the MTA from your house, your IP will be marked as residential, and probably also still 'dynamic' (just made sticky by your ISP to your Router/Modem).

The best way to give your outgoing mail 'authority' is to relay it through a smarthost. Some ISPs offer this - All my outgoing mail from the Exchange server in my Garage routes through my ISPs smarthost - it's the only way I can be sure that the big webmail hosts (hotmail/outlook, gmail, yahoo) actually get the mails. If I try to route direct, the mails get blocked.


I've been doing so for years. Having spent almost a decade in the ISP industry, I don't trust other providers for anything but transit.

I have a 50/10 connection from Comcast Business with five static IPs. One of those hosts what used to be a colo box with an ISP in Austin (I'd worked for them years ago, and had a services-for-colo agreement that lasted until an ownership change).

For about five years now I've had no problems sending or receiving mail; just keep a common-sense best practices configuration and do regular checks to make sure that you're not relaying/sending spam and aren't on any RBLs. "Nux"'s comment is a good list.


When I was in the same situation some time ago, I used https://www.mail-tester.com/ With the score around 7 or 8, Gmail didn't mark it as spam anymore.


Of course it is. I've been running my for ages; the current setup is postfix, dovecot, dspam, opendkim, opendmarc. The last IP change was ~1 years ago, the last domain change is ~1 month, no issues.

A long while ago, when I wasn't running the mail server in lxc, and it was a server which also hosted web frontends, I got it "hacked" once; a rouge perl script sent 10s of thousands of emails within an hours. Thankfully, this was in ~2007, so removing the node from blacklists wasn't impossible.

Anyway: add dmarc, and make sure you have TLS to send/receive. This latter is probably the most important bit.


We've used our own mail server to send all email reminders from www.cronote.com for the past five years. I followed the following tutorial step-by-step. The hardest thing was understanding how to setup the DNS records correctly:

http://flurdy.com/docs/postfix/

It'll take about three hours to get everything working right (and passing spam checks), but it's a great introduction to running your own mail server, and when you're done you can simply create an image of the machine to use in the future.


I just set up mail myself last week. Postfix on FreeBSD over at DigitalOcean. Did everything you did and was quite frustrated that my wife seemed not to be getting my email. The thing is, I sent a test email before adding SPF, and then one before adding DKIM, and Gmail figured out they were all part of the same "conversation" so because the first one seemed spammy the rest were penalized.

I made my own fresh Gmail account and messages go through fine. So I'd try that: make a fresh account from the one you've been testing with, and see if mail goes through.


I used to and stopped just because it was a pain in the butt to keep my emails from getting flagged as spam all over the place. Several years ago I switched to using Dreamhost which was great for a while but I'm running into the same issues again, an increasing number of popular mail hosts are flagging emails from me as spam. Likewise Dreamhost has pretty much given up providing reasonable spam blocking tools.

I'm now considering something like a google organization account, or whatever it is called, where it's really gmail but with my domain name


I've run my own mail server for >10 years. Getting it set up the first time is a bit of a chore but one it is running it requires nearly no attention. The hardest part is keeping your IP address off the spam black lists.

My setup:

Debian Linux + Postfix + Dovecot + a little greylist milter I wrote myself in Python. Happy to release the code if there's any interest.

I also have a script that automates the process of setting up a mail server but it's not quite ready for prime time. If anyone is interested in being an early adopter let me know.


Those of you who run your own email servers -- do any of you run it from a residential connection with dynamic ip address? Or do you pay extra for a static ip address/host using a VPS?


I've done it; personally, I pay the extra $5/month for static IP addresses.

Depending on your upstream provider, though, it ranges from difficult to impossible to get the reverse DNS for "your" static IP addresses set to something reasonable. Maybe for a business account they would do it...but for a home connection, forget it. Trouble tickets asking for reverse DNS updates will be ignored.


I used to run my own MTA, and then I get fed up staying ahead of spammers.

I chose postfix which has a lot of off the shelf support for blacklisting and such, but even so I found that I couldn't stay ahead of spammers and their (new at the time) techniques like reflection attacks.

Anyway, while it's fun to play with this, unless you want to spend time every week keeping it up to date, etc., I found it mostly a drain on other better things I could be doing.


I too ran my own mail server for years. I wasn't as rigorous as you in applying patches.. perhaps monthly or quarterly. At the end of the day though, it was the superiority of the Gmail client that won me back.


No, not really. It's a constant battle to get delisted from spam blacklists and your site keeps popping back up. Even most companies don't bother with it anymore.


I've given up on it for transactional email for any of my small-time SaaS services. Coaxing gmail into accepting my emails was a constant fight. Just when I thought I was good, they'd start blocking me again for some new obscure reason.

I remember fondly the days you could send an email to postmaster@domain.com and get a response from a real human being and at least attempt to have a rational conversation to resolve your mail delivery issues. I've tried that with google, yahoo, et al. and have never received a response.

I just use mailgun/mandrill/etc. these days and forget about it. I also pay $10 a year to a provider for my personal email address.


> Even most companies don't bother with it anymore.

This is a little bit like "correlation is not causation". Companies are moving a lot of stuff to the 'cloud', and email is just one of the services that goes with it. It's simply cost-saving to not have dedicated IT personnel. Even if hosting your own mail server is easy, you need someone who takes care of it every now and then (disk full, adding users, blacklist incident once a year, etc.).


Your company is amazingly lucky if it only gets a blacklist incident once a year, and if that incident is easily resolved.

The internet seems to have decided that if your mail doesn't come from one of the major commercial mail brokers, it's as good as spam. I ran several private mail servers for several years before giving up. I'm generally suspicious of moving to cloud/services, but outgoing email is the one thing that I can safely say 99% of companies would be better off moving to a service.


I did almost exactly what you describe and it works flawlessly, but I only send a small volume of mail (my own). Digital ocean droplet $10 a month, but it would run on a $5 one if you don't also have caldav/sftp/kerberos/vpn/etc there like me.

Fun Fact: Microsoft Exchange has no native support for DKIM and tons of business run that in house, often on 'business class' connections without correct reverse IP records.


> Digital ocean droplet $10 a month

I don't know whether OP uses a home connection, but that makes a big difference. It has some pros and cons:

A VPS (with its dedicated IP address):

- Usually has a good reputation, though you might have bad luck. Switch early on when you notice this. Doing it later just nullifies any trouble you had building its reputation.

- Is hosted somewhere in the cloud. (I used to hate the term "the cloud" but it's actually pretty apt and funny if you start seeing it as "they have no idea where their shit is and who has access to it, both physically and network traffic".)

Hosting at home:

- Might be trouble with your ISP if you don't live in a country with net neutrality and port 25 is closed.

- Gets you a lower reputation score by default, so it takes slightly longer to build.

- Gives you full access control over your email.


There is also a third option, set up a OpenVPN server and connect the home hosted SMTP server to the internet though it, gives you the best of both worlds I figure.


That might actually be a good idea! I wonder whether a hosting provider or ISP is more reliable in not snooping SMTP traffic, but assuming that is equal, it sounds like the best of both worlds indeed.


Yes, with the caveat of non-guaranteed deliverability (in other words: one day out of the blue gmail/hotmail/whoever will drop silently your emails)


Gmail blocks based on IP. If you're running off of a dynamic IP at home you're going to need a smarthost.

If you're running from a VPS you may need a smarthost.


Hillary Clinton is the expert here ¯\_(ツ)_/¯


You should think about security though. An interesting article you should read before deciding to use your own mail domain: https://medium.com/@N/how-i-lost-my-50-000-twitter-username-...


Make sure you aren't delivering over IPv6. I had all my IPv4 rules set up, and mail worked fine - except delivering to gmail. Turns out I was delivering over IPv6. I wound up shutting off my IPv6 interface.

As a good netzien I should work to behave on IPv6, but it was just too much of a pita for my tiny, single user server which works fine on IPv4


Ars did a series of articles on running your own mail server: http://arstechnica.com/information-technology/2014/02/how-to...


I have been running an Exim4 email server servising several domains for many years, mostly without issue. I do have SPF configured and it is running on a commercial VPS server.

For spam defence I use grey-listing based on DNSBL lookups, some standards-enforcement ACLs and then SpamAssasin via MailScanner on the messages that get through.


Been running it for over 15 years. No SPF. No dkim. Never had complaints, though threads like these have me worried.


I think 15 years of "good behaviour" is exactly what they're missing. gmails 12 years old. If you've had a well-behaved service for as long as they've been keeping track, that probably weighs in your favour much better than anything else.

That said, SPF is worth looking into - but it's more a "good neighbour" thing. Rather than having any direct benefit for you, it helps other hosts spot mails that claim to be from you. "Be the change you want to see" type deal.


You and me both. I think the risks are overplayed, but at the same time it's probably about time to join the future with SPF/DKIM/DMARC.


And IPv6 is that future. My resource is that when you start connecting on IPv6 many mail providers will consider SPF and proper reverse DNS mandatory.


For that, I've been looking into freeposte.io, so I can hop onto the container-train at the same time.


There is no excuse for not having SPF and DKIM signing set up in 2016/2017. Bet you don't do opportunist TLS 1.2 either?


The spam filter is a product. And if it doesn't work, like legitimate mail getting marked as spam, they will lose business. Just look at Yahoo where it's currently impossible to get white-listed, who are losing business because of this. (Yahoo mail was as big as Gmail is now about 15 or so years ago)


What exactly is happening to your outbound test messages? Is the recipient MTA accepting delivery but filtering? Are you seeing rejects from the MTA? If so what's the error? Try sending messages that are ordinary looking (not "Testing, testing...").

How long has the sender domain been registered?


What exactly is happening to your outbound test messages? Is the recipient MTA accepting delivery but filtering? Are you seeing rejects from the MTA? If so what's the error? Try sending messages that are ordinary looking (not "Testing, testing...").

How long has the sender domain been registered?


In addition to the other things; DNSSEC and then registering your domain for at least 3 years might help.


I followed Ars Technica's article years ago, and modulo a few minor alterations, their instructions seemed to have worked pretty well. I will note that I mostly receive, rather than send email, but I've had no problems I'm aware of.


Sure! I run several servers (and developing them, see https://poste.io). Watch DMARC reports and give it little time, most of time it will solve itself. You can also register to feedback loop...


Definitely recommend FBL for anyone running a newsletter. That way if you do get a spam complaint you will be notified. Take a look at the list here: https://www.port25.com/list-of-current-feedback-loops-offere...


You can run something like Zimbra. I ran my own personal server for almost 15 years and intend on going back to it.

I trusted Microsoft once with my hotmail and they somehow deleted 16 years of my emails and correspondence with a few friends who passed away.


Yeah, all you need is a business-class Internet connection (residential Internet services block mail services), and a server.

I run my own servers both onsite and in the cloud. I have my only little personal "corporate" network.


I'm not sure it's a great idea from a privacy viewpoint.

Why should anyone who has ever received an email from you know your IP address? Especially if it's a home server, that will give them some idea of where you live.


It is pretty hard to do:

https://sfconservancy.org/blog/2015/sep/15/email/


One important thing is that for Gmail you need to send your e-mail using TLS.


Surely you mean SSL (as in TLS) ? SASL is something else.


Yes, TLS was what I meant..


I do that, using postfix.

You need to also run spamassassin, with auto update, and check in with a number of rbl servers on the receiving side. (This is more important when you forward aliases to gmail and such)


I run my own mail server on a dynamic IP. Not being marked as spam is mainly a matter of how you send mail. Make your next SMTP hop a server of decent repute. Don't send mail directly.


Why on dynamic IP? Why?


Why not? It works fine. Doesn't actually change for weeks or months at a time. Implicit in your ISP hookup; no extra cost. Dynamic DNS takes care of it.


Running an MTA isn't too hard, but it's annoying. Postfix, qmail, and opensmtpd are the easiest to set up. I never actually got POP/IMAP working, so I have no advice there.


I run my own for my own personal use...not public available accounts. I've never even looked at SPF, DKIM, or cared if I was in blacklists. I rarely send mail and receive lots of it.


Are you running it out of your house (e.g. a residential ISP). If you are you'll probably never have much success, all the major email providers block residential IP addresses.


> all the major email providers block residential IP addresses.

Not really. They'll suspect them, especially ones that newly start sending email, but only very rarely block them.


VPS provider. Respectable one.


Your IP block assigned to domestic suppliers is probably the thing that kills you.

My VM in the ISP has no issues, but from home, blocked at lots of places.

What you can do is use your ISP as a mail sending relay.


Some ISP don't allow SMTP at all. FWIW, if you are hosting on Amazon for example, you can't run mail server on port 25 from an EC2 until you submit a support ticket.


True, but this is a small price to pay to setup a mail server in EC2. My reverse DNS request was answered in 48 hours. It took me far longer than that to configure the server. Most spammers won't bother with rdns, dkim, and spf, which is why you should. It's at least a first level signal that you intend to run a clean mail server and that others should trust you.


I find this a bit of an odd claim, since I have a mailserver running on port 25 on EC2 right now, without a support ticket, and have no problems.


Correct and I should have been more careful.

AWS throttle port 25. So for us to use that host as a relay server, we wouldn't be able to because of the throttle.

https://aws.amazon.com/premiumsupport/knowledge-center/ec2-p...


Yes - I just started doing it again. I'm glad I did. I just made sure that I could send/receive mail without issue from the large email systems.


Sure. My setup: https://github.com/jakeogh/gpgmda


I have been running my own email server on a VPS running slackware and sendmail for over 3.5 years now and have not had any trouble sending mail.


Is there a comprehensive guide to running a homebrew end-to-end encrypted mail server? Something roughly analogous to tutanota or protonmail?


Have you looked at Zimbra? https://www.zimbra.com/


Postfix, Mail Avenger, Linode, unison, mutt, -spf, -dkim, -dmarc. Fine for >10yr, although I'm sure the age helps with rep.


Ask Hilary Clinton


Have a look at the message headers when it goes to Gmail or any other target. There will be a hint, why it gets marked as SPAM.


Related question : how to get push notifications from a self hosted MTA? The closest option I know of is app called CloudMagic


This has to be the most arcane discussion I have ever read on HN. Fascinating and terrifying at the same time.


Check if your Server is not in RBL list, My company was host SMTP and get add in RBL very often.


Anyone aware of any newer MTAs built using Python, Go or Node? Maybe with nice JSON config.


Yes, am doing it successfully using the awesome iredmail.


Try Mail-in-a-Box. It works very, very well :-)


Anyone here have tried mailpile? Any thought?


Check with mxtoolbox.com


DMARC


Sure, but do you want another chore?


hi


It's gonna be really hard. I work at a hosting company and our staff has to work constantly to make sure people's servers get taken off of spam blacklists, or IP blocks of ours need to get removed, etc. There's a lot of stuff to navigate out there, basically kludges that have been put in place because email is just such a terrible, insecure system.

I'm sure if you're willing to put in the effort you can do it. But from my point of view, I'd probably just get a managed VPS with a hosting company who will take care of all the headaches of dealing with spam filters for me. They can be had pretty cheaply and the money is well worth it if you get good support.


To be fair, your viewpoint is very different from OP's. Where OP is trying to run a one-person mail server, you probably manage thousands of IP addresses and VPSes, which makes for a lot of abuse and effort on keeping reputation, and is an entirely different ballpark.

Therefore I'd advise OP to keep at it, not just switch to a hosting company. Also for the learning experience, though that is secondary.


Are the IPs being added to blocklists without ever having sent spam?


The spam filtering companies do make mistakes sometimes. Sometimes someone is sending out legitimate mail but it's a new IP sending a bunch of mail and they err on the side of caution.

Or sometimes your IP will just get caught when a spam company decides to spam flag an entire block of IPs.

I think if OP keeps at it, they will find that it's not worth the hassle and for a very reasonable amount of money they could have just had a personal email server where someone else deals with the headaches.

Of course, again, I work for a hosting company so I may be biased here but I'd probably be saying the same thing even if I didn't.


> I don't know what else to do

These days you also have to get a bunch of different VPSs and test sending e-mails from their IPs and choose the ones working. That's what e-mail marketing companies do. Because a lot of IPs and subnetworks out there have poor reputation or even completely blacklisted and that reputation is not generally recoverable.


That's what e-mail marketing companies do.

... and that, in turn, is why so many IPs have poor reputations.


https://mailinabox.email


You didn't understand the question. I already have the mail server running. That's not the problem.


This is probably no help to you, but since you're doing this already, you may know. Is it feasible to self host your own email server but then use something like Mailgun for SMTP?


Yes. I do this. It gets rid of the spam folder problem while allowing you to host the incoming email yourself.


I might go down this route. Great advice! What service do you use? Thanks


I'm with Sparkpost for my personal stuff, at work we use Mailgun.


For what purpose?


I believe it allows you to send outgoing mail using Mailgun which is trusted (doesn't end up in recipient's spam folder) but receive incoming mail to your server.


Most if not all ISPs have MTAs for customer use. I don't know how Comcast would like me submitting from Postfix, but I don't have to worry about that and my personal ISP can handle being a next-hop MTA just fine if I need it.


Sendmail is your friend.


People like you should be punished.


Basically not, if you have other things you're trying to do with your life at the same time.

It's like trying to make a toaster from scratch, or growing your own wheat to make your own bread. It's possible, but it's also impractical and you'll end up with a worse result than you can get off the shelf.


I don't know if "run your own" means your own everything, but Amazon SES is a pretty viable option. Even if it's not, they have a pretty good checklist for keeping yourself out of spam filters: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/deliver...


Grab https://github.com/sovereign/sovereign, follow instructions, done.


You didn't understand the question. I already have the mail server running. That's not the problem.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: