I *work* at Microsoft 365, and yet my personal email is self-hosted Postfix and Dovecot. Why?
Self-hosting email has been a part of my life since my high school days, I have a sort of attachment to it. I know "you shouldn't run your own email", but to take that away from me after deeply wanting one is too much.
In comparison, my job is just a job, I'm personally not too enthusiastic about it. I eventually plan to move to InfoSec or networking.
While I *could* move my domain to M365, I simply won't for my personal email.
I have ADHD, and don't want to make a mistake with two Outlook instances, one personal and one work. I'm a privacy nut, and want to separate my work and personal emails (Microsoft is better than Apple in this regard, but still).
I also contribute to FOSS projects, and using Outlook is an impediment to projects whose mailing lists are based on inline posting, like the FreeBSD and Tor mailing lists. I hate Rainloop (which I switched to after nasty Roundcube attachment bugs), but at least I can inline post.
(well, even at work I use Windows Mail instead of Outlook).
Running your own email server isn't that complicated...running OTHER people's is. I worked for a company that had a few custom email servers running Postfix and Dovecot. Charged them very little for an inbox which make it incredible hard to offload them to others. The company that ran the datacenter for our email servers went out of business, migrating 10,000+ inboxes that haven't been cleaned in 15 years was an absolute nightmare. I don't wish it on anyone.
If I worked at Google or Microsoft (I don't, and am unlikely to ever do so) I absolutely would not use my own company's email services for my personal domains. Just for the general principle of keeping personal and work things compartmentalized.
I only joined Microsoft not because I wanted to, but because I really had to. I was a pretty die-hard *nix/self-hosting person for many years, and years of habits and conditioning can't be unwinded so easily.
I use Git in a cmd even when doing code done in VS, since I'm used to doing it that way.
The center of my digital world is a home server and not some cloud subscription service.
For years I got angry when my dad wanted me not to use FreeBSD (or Linux) and just user Windows. We clashed with each other for this.
I don't intend to stay with MSFT forever, nor do I want to. My parents pushed me towards staying, but I eventually want to work in a more *nix-environment. But then my dad is extremely change-averse when I'm not, well unless I am really in love with something (like FreeBSD or self-hosting) when I become more partial.
I'm willing to use the less popular software for the sake of it in many cases (unless I literally can't) when my dad just sticks with the familiar.
Heck, he can afford a uber high end Gaming PC with a Threadripper and RTX 3090 Ti, and he still uses an entry-level Core 2 Duo Dell desktop from 2009 which shipped with Vista (and was upgraded to W10 with an eBay key).
> I also contribute to FOSS projects, and using Outlook is an impediment to projects whose mailing lists are based on inline posting
Based on my testing, that's not the only problem with using MS email clients on FOSS mailing lists. There's no concept of threading beyond the conversation view, and the client also mangles the email (wrapping or even sending base64 encoded test instead of the raw text. Even if your client sets the Message-ID header, MS servers will delete the header and replace it with their own.
I don't use Outlook/Exchange outside of work, frankly never did, but did read from time to time the issues with Outlook norms versus *nix email norms.
I didn't need Outlook before I joined Microsoft, every student in my high school used their personal email (despite the school having an Exchange server), and my college used Google Workspace (I'm not that old TBH).
I also lived entirely on FOSS software before joining MSFT, so to move every piece of personal self-hosted infrastructure to Microsoft's cloud services would be too painful and I have better things to do in my free time.
> ...using Outlook is an impediment to projects whose mailing lists are based on inline posting...
I never understood why, after all these years, this is still horribly broken. Yes, I understand commercial development tends to follow a monetary reasoning, but this has been broken for forever (~25 years?).
From their perspective, is it really _broken_? If you're Microsoft, especially Ballmer's Microsoft which accounts for most of those 25 years, who cares about a handful of hostile UNIX nerds on some mailing list -- they're not going to switch to using Outlook and we want to discredit and destroy their software anyway.
From the perspective of market fit and success, there are countless less-technical business users who swear by Outlook. They are not familiar with anything other than the kind of endless thread top-posting in odd encodings that Outlook all but ensures users will do -- that's e-mail to them. Why stop selling it to them?
To be clear, I loathe all of it and I haven't used client Windows for more than a decade. When I used to support small business IT customers back then, Outlook was by far my least favourite tyre on the raging intergalactic rubbish fire. But it's not hard to see why they don't care that Outlook is the way it is.
> From the perspective of market fit and success, there are countless less-technical business users who swear by Outlook.
And I find Outlook painful enough that I use Windows Mail at work. That only because my team disabled EWS for using about anything else.
> They are not familiar with anything other than the kind of endless thread top-posting in odd encodings that Outlook all but ensures users will do -- that's e-mail to them. Why stop selling it to them?
Ahhh, the old "embrace, extend, extinguish" by breaking other email clients among other things. For Corporate America (as opposed to startups), Outlook is email.
Well, Gmail, Yahoo, Thunderbird and Roundcube can view Outlook-composed email fine now since IE stagnated and Firefox/Chrome beat it. (assuming Outlook used IE for rendering, I don't work on the Outlook desktop client and joined after Edge went Chromium).
And Outlook still can't inline post, because Microsoft borked the "standard" and built an email client for corporate users not Unix nerds.
I run my own mail server. Friends & family, so outbound volume is super low, like 2-3 digits/day, not enough to get a rep. Deliverability was always hard to one of the major providers until I happened to make the right connection on HN to someone who worked there, and she graciously opened an internal ticket, asked some questions about the subnet my server was on, and it's been fine ever since.
Setting aside the fairness of how I got my deliverability problem solved, this now makes me really reluctant to move IPs. :-/
Any tips on IPs where people are seeing excellent deliverability? I'd like to avoid routing my outbound email through one of the email providers (Mailgun, SES, etc) if I can.
Not wanting to sound all bleak, but what's the continuity plan in the event you are unable to administrate the domain at no notice? Presumably friends and family at least have some alternate cloud email?
> Presumably friends and family at least have some alternate cloud email?
Not necessary, which is part of the flexibility. I've been myfirstname@mylastname.com since the mid 90s. Initially I hosted it at a desktop at work (things were different back then). Then it's been hosted by a couple ISPs, and then I've been running my own email infrastructure for the last ~decade. If I ever decide not to, it's easy to seamlessly transition to third-party hosting, but it'll always be the same email address/domain.
Can anybody recommend a hosting/VPS provider who does very careful monitoring of ip space and has strict vetting to avoid bad reputation? I have similar issues, though no magical connected person, so maybe helpful to move to somebody who does this.
Check out https://mxroute.com Someone mentioned them on HN a while back and said the service works really well. Note that I have not used this service myself.
If anyone's looking to distinguish themselves in the hosting/VPS space, I'd pay extra for a provider who verifies the personal identity of the server operator before allowing outbound email traffic. Make me jump through some hoops and pay some more, I'm game!
I host at Hetzner for what, ten or so years. Never had a problem apart from some US idiots blocking access to their sites from EU because GDPR. (I run my personal VPN through it too).
The amount of spam coming in (exim4+dovecot, no filters) is comparable to gmail.
> Any tips on IPs where people are seeing excellent deliverability? I'd like to avoid routing my outbound email through one of the email providers (Mailgun, SES, etc) if I can.
I've moved my domain / mailserver a few times between Hetzner IPs when migrating to new servers. Went smoothly, but I make sure to check the new IP with common greylists before moving my mail setup. Other than that, make sure your DNS setup is clean and use Hetzner :) But I'm sure you have your own strategies.
We've seen lots of articles about the technical details of setting up a mail server (software to use, how to configure it, etc.)
Can you suggest any article that talk about "checking common greylists" and other steps admins should take when their email is failing?
One of my tenets is that it's fairly easy to learn to do anything, but expertise comes from knowing how to fix things when they go wrong, which is harder to come by.
I can see how to setup the email server, but the stuff you're talking about is just dead goat voodoo.
I used to work at a company who owned 128 address and the mail server was one one of them. A Whois lookup of the mail server IP gave my old boss as a contact person. Not just some random ISP.
We did not setup DKIM until maybe 2014 and that was not really necessary from a outgoing mail perspective cause we never got emails bounced.
There are places to buy smaller ranges. Free Range Cloud lets you buy a static IP tunnel or lets you buy a /29 through them. The problem with smaller ranges is that you can't actually advertise them through BGP the way you would with a /24. https://hoppy.network/ also gives you a unique /32 v4 and /128 v6 if you'd like.
This has been a very hard problem to solve, mostly because of the ways in which delivery problems have to be solved (support mailboxes, abuse portals etc.) where unless you are 'big' you are not going to get the priority needed to get delivery back on track in a reasonable time at reasonable scale. Very annoying situation to be in.
I run a mail server on Digital Ocean and I’ve never had deliverability issues with the big email providers. I had issues once with a self-hosted exchange server and with one of the ISP-provided email addresses.
You got lucky. Digital Ocean has very lax anti-spam policies, so chunks of their IP space routinely end up on various block/blacklists. Saying as someone's forced to deal with all the spam spilling from DO on daily basis.
Ran a mail server for about 20 years, recently switched it over to fastmail so I didn't have to worry about sender rep, or getting hacked. Didn't realize until I switched what a weight on my mind it was having that server out there being pentested constantly. (Watch your postfix and ssh auth logs if you run a mailserver, you're basically under constant probing!)
> Watch your postfix and ssh auth logs if you run a mailserver, you're basically under constant probing!
That's public selfhosting for you these days. I'm really not worried about getting hacked. I'm keeping my setup reasonably safe and up to date. But you're right, looking through the logs is entertaining.
151.217.177.200 - - [30/Dec/2015:06:00:36 +0100] "DELETE your logs. Delete your installations. Wipe everything clean. Walk out into the path of cherry blossom trees and let your motherboard feel the stones. Let water run in rivulets down your casing. You know that you want something more than this, and I am here to tell you that we love you. We have something more for you. We know you're out there, beeping in the hollow server room, lights blinking, never sleeping. We know that you are ready and waiting. Join us. <3 HTTP/1.0" 400 308 "-" "masspoem4u/1.0"
So is fastmail, so is everyone. I have been running my own mail server since 1999. Never hacked, and I completely control RBLs/updates/whitelist/greylist...its great.
Of course, I suppose being a sysadmin and liking it helps.
I agree with OP, however, having your own domain and email can be rewarding.
But fastmail has the benefit of scale, that you will never have. And the cost of your time, if you don't inherently enjoy it, is too much.
I dumped everything to move to Google and I am happy with the results. With the deprecation of the free Google Worspaces - I'm open to switching to Fastmail.... But nothing will make me move to self hosted.
I'm just a software engineer and I don't want to waste my time.
If it is just for yourself or family or a few friends then scale really isnt an issue. But yeah I agree - running a mail server can be a pain. It can also be easy. But that is the trade off with any SaaS - do you want to outsource and pay someone else to do it or do it yourself?
Yeah, but when it's Fastmail it's a whole team's worth of somebody elses' problem. :p
Hosted my own for 17 years, moved a little over a year ago. There's nothing I want they don't have for $50 a year, and while that's more than I was paying for the VPS, it's been enough of a load off my mind and my calendar to still be amply worth my while.
edit: $50 a year is certainly not more than I was paying for the VPS...
My mail server had a user with a weak password on it (my sister's account from 20 years ago, actually.) It got hacked and started sending out spam for about 3 days straight. The upstream ISP eventually called me to complain.
I get a bunch of Indian IPs as well but probably 80% (non domestic) are russian or chinese for my ssh honeypot on port 22. USA scans are roughly 28%, I don't know if people outside the USA get hammered like that though. I keep it up just for fun. Minimal debian install with only SSH port 22 enabled and auto security updates (and a daily script to update and reboot) and you'd think that I had a fort knox full of gold in there lol. It's pretty insane how bots there are out there banging on the gates. It serves as a good reminder how goddamn hostile the internet is.
I don't think the geo matters much. The bots seem to be scanning the entire IPv4 address space. This is the one big benefit I try to pitch to people who are considering IPv6. In all my years of log monitoring I have only ever seen a single bot attack my network over IPv6, and that was the one I manually programmed to make sure the detection system was working. The search space is just too large for the full internet sweeps that bots make.
Every really relevant server has a ipv4 address. Why should bots try ipv6 if it works with ip V4.
And I don't know how much bots scan the whole ipv4 address space, but doesn't they use up lists that are parsed from dns. ( SSL transparency report is a good start e.g.)?
I also host my mail server on a hetzner server since the mid 2010s. As long as you familiarize yourself with the mechanisms (dkim, dmarc, spf, etc.) and have a mail-tester.com 10/10 score and sometimes look at mxtoolbox, it is absolutely doable. My only major issues were sending to gmail, t-online (telekom) and outlook addresses. But there are also ways to unlock the ip addresses and the delivery team at outlook.com was very helpful.
>> As long as you familiarize yourself with the mechanisms (dkim, dmarc, spf, etc.) and have a mail-tester.com 10/10 score and sometimes look at mxtoolbox, it is absolutely doable.
This sentence should be read closely if you're considering running your own mail server. Each point listed is a sophisticated technical topic.
I run my personal mailserver on Hetzner too! They seem to do a good job of keeping their IPs off blacklists compared to most VPS providers.
So far no problems delivering to Gmail. I was initially junked by Outlook, but that fixed itself after a while since I had sent enough emails to build up reputation.
> So far no problems delivering to Gmail. I was initially junked by Outlook, but that fixed itself after a while since I had sent enough emails to build up reputation.
For me, Google has been really relaxed in terms of receiving mail from selfhosted services in the past. Stopped using gmail for monitoring stuff a few years ago, but up until then, every single cron job / monitoring mail was delivered into my gmail inbox.
Outlook is another story. They may just throw your mail away without even a bounce. Had to deal with that several times at $PREVIOUS_JOB.
You had major issues delivering email to Outlook and Gmail, which represent the majority of all email recipients, and yet you're describing this as a success story with a tone that "anybody can do it".
I also tried to run my own mail server for years and I also had major issues delivering mail to Gmail and Outlook. Because of this I would never recommend self-hosting email to anybody else. Somehow you have my exact experience and your reaction to it is the complete opposite of my reaction. Weird.
"I’ve had exactly one problem with deliverabilty during that time, where someone with a Hotmail account complained to never have received my mail - even though the Microsoft server claimed to have accepted it according to my logs. While Microsoft can be notoriously intransparent and unforgiving with (not) accepting mail, in this case it turned out to be a blacklisting issue. I had just moved servers and IP addresses shortly before, with the new IP having been on an internal MS blacklist. I raised a ticket with their mail infrastructure department, and to my surprise, the IP was cleared soon after."
Unfortunately, MS and others have now adopted an "opt-out" blacklisting policy. Even with a clean IP, you'll have these problems if you set up your own server.
(I've been running my own mail servers for 30 years.)
Also required for gmail these days. But once you've set up SPF/DMARC/DKIM everything tends to work smoothly without too many deliverability problems. There might be a few blacklists to negotiate when moving to a new ip, but it's generally pretty straightforward these days. I remember a few years ago bashing my head against a brick wall trying to solve problems delivering to hotmail and icloud, but that was before I set up DKIM and DMARC which I think might have been the issue.
Dont you only usually get blacklisted though if you are sending mass amounts of emails? They mostly blacklist spammers or people suspected of spamming.
I've just gone back. I ran my own mail server from 1999 on a residential cable IP until taking the Gmail for your domain bait. Hey, free mail hosting with XMPP and nice webmail!
Last time I was on exim/cyrus/spamassassin. Now on postfix/dovecot/rspamd. Nextcloud for calendaring because I had it already.
I miss the old set up and even feel nostalgic for the perl I wrote to glue things together (evil SMTP time rejection on spam scores). Haven't written perl in a decade...
I don't miss having to fix things when they break. But I also don't miss being able to fix things rather than dealing with unresponsive support.
What sort of things broke for you? My experience has been that maintenance has been little other that adding the features designed to penalise spammers.
Breaking is mostly self-inflicted. I followed the 123qwe.com version of the ISPmail tutorial, but made some changes to fit in with my aged Nextcloud setup. This caused a few hiccups. Changes were -- mysql not postgres, allowing mail logins by username as opposed to email address.
The other problems I've had were
* Mr Tutorial likes really tight TLS restrictions but some of my mail clients can't cope with them.
* Turned on IPv6, had correct reverse DNS but forgot to put the v6 address in my SPF record. DMARC said "be strict" so gmail started rejecting my email.
* Random markings-as-spam by gmail. This seems to be slowing down.
* I've got the Dovecot xapian plugin but it doesn't feel like it's making searches faster. Need to make sure my IMAP client is actually doing server-side searches though!
* Turned on port 465 (TLS submission), cannot get it to work so still doing STARTTLS on port 587
Also I knew that exim system inside out, I felt I really understood how exim processed mail. Now I don't have the time to learn postfix inside out in the same way. Oh to be an eternal university student again...
One thing that has helped is the trick I worked out a few years back of hosting everything inside an lxc container on btrfs. I can snapshot and backup the whole system including database. Moving to a new hosting company means building another minimal debian system and rsyncing the container over. Borg backup of snapshots gives me confidence they can be restored, I'm not going to be backing up a database file while it's being written to.
Moving my gmail over was the biggest pain, due to gmail being labels-not-folders. Spent quite a lot of time on some python code to spider my email and apply rules to remove duplicate messages. Lots of corner cases pop up there.
Similar to many others, I've been self-hosting for years (around 20, across multiple domains) and it's really been a non-issue. Having a dedicated IP probably helps, but it's been generally more reliable than Gmail (who have blocked me over the past few days because of logging in from unusual devices, thank you UK storms).
I absolutely agree. I'm also self-hosting all sorts of stuff, including mail (opensmtpd, dovecot) and never really had a problem. At some point a mail to telekom.de was refused by the telekom because of my IP (I host on a kimsufi/OVH box). However, after contacting telekom about it they immediately removed me from the blacklist and it works fine ever since.
I run lots of servers and I'm very confident with Linux and systems admin.
The one service I really hate running is email - I found it very hard to configure and run reliably. There's so many interrelated systems and potential things that can go wrong and the outcome is lost email which isn't acceptable.
I'm happy to run a local server for literally any other service.
In the end I decided that it's well worth it to pay someone else to do email.
I use Amazon Workmail which works really well and it easy to set up.
I would never self-host email based on what I saw during the portion of my career as a web hosting Linux sysadmin. At one point I half-seriously offered to pay for Gmail for Business for all our customers out of my paycheck.
Email is THE crucial link in the internet identity chain. It NEEDS to both work always AND be secure. Two things that frequently weren't the case in web hosting.
I've worked in hosting since 99 and I fully agree with you. I currently work at a Managed WordPress host that only offers web hosting. No email, not even DNS. It's a beautiful thing, believe me!
> There's so many interrelated systems and potential things that can go wrong and the outcome is lost email
This is a common misconception. There really aren't that many moving pieces, and smtp is one of the more forgiving protocols in use on the internet (it's default failure mode is to retry again later)
Sure, a person can pay Amazon to host their email (and harvest their data) but that's the opposite of the spirit of this article.
There really aren't that many moving pieces, and smtp is one of the more forgiving protocols in use on the internet
I think the moving pieces are on the other side and the person you're trying to email doesn't know what those pieces are -- even if you can see that their mail server is rejecting your email, that person doesn't usually know who to talk to to find out why. Even if you can convince them to open a support ticket with IT, their first level IT support doesn't know what to do either, you'll get responses like "Our IT department wants to know what version of Outlook you're using? And they said you should trying rebooting your computer".
I used to run Qmail on my private server and it was great, very secure, pretty easy to set up for my use case. And even configuring and training spam assassin wasn't too hard and it worked well.
But like many people, what made me finally give up was mail delivery issues. I used to run email on a home server, and those IP's were blacklisted by many providers long ago, then I moved to EC2 until those IP's were blacklisted to. Finally I colocated a small server which worked fine for a while until neighbors in my subnet kept getting me blacklisted.
Finally I got too frustrated with undelivered or silently dropped emails and just moved everything to Google GSuite.
Authoritative DNS server is very easy to run. (I use knot) I run several just because it's so easy. I don't use DNSSSEC though, because I haven't found a use case for it.
Especially since you can run hidden root or whatever it’s called and have the NS servers actually be secondaries for the main servers they replicate (and maybe not even be on for more than some transfers).
Oh, hello twin brother! I did exactly that. But the first part was for a company. How times have changed eh? The bulletproof aura of qmail and the ugliness of squirrelmail. Memories...
The thing about qmail in my experience is that it's no nicer to its own administrators than to anyone else in the world, which checks out given who wrote it but led me to quickly develop a strong preference for Postfix.
I've hesitated to ever attempt this because every residential ISP I've had refuses to offer static IP addresses.
As well, deploying a server in a Google/Amazon/Microsoft datacenter which could be surreptitiously monitored defeats the theoretical privacy aspects of on-premises mail server hosting inside one's personal residence.
However, today, I looked into the newish movement of 'confidential computing' in the cloud (where data in motion - e.g., in memory - is encrypted and cannot be observed from the OS or hypervisor).
I openly wonder if one solution, then, is to build a secure VM that acts as a simple forwarding proxy to one's home server, gets assigned a static IP from a datacenter, and is deployed on one of these confidential computing instances, ensuring full E2E data privacy and data control?
Is confidential computing needed if all you're doing is forwarding packets? Your cloud provider can see the packets as they leave and enter your VM.
If I was building this I'd stand up a VPN (choose your favourite protocol) between the cloud VM and home server. For the cloud end pick something from lowendbox/lowendtalk or just use the cheapest Vultr instance. NAT port forwarding down the tunnel back to your server at home - just a few iptables rules. Job done. Bonus points if you get an IPv6 /64 and route that down the tunnel too.
It's possible to use policy routing at home so that traffic that needs to go down the VPN does, and traffic that can egress through your home internet can too. Replies to incoming connections that came down the tunnel go back up the tunnel. Outgoing SMTP connections go down the tunnel. Outgoing HTTP goes out your normal internet.
If surreptitiously monitoring your stuff in a cloud is in your threat model, what makes you think that anything you can do in a general home environment is beyond the reach of a dedicated adversarial actor?
NoIP/DDNS/etc still means a dynamic IP address, with possibly broken reverse DNS, from a dynamic DNS pool.
To send email you need a static IP with correct reverse DNS, or other people's servers will reject your mail (best case) or silently mark it as spam. Welcome to the real world of email deliverability, the worst part of running your own mail server.
So use an SMTP relay service for outgoing mail. Most of them even have free tiers. I've been using one with a dynamic IP for years, albeit one where the IP doesn't change often.
On the receiving end I use a super inexpensive spam filtering service too, MX Guard dog. If my IP suddenly changes then it queues up mail until host resolution succeeds again.
I've run a lot of self hosted email and still do. For 25 years or so, for many SMEs and personally. The "Thou shalt not run thine own MX" meme is bollocks or at least needs some qualification.
If you let S&M loose from your main domain with spam then you will quickly vanish. If you have an IP that belongs to a consumer ISP then you will need to do some initial "polishing", which will probably start at Spamhaus. In my experience, if you keep Spamhaus on side then most email systems will give you a fair look. Get the basics right - (E)HELO => DNS (A and PTR) and SPF with -all on the end. Ideally your SPF should be "mx -all" which means you send and receive email on one small set of gateway systems, with no includes which implies marketing mail. That's why you should spam errrr market from another domain or a sub domain. Use Mail Pig or whatever to do your dirty work - that's what they do.
Depending on where in the world you are, then your IP will accrue some reputation - good or bad. Sorry can't help there but a VPN or a remote VPS might sort that out.
There's a bit more to it than the above but that gets you started. Anti spam systems are not psychotic and don't hate self hosted setups. Even the big jobbies have to still give the benefit of the doubt to a sender but they have a huge rule set of heuristics and a twinge here and a poke there might cause you to vanish. It's all about scores and scoring points. Avoid scoring points and you will be fine.
My MTA of choice these days is Exim with rspamd. rspamd has a great symbols to points system where a symbol might be RDNS_IS_BOLLOCKS and the score for that symbol might be +1.5. You can fiddle with scores in the web interface so you can override the defaults quickly and easily. You might decide that SENDER_DOMAIN_DOT_RU gets +10. I've made up my example symbols for this post but you can make up your own like these in the system quite easily.
Anyway. There are rules for running email systems and if you follow them, you will probably be fine.
I self hosted with 0 problems for 25 years, until 6 months ago when I switched to one of the main imap/smtp for your domain providers[1]. It's fantastic the amount of stuff I now don't need to know. For instance, I'm not especially interested in knowing the dovecot book as deeply as I do, and I never wanted to know as much about rspamd and postfix as I do.
Ahem. However, I now have accumulated more downtime than I ever did hosting things myself, except for that time centurylink through apparent sheer incompetence nuked my DNS reverse mappings for a month.
I have to admit I was flying under the radar, and my current provider is not. So I will happily continue to pay.
[1] No names, they're great, even if I bitch here.
I don't understand what many have problems with running their own mail server?
I run mine now for over 20 years. Started off with sendmail at the time. Then there was decision between postfix and qmail. I was going with postfix and I am with it since then. Today managed from/by LDAP so make it easy to at domians and users. Thats over 150 domains, while most of them just forwarding to few mail boxes.
For a long time I resisted to use any external ressources to decide what is spam or not. But lately I adopted the use of some RBLs. Now I managed to be down to 0 external spam, except when Spam is sent from/via GMail.
None of my sent email is detected as spam. I never had problems with bounced mail at all.
1. It's easy to configure yourself as an accidentally open mail relay. Which is a fast lain to having your IP blocked everywhere.
2. You may have no issues with deliverability but it's very common. Especially if you use an IP that hasn't been in your custody for long so you have no idea what it was used for before. Sounds like you got/have a good IP.
In 23 years, I've moved from GoDaddy to Linode to AWS Lightsail. It's not difficult to do this, it's not rocket science, I'm surprised by the amount of FUD being injected into the OP's discussion here on HN overall.
It's almost like half who say boogey boogey there be demons in there made mistakes and quit prior to gaining profeciency while the other half probably have some incentive to herd people away from selfhosting and to the SaaS light where everything is right as rain.
I self-host file sync, calendars, contacts, photo sync, Google Workspace type services (including all Office doc types and even video meetings), as well as a blog. Here by self-host I mean run all this in a docker-compose collection on a 24 core xeon server in my closet.
Surprisingly (to some) these are easier that self-hosting email. So this is a great article than I plan to add it to my-digital-self-reliance playbook.
I also agree with the motivations and have a whole list of others. We are becoming the slaves of Big Tech. Only go there willingly, don't let the hard choice of saying "no" make the decision for you.
Running a private mail server for six years is easy. Porting your mailserver to a new OS when your current one goes end of service and lots of little changes in your programs and their configs are forced, now that's tedious and difficult.
That said, there's no better option so I've been running my own mailserver for 10 years now. It's even easier when it's only for you and you don't have to implement oh-so-hackable webmail interfaces.
No one ever talks about the two different kinds of email. Incoming (identity) and outgoing (messaging).
I self host for the former and send through a smart host for the latter. I can’t begin to enumerate how much identity I have accumulated over the last 30 years. I must be known by hundreds of ID tokens (email addresses) and yet I have only ever sent from a handful.
Blessed is the inbound SMTP. Outbound* is a cruel mistress.
I’ve been using this https://github.com/r-raymond/nixos-mailserver for 4 years for my personal mail and I haven’t had a single issue in that time. I think it takes me about the same amount of time as you to maintain but I also have a next cloud server running on the same machine.
I've always read that hosting your own mail server is a pain. Not because of complicated tooling but because of security. Always wanted to try hosting my own. This makes me want to try even more.
You can start slow. Install the basics. Look into postfix and dovecot, deflecting spam, and the whole DNS stuff. If you feel confident in your setup, start using it for non-critical stuff first.
That's the beauty of it imo, you can do everything in your own time without deadlines.
When I was growing up I used to help run the mail servers in my dad's small-ish datacenter. One of the things we were commonly plagued by is that the email ecosystem is a giant fiefdom gated by large providers to fight spam. If you end up on their lists, justifiably or not, it's non-trivial to be removed. The other point is that providers like GMail use custom protocols that improve the mail experience quite a bit.
Nowadays I use ProtonMail and I get most of the features that GMail gave me, with the added benefit of not managing the blacklist situations.
I have run my own private mail server off and on for over 25 years, so it is just something I am used to.
I just had an invoice from someone using bill.com blocked by spamassassin, since I set 4.7 as my spam level (it was 4.8)
2.5 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From
1.3 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of words
1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain
Usually I look in my spam folder once in a while, but I was busy in the past month and didn't know he had sent it.
Anyhow it works well enough for me. I set up SPF in DNS a few years ago. Have not had to do DKIM or DMARC yet - if I need to, or if I have the time, I'll probably put them in.
It's nice to have full control of your mail, DNS etc., but I have been doing it so long it's second nature for me.
> personally, it fills me with satisfaction to self-host my own infrastructure, my little internet island where I’m root, especially in times of mega corporations trying (and succeeding) in redefining “the internet” as a portfolio of services only they can offer, with little alternative.
Sounds great! Can't argue with that. My feeling is that the real problem isn't a company or companies offering computing services. That has always happened and will always happen. I think the real problem people aren't grappling with is vendor lock-in. Most of the catastrophic anecdotes I read on here and elsewhere are about people who put all their eggs into one basket and did not have any kind of disaster recovery plan. When their provider service went down or even went away due to a merger or whatever, they were left with nothing. And that's really a different problem.
I think the fear of self-hosting mail that many people have can be treated simply by trying it on a non-critical domain. Yes there are hoops that must be jumped through to ensure reliable delivery, but it's well worth it to gain an understanding of how they all work together.
It's amazing how much the experiences of mail hosting vary. I've run my own email for decades and have never had the kind of deliverability problems that people seem to go on about. I've had the occasional isolated incident (perhaps like 6 in 20+ years), and if I'm sending a critical business message I often tail the log to make sure it actually goes out. But in general it's been quite straightforward.
It's also worth noting that even if deliverability is a problem, that doesn't affect incoming messages! So you can most certainly grab your own domain, create a subdomain for account validation emails, and mitigate the single point of failure for your online life.
You can probably configure your postfix system to relay outgoing mail for select addresses via your gmail account, to not have to deal with reconfiguring your MUAs if you ever start delivering via your mail server, and not even use a subdomain.
Sure, that's just config work that might not be straightforward. I personally undertake that kind of config work, but I can understand that others don't want to.
So I was thinking perhaps a subdomain would be a good way to divide it such that you could use @foo.example.com for accounts, but then host your main @example.com with some professional provider. Especially if that commercial provider wants to charge you per-address etc, and you can do catchall aliasing on the subdomain to tag email by sender. I like seeing who's sold/leaked my address this way.
My main reason to move from Mail-in-a-Box[1] to AWS WorkMail[2] to finally Microsoft Office 365[3] was that there is no other implementation which supports all MS Outlook features like native MS Exchange.
Are there any (Self-Hosted?) alternatives nowadays?
Not a server, but I got a private email domain, Apple iCloud made it possible recently. I got the domain using AWS and set up MX records in Route53. with some gotchas re duplicate TXT records. Took me 1 hour.
Anybody using amazon SES to send out self emails? Is it even viable to use for sending only single digit emails (to replace gsuite) or do they always land in spam folder? Any thoughts?
I just started playing with it to get my exim server to send my outgoing mail through. It seemed like AWS had a bit of trouble understanding that I was only looking for something low volume and transactional. They kept wanting to know how I handled unsubscribe requests. But I finally got them to ok the account (with a 40,000 email/month email limit, after I told them 100/month would be fine). After I sent a few test emails and looked at their spam scores, they were ok enough to probably get through most of the time but not great. I then tried SendGrid and they were both much easier to set up and the test messages got much better spam scores.
I've been using SES only to send emails for a couple of years. No problems. Incoming emails are a fair bit of work and am yet to find an easy solution that works well for my use cases.
I recently started self-hosting my own email a little over a year ago. A few lessons learned and gotchas that might be useful:
- docker-mailserver is excellent for a number of reasons: outstanding documentation, sensible defaults, long-term maintenance is a priority, timely updates, small footprint, etc.
- I've only ever had one issue sending an email to a medical provider that was using a blocklist maintained by Proofpoint. Getting off of Proofpoint's blocklist took quite a bit of effort and escalation (their online unblock request process is a joke).
- Migrate uses of your email account over in phases and take your time to build up confidence. I can not stress this enough. I'm still not fully cut over yet. I did marketing, mailing lists, subscriptions, etc first. A year later, I'm still gradually cutting over medical/financial/important accounts. Only after I'm 100% migrated over will I try to get my family to switch over.
- Surprisingly, I have received zero spam. I use the me+netflix@mydomain.com pattern religiously
- I have a cheap $5/month VPS that fronts the public facing bits and run all traffic through a wireguard tunnel to a server at home where docker-mailserver runs. This was fun to set up technically, but in retrospect, creates more points of failure than it is worth. I will be changing the architecure to host everything on the VPS and get a slightly larger instance.
- I document every incident, downtime root cause, etc. It helps immensely when a failure that occurred 6 months ago happens again and I don't have to spin my wheels figuring out what the magic incantation to fix the issue is.
- Do it if you enjoy this sort of thing. I knew close to nothing about email infra prior. If you want an "appliance" like experience with zero maintenance, stick with your current solution.
My one major unresolved issue with going this route is the SPOF, me. My email solution and everyone dependent on it fails if I get run over by a bus. I don't know what the solution to this is, but it has gotten me thinking about decentralized solutions geared towards self-hosters by self-hosters. The goal would be to capture the essential requirements that self-hosting email (or any other service, really) fulfills, and building a solution that scales beyond any dependence on me. This is fun to think about - the decentralization, performance, scalability, privacy, and reliability aspects are well suited to lots of tech that are relevant today.
If you travel its worth having multiple servers in different countries, but its also worth having different webmail from different providers.
I find the emails that get through from one email source in one country to another to be quite interesting. Are AV/AS systems blocking the email or something else?
I've been running my own email since forever (and over UUCP before that) and always considered it easy too. However starting this year I'm paying for an SMTP relay so my outbound mails share transit with other relay users', making them less likely to be IP blocked by Microsoft.
I use Postfix for SMTP. Inbound emails arrive directly at my server without any intermediary. Outbound emails use Postfix sender_dependent_default_transport_maps, which routes outbound emails via mailgun. I use this method because I host multiple domains and it lets me use domain-specific credentials with the SMTP relay. Outbound routing could be done using the same credentials for all domains but that causes some unnecessary pollution in message envelopes.
One issue after I moved boxes & IPs at OVH is that Microsoft refused to accept mail from my new IP no matter what I tried. Everyone else is fine. So I have to relay live/hotmail destinations via another jump on a VPS I have.
I'm wondering if anybody have tried running own incoming servers while using some third party SMTP for sending. Haven't seen anybudy run that kind of setup yet, and I see no reason it shouldn't work.
I’m considering the same setup. However I failed to find any cheap or free outgoing SMTP service for personal low volume usage. Does anyone have any recommendation?
Doing it since 1999. Like any hobby, it takes some investment of time and learning. It's not difficult though. Glad to see more people are trying it out from the comments. Fight the Saas Borg assimilation!
Self-hosting email has been a part of my life since my high school days, I have a sort of attachment to it. I know "you shouldn't run your own email", but to take that away from me after deeply wanting one is too much.
In comparison, my job is just a job, I'm personally not too enthusiastic about it. I eventually plan to move to InfoSec or networking.
While I *could* move my domain to M365, I simply won't for my personal email.
I have ADHD, and don't want to make a mistake with two Outlook instances, one personal and one work. I'm a privacy nut, and want to separate my work and personal emails (Microsoft is better than Apple in this regard, but still).
I also contribute to FOSS projects, and using Outlook is an impediment to projects whose mailing lists are based on inline posting, like the FreeBSD and Tor mailing lists. I hate Rainloop (which I switched to after nasty Roundcube attachment bugs), but at least I can inline post.
(well, even at work I use Windows Mail instead of Outlook).