Writing applications is hard. Designing a website is hard. Professional work is hard.
But setting up your initial mail server is not hard. Read some guides (ahem: https://flurdy.com/docs/postfix), fix a lot of typos, and you're up an running.
Same with developing the initial version of an application, or designing the first draft. Not rocket science. Not easy but not that hard for an experienced person with simple initial requirements.
But what is hard is keeping it running with more and more users, more requirements and evolving tech and less time to maintain it. Never mind forgotten skills and less priority.
Mail servers have a special knack for running out of disc space or going down for some reason only when you are on holiday and can not get online easily. And you usually only find out after a day or more as you start not receiving messages but don't really notice it for a while.
Or you friends, family or colleagues using it start complaining that some random person they are emailing is not receiving their email and you need to look into it... (Ps. Use DKIM and then DMARC reporting)
If you just want a reliable mail service, just use https://fastmail.com. If you want even more freedom and configurations, do try to set up a mail server. But it is hard. A bit hard.
> If you just want a reliable mail service, just use https://fastmail.com. If you want even more freedom and configurations, do try to set up a mail server. But it is hard. A bit hard.
I used to run my own mail server, but I eventually migrated to Fastmail. I liked then because their software stack was similar enough to mine (compatible filter rules) and I knew where to find their admin on IRC.
The reason I made the switch is because I was sick and tired of my Email reliability depending on the reliability of my home Internet connection, and my co-lo/remote-hosting situation (at the time) had different reliability issues I didn't want to risk.
You switch to a third party mail provider for the same reason you buy a new car instead of driving around your classic you've been fixing up as a daily driver. Not because you don't like it, but because sometimes there's benefit in offloading the annoying bits of your hobby so you can focus on the interesting bits.
Fixing up an old car is fun. Doing it in chunks of a few hours so you can drive it work the next day is less fun.
Running your own mail server is a fun challenge at first, as you have to figure out why some random site won't talk to your server, or why your load spikes at random times. Then you decide that diagnosing other people's broken configs and dealing with drive by spammers is annoying and boring.
yeah, but compared with other services like WWW mail is IMO harder. You deal with two main Protocols and a lot of software. While mail takes 11 components in your tutorial, you'd get a website in 4. And some parts are hard to debug. I have very little means to debug why my mail wasn't delivered to my bosses gmail.
I've built https://kopi.cloud to try to provide an alternative option in between being locked in to a big mail service and running your own mail servers.
It's a consumer-oriented SaaS for taking control of your personal mail - there is no enterprise pricing, nor even a subscription pricing option - though you can you can choose to pre-pay for the service years in advance.
Kopi uses Amazon SES for email delivery. We know Amazon wants to make sure that their service is a viable product for mail delivery at the commercial enterprise level. The idea is that Kopi can survive as a viable retail product by riding on the coat-tails of that desire and arbitraging the cost of mail handling via SES so it's cost-effective at consumer level.
The intention is to run it as cheaply as I can, ($1USD / month at the moment, though I make no guarantee that it can stay that cheap forever) and to give people the ability to choose between different mail services.
The idea is that you can manage your own domain, and setup your DNS to use Kopi as your mail exchange via DNS MX records to forward your email to whatever mail service you choose to use.
If people don't want to even run their own domain, they can can use the shared Kopi domains if they want - with the understanding that those mail addresses are locked to the Kopi service, but at least they can change mail service whenever they want.
The goal is to be easy enough to use that you can set up your email handling via a smart phone, and cheap enough that running your own email handling using free software should seem expensive by comparison.
Though I'll admit - marketing this thing is a bit of a problem right now :)
Thanks for mentioning some alternatives, but I've just checked the first example and it does not support custom domains (which Fastmail does), so it does not "provide more".
Depends on your definition of “more” and also who you are as a user.
Just for arguments’ sake, I could argue that Fastmail’s privacy stance and policy don’t matter much compared to Posteo because of what’s written on their websites as well as which jurisdictions these are located in, and hence Posteo is better for privacy. I could also argue that for non-technical people, having a custom domain and managing it is a pain, compared to just choosing some domain from a list of predefined domains in a drop down and not having to worry about domain renewals, MX records, etc.
I’m not saying that custom domains are bad (they do allow you to switch to another provider easily), but there are trade offs to consider in both cases, and hence what may be “more” for one may be “less” for another.
Also consider the price difference when multiple mailboxes are required. Fastmail becomes quite expensive very soon compared to the others.
Where did I apply it to everything? To falsify an argument you only need one counterexample. I'm happy with all the alternatives (as mentioned before), but I'm not happy with a claim that can be falsified with the first example.
> To falsify an argument you only need one counterexample.
If you want to apply logic like that, you have to apply it correctly. To falsify a universal claim you only need one counterexample, but the original poster never used a universal term (all, never, etc). If it wasn't a universal claim then you can't apply the rules for universal quantification to it.
More to the point, my comment wasn't attacking your logic, but that you seemed to dismiss the entire list based on the one example. I didn't see where you mentioned you were happy with the alternatives and only critical of the one, if that was the case then I just missed the larger context and I apologize.
One of the "more" things many other services provide is far better privacy. European law is much more protective of human rights than American (where Fastmail hosts its servers and emails are "business records") or Australian (where Fastmail is located and where even warrant canaries are illegal).
I also use Fastmail to host my calendar and contacts - next to my email. I enjoy the web interface and lI especially like that the iOS integration just works.
I think the biggest pain in hosting your own mail server is getting your outbound mail delivered into the mailboxes of the large providers without being marked as spam. Especially if you don't actually send a lot of mail, so you can never really build up a good IP reputation.
That's why I generally recommend a hybrid setup: Host inbound mail completely by yourself so that you have full control, but ship off outbound mail to a trusted relay of a privacy oriented provider. For example https://posteo.de/en doesn't filter any outgoing messages by sender, so you can send mail from your own domains. Your local Postfix can DKIM sign your mail before sending them to Posteo and if you add include:posteo.de to your SPF record all your mail will be DKIM signed, SPF authenticated and coming from a reputable IP, so all your deliverability issues will be gone.
> I think the biggest pain in hosting your own mail server is getting your outbound mail delivered into the mailboxes of the large providers without being marked as spam
Why do you think that?
The article mention exactly this could just be a myth that perpetuates because people repeat it without actually trying it.
In fact, I used to think exactly like you when I originally set up my personal mail server and opted for the hybrid setup you suggest.
However, when the third party service I used to send mail shut down a couple of years ago, I quiclky set up outbound mail on my own server. None of my emails are considered as spam, and I haven't touched the mail configuration since.
> None of my emails are considered as spam, and I haven't touched the mail configuration since.
Sounds like you never actually tried to measure the delivery rates of your outgoing email. No email server is actually able to get 100% of its outgoing mail past spam filters.
I've ran my own email server for a few years and I can confirm that the article is making false claims. It's incredibly difficult to deliver email as a low volume sender. The article falsely claims that it's not difficult and that people simply haven't tried it. Well guess what, I did try it - for years - with countless hours sank into it.
I wrote a blog post documenting my numerous, unsuccessful efforts to get my email delivered. I wish people would stop spreading these baseless falsehoods.
> Why do you think that? The article mention exactly this could just be a myth that perpetuates because people repeat it without actually trying it.
Not sure about OP, but many people have tried it and it is a PITA. Even once you have everything setup right (SPF, DKIM, etc.), you still have to deal with IP issues. I ran my own mail server for years (actually, still do for incoming) and it worked fine for about 10 years until I needed to switch providers and got a new IP. Then I found I had to deal with an IP with no reputation and IP whitelist/blacklists. The deal breaker was finally dealing with blacklists that you had to pay to keep off of.
UCEPROTECT was the one at the time. They would randomly add my provider's entire IP space to one of their lists and during those times all email to a couple ISPs would bounce and the only way to get my IP whitelisted from those blanket bans was to pay $$. Switched to using an external provider for outgoing mail and haven't had a problem since as they take care of that crap.
UCEPROTECT was the pits and its operators were unprincipled lowlifes. Luckily they (like all the pay-to-delist BLs) never got more than minimal traction, so ignoring them and their BS mostly worked ('course you needed a /19 to be effective at doing that)
You use past tense... why? They still exist and seem to be continuing their blacklist/blackmail BS. The ISP in question was AT&T and their subsidiaries (I mostly got hit sending to bellsouth.net), which I wouldn't consider to be that small of an ISP and was enough that I eventually gave up.
Past (more than 5 years ago as I reckon) is when I locked horns with them. They may have switched policies, I may have gotten smarter than them, who knows.
I was the ISP, & they had BL'd one of my servers for delivering NDRs. They used to blacklist /16 netblocks wholesale for this kind of, "sins", and demand ransom for getting out.
No - the article contradicts itself. The author says it's "a myth" and "just set up spf/dkim", and then later says "This is already happening with one specifically requiring mails to be sent from another Big Mailer Corp to hit the inbox, or requiring that senders be added to the contacts for others. Any other sender will hit spambox unconditionnally for a while before being eventually upgraded to inbox."
Which is it? Those of us who have run mail servers know that you get unconditionally spammed all the time.
Not GP, but it matches my experience: with reverse DNS, DKIM, SPF, being in DNSWL (and not in DNSBLs) some of my messages end up in Gmail spam folders (very recently received a reply to a message I sent 3 years ago because of that, for instance). A while ago there were strange issues with Google-hosted mailing lists as well, with some messages not being visible even to an ML administrator, while Gmail servers accepted mail just fine (maybe it still happens, I'm just not writing to those). I hear similar stories from other mail server administrators too.
I think it's more of a Gmail/BigMailerCorp issue, and still using a private mail server myself, but I don't think it's fair to say that it's all easy and works flawlessly, even with strangely behaving mail servers on the other end.
I recently checked my Trash folder in Gmail. I had emails in there that were several years old, right below the text that says Gmail deletes emails in the Trash folder older than 30 days.
Indeed, a quick search suggests that spam is automatically removed every 30 days there. The wording was "Google hid it from me", so either I have wrongly assumed that it's about the spam folder, or it is possible to disable autoremoval.
Edit: or, as the sibling comment suggests, it behaves strangely with respect to removal as well.
Or user accidentally archived it. “X did Y” is often a user’s way of saying “I accidentally made X do Y” without having to accept blame. It’s the “dog ate my homework” of excuses.
I think that because I tried it and my outgoing emails weren't being delivered.
Big mail companies have blacklisted entire C-blocks of IP addresses at VPS providers, because parts of those blocks have been used to send spam in the past, and you won't know until you actually check with people you are sending mail to, to ask if they've received them. It sucks.
New domains and new IPs are penalized heavily by most services' spam filters.
My guess is that a lot of people try it, have poor delivery rates, and abandon the effort before they have enough time to develop decent domain & IP reputations.
I've been running my own personal mail servers since 1995, on my current IP addresses since ~2016. If I add a new domain to my configuration, it takes 1-2 weeks before Gmail will reliably accept mail from it. Once I wait that out, delivery is mostly flawless.
As a person who does run a number of (smallish) mail servers, some outgoing mail does get classified as SPAM and I have to fix it. It's very rare - every 3 months or so maybe, but it does happen. (Possibly the most irritating thing about this is they won't tell you what emails made them think you are sending spam, lets alone why they thing it's spam which makes root cause analysis damned near impossible.) The reverse happens too - we occasionally block non-spam emails.
But, and it's a damned big but, AFAICT, my email servers make less of those mistakes than the big email providers like Google or Microsoft, so the overall reliability of my home grow system is higher. It's got to the point that when someone complains they didn't get an email I got, my first question is "do you use gmail?".
The other big but it is it complex. It isn't just a single server - they are a network of them that distribute email across a geographically dispersed organisation. Despite being distributed it archives copies of all email that passes through it (as required by law here). There are some 2.5K of configuration lines driving it. So yes, it requires some effort to set it up. However, like your case those 2.5K lines haven't changed overly in over a decade.
The flexibility the system provides us in the way we handle email is invaluable. As a consequence of that flexibility a lot of emails are never seen by humans - they routed to a place they can be automatically processed.
Personally I wouldn't say email particularly hard, or particularly easy. It seems no more difficult than all the other things you have to do to manage a cluster of servers. The rewards for getting do it inhouse are pretty big, because just like everything else a programmer or sysadmin has control of, over time it will be scripted so heavily it will become almost invisible.
I tried it a couple years ago. Delivering to large providers wasn't the problem. The problem was smaller ISP, one of them flat-out banned all IPs from my host provider (Digital Ocean), others for no reason I could find banned my specific IP. (I was not on any public block list BTW)
Getting off their block list was either impossible or involved digging some old forum posts to find an email address that you could try your luck with, usually with no result as it seems nobody was reading it anyway.
I work in the industry and it's no myth. Even using expensive relay servers doesn't guarantee the big email providers will deliver your or mail or care much to explain why it isn't being delivered. Big email won't even respond to their own customer complaints that they aren't receiving email.
I would also recommend setting up a dmarc record so you can monitor your deliverability.
Spam however is still a hassle and you're never going to match Gmail or Outlook when you're small. Say it takes a few hundred identical spammy emails to trigger a filter. For Gmail it means the chance of receiving that spam is less than 1 in a million. If you're small and managing say a 1000 users - a large percentage will end up with receiving the spam.
You can't actually measure deliverability with dmarc reports. They will only tell you the portion of emails that are bounced, but they don't differentiate between email in inbox and email in spam folder.
SPF and DKIM are best practices to get delivered to the inbox and DMARC reports are the richest source of aggregate email authentication performance results.
However, you are correct, DMARC reports will not provide the disposition of an email getting delivered to the inbox or spam folder. The report is generated based upon the transaction at the MTA level before the message is handed off for delivery.
In essence this is one of the principles, the thing is the emails don't need to be perfectly identical, just identical-ish with the identicalish matching implementation being not public (as this would immediately lead to spammers adding just enough randomness to whatever they are sending)
The article handwaves this by pointing out that it’s “proof-of-work”. Which is a fancy way of saying “it requires a bunch of work, continuously as the rules change over time”.
Which is why I’d consider it “hard” to run my own mail server. The complexity of running a daemon with a config file isn’t the issue, nor is it that any individual task of SPF/DKIM/etc is complex. It’s that I generally want to put 0% of my time into making sure my emails are being received, and I’m receiving other peoples’ emails.
In my experience of over 15 years, the "rules" have only changed 3 times since I started. First to require good reverse DNS, then to require SPF, last to require DKIM. The transition on each of those was generally several years with plenty of publicity.
That hardly covers it. The problem is that who "you" are, your identity as a sender, is not entirely within your control. The big mail operators figure in a ton of stuff, like the reputation of your subnet - /24 maybe, or /96, it's an approximation - or the reputation of your AS, your domain registrar, all kinds of stuff. If the guy on the next IP over is sending spam that's going to pollute your reputation.
If you have your own ASN and your information is registered with reputable parties in lawful countries, you might not have much to worry about. If you are using a cheap cloud VPS service whose network is registered to a Romanian holding company, nobody will deliver your mail.
It would be nice if these big mail services accepted actual proof of work as a way to guarantee that your mail gets through. Maybe someone could extend SMTP with a bitcoin-like challenge where the server sends the client a nonce and a difficulty factor, and the client has to reply with a suffix that you can append to the nonce so that the whole string hashes to something with the corresponding number of zeroes at the end. People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably, especially if you increase the difficulty factor based on how many emails you've sent in the last hour.
> People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably
The problem with this idea is that spammers don't use their own computer, they borrow other people's computers to send the spam. It also heavily penalizes people who legitimately need to send bulk email.
One of the big corner cases I've noticed is transaction-oriented email-- the password resets, "welcome to your account" emails, order details, shipping notifications. People want these, but they're probably harder to deliver than personal email or much spammier bulk mail sent directly via outsourced experts.
These have a lot of iffy issues.
* Since they're going to be generated by the ecommerce service, it may not be running on the same server or IP block as your main mail infrastructure, so you might have to specially configure around that.
* The content-- hundreds of messages a day, at seemingly scattershot recipients and highly templated-- screams spammy nto the wrong algorithm.
Yeah, there's services like Mandrill, but the whole ecosystem feels broken. It's not serving mailers. It's not serving the recipients very well. It seems designed to please a small, delicate, vocal audience. The people so upset at the prospect of false negatives (spam in the inbox) that they rally around providers who are prone to aggressive false positives. Who are these people and what makes them tick?
I could see this backlash in a situation where it's high risk or cost, but users can manage their own filters in any half-decent mail system, and frankly, clicking through a few pieces of spam a week is not a big deal-- arguably far less a hassle than spending an hour on the phone to get a tracking number that a spam filer ate.
(Note: deleted and moved here because I put it in reply to the wrong comment)
I work for an email marketing provider (opinions my own, obviously), and I was extremely surprised how many people genuinely want to receive lots of commercial bulk marketing email. And by "want to" I don't mean "don't mark as spam", I mean "continually sign up for different marketing newsletters, and consistently use links in newsletters/marketing emails to make purchases from multiple companies".
It's an alien phenomenon to me, and it may be a minority of bulk email, but it definitely is useful to a lot of people. Maybe it's their equivalent of coupon-clipping-based shopping, or maybe they just like learning about things to buy in that format? I have no idea.
That aside, the last thing an email marketing platform wants to do is send spam. Every time someone marks an email such a provider sends as spam, the reputation of their sending addresses is damaged, and therefore their ability to deliver mail to aforementioned people who want to receive them (and therefore the ability of the marketing platform to make money by facilitating that kind of desired email) is jeopardized.
TL;dr there's bulk marketing email and then there's spam, and surprisingly little overlap (and overlapping incentives) between the people who send each.
Sure, I do this. A lot of stuff people buy is often a lot cheaper with promotions. Just the other day, I bought a $600 set of tires for $300-380 (depending on whether or not I remember to do the mail in rebate). That alone is well worth the few clicks it took to filter mails from that list out of my inbox and into a label for car-related marketing.
I have a couple of broad categories for these filters like car stuff, computer stuff, other-hobby-related-stuff and of course clothing, and just take a look at the appropriate one when I need or want something. Takes virtually no time, and saves a lot of money.
Of course I'm not gonna lie, I do end up buying things that I wouldn't have bought otherwise by doing this. But some of those things are pretty cool, too. :) I do have a few more cheap ARM boards of various types than I can probably use, though.
The paper's argument assumes that the PoW scheme is required though. It could be an option which just guarantees that individual mail can get through, while still keeping the usual reputation-based spam filtering for clients which don't want to participate.
If there's one lesson from the history of spamming, it's that spammers are among the fastest adopters of anti-spam technologies. If PoW meant that spammers could only send 1/100th the email, but the email was 1000× more likely to actually show up in an inbox, spammers would jump on it in a heartbeat.
The value to spammers isn't in email delivered, it's the actual user engagement at the end of the process. A million messages delivered to the inbox is worth far more than a billion messages delivered to the spam folder.
A million messages delivered to the right inboxes are valuable.
The quantity-above-quality model in the spam industry probably lead to low-quality or vague list criteria. Broad demographic groups and domains. A lot of third party data of questionable legitimacy. If you've been running the list a long time, some "email address foo@bar.com clicked on campaigns A and B" data. You might be able to say on a large scale "List A is likely to convert better than B", but it still comes down to spray-and-pray at the individual address level.
If you went to a spammer with a few "sucker lists" of 100k emails each, and told him "you can only send 1k per day", will he have the data to triage his list to get a decent return on it anymore?
This would also snowball over time. Without being able to do high-volume campaigns initially, they'd have a harder time building up the knowledge they can use to manage their lists, and the quality would decline over time.
Yeah, but if the marginal cost of the next message is higher than the marginal expected increase in revenue then they won’t. It doesn’t matter optimizing inbox delivery rates if per-delivery you lose money.
Keep increasing the PoW cost and think about where that line of argument leads you. "One message delivered to the inbox is worth far more than a billion messages delivered to the spam folder." One single delivered message is going to be profitable for the spammer? Clearly not. This proves the existence of some break-even point at a lower cost. Mail providers just need to set the cost above this point.
Given that spammers mostly use others' computers to blast their email out, I suspect that the price point that would truly throttle spam would be higher than the price legitimate users of email would be willing to pay. I'm not sufficiently knowledgeable in spam and black market economics to throw any actual numbers around.
It's an interesting idea, but wouldn't this be easy to abuse though, like Gmail could force Fastmail to do huge proofs of work before accepting their mail?
> Which is a fancy way of saying “it requires a bunch of work, continuously as the rules change over time”.
Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules. No need for any third-party handling your mail.
>Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules.
Some of the "rules" for reliable outgoing email delivery cannot be encoded inside the source code files of a github repo, or email setup bash script, or a Docker image, or a EC2 virtual image, etc.
An example of an unspecified "rule" outside the boundaries of a local machine holding the email server is recovering from an unexpected blacklisting of ip addresses. Example.[1] A preconfigured Docker image of Dovecot isn't going to magically ask Amazon support staff why the ip address is blacklisted and/or move the server to a different ISP etc. If a bad actor (that you are unaware of and have no control over) happens to share your ip address block and sends out spam which then causes Gmail/Hotmail/Yahoo to reject your server's emails, there's no open-source programming code that can detect and fix those problems happening outside of your control.
Or put another way, if HonestJoeBlow could download a constantly updated Docker image that has the so-called "correct rules" for sent emails to always be accepted by Gmail, it would mean the DishonestSpammers could also download that same Docker image to get their emails delivered.
The issue is that "trust" in the email ecosystem is an emergent property among participants and therefore, it can't all be embedded inside email server configs, or in a DIY blog article trying to explain the exact 12 steps to make email delivery work perfectly with all receivers. Eventually, something can break (because a receiver changes their idea of "trust" and "spam") and it requires troubleshooting & debugging to get email delivery working again.
A tldr would be: You can't put "sender reputation" into a downloadable software package.
This would allow spammers to evade the rules all the same. For anything that is standardized or agreed upon, we can of course have packages like this. Time zone data (tzdata), IP Geolocation databases, etc.
This is true, I run a small mailserver for myself, friends and family. I eventually gave up trying to get mail delivered to the major mail services and now just send postfix mail through Amazon Simple Email Service.
Email is hard across time. Meaning yes, I can setup a mail server, and yes I can configure DKIM+SPF, and yes I can initially get good delivery to all the major providers. But that's just the beginning. Email is an ongoing maintenance responsibility. And problems with mail delivery are almost always an emergency for yourself or your clients.
Nothing quite matches the gut punch of being on time for a deadline then having to push it back to troubleshoot mail problems. Or losing a weekend because while you know enough to get started your experience isn't deep enough to fix deliverability problems in a timely manner.
> I work on an opensource SMTP server. I build both opensource and proprietary solutions related to mail. I will likely open a commercial mail service next year.
Translation:
"I am an intelligent guy who works hard. My entire professional career is devoted to designing and maintaining mail systems. I work full time to achieve mastery of my craft. Yet I can't understand why people outside my specialization might find it difficult to do my job."
Email _is_ hard. That's why I'm very happy to outsource mail to experts like OP.
I don't agree with you. I run many, many small email setups and some rather large ones. I am based in the UK.
The only thing that gets mail dropped big style these days, in my experience, is the saddest one: Reverse DNS entries, ie not getting PTR registered. This anachronism still seems to be a fixation by everyone.
Content checkers seem to be pretty decent these days. My fav. is rspamd nowadays with Exim on the front. I generally wind back from doing anything fancy and leave it to defaults. Even more bizarrely, I do add a domain blacklist for all return addresses. As you know email addresses are easily fakeable but the semi legit spammers do stick to a list of domains and hence can be dropped with a few rules. They will go away after a while.
That's actually kind of the problem. It's the same thing with phone systems. Any IT person worth a shit can follow a decent tutorial and end up with a system that works. Until it doesn't.
What happens when it doesn't is the issue. For a lot of businesses this will be an immediate critical problem. This is not a great time to be learning by experimentation.
If these services are not critical then go nuts, but if your business depends on these services you really want to have someone whose full time job it is to run them. If your business is not large enough to support having a person's primary role be administering these systems you should really have a third party vendor do it for you.
The article is right, you don't have to use one of the big names, but most of the time it's not a good idea to run your own if you consider email to be a critical service.
Let's see... Postfix, Dovecot, SPF and OpenDKIM on the server-side, each of them with their own config.
Then you've got to set up your domain, and domain headers on your domain host. Oh, DMARC is also another thing.
Then, most ISPs will outright refuse to accept incoming mail from your IP address, since they've basically changed from blacklisting to whitelisting. So you've also got to relay your outgoing mail via your domain host.
And then spam rules. I took the recommended rules from the Debian/Postfix/something-something-sorbs.net website, and I rarely receive email from e.g. eBay, because they've been marked as sending spam. Often happens with gmail addresses, too.
Despite all this... I still run my own mail server, but hotdamn, you're calling this not hard?
EDIT: Oh, and nowdays you've also got to entangle your TLS certificates into the whole process somehow. I managed it, but don't ask me how, I'd need to read up on that.
And Postifx is not just one thing, it's: smtp, smtps, submission, relay etc... And Dovecot: pop3, pop3s, imap, imaps, lmtp, passdb, userdb, sql, ldap, seive etc.
Look, I don't want Google changing this tomorrow because to something tied to and controlled by them because they don't do evel, isn't faster or people like it, but please, don't say it's not hard because you will be misleading people for sure
Edit: to be crystal clear, I agree with parent but don't agree with op
And then there's also the password/username database, that defaults to tying into the Linux pass/user database, so if you want to set up some custom login-data... ugh, I don't even want to think about it.
I set it up once, maintain it here and then, and am keeping it, because dismantling the system is more work than maintaining it presently, but seriously... with the above setup, do it only for educational purposes.
...I'm guessing I spent a week or so properly setting everything up.
...but I'm also willing to try a universal, one-size-fits-all program, if anyone knows anything for Debian?
I used to run postfix/ssmtp/etc on Gentoo. From time to time, when updating the single components, something stopped working and then I had to quickly debug and find a fix. In the end it was too much time-consuming for me to maintain.
I then decided to search for a big-monolithic-do-it-all-solution (from a SW-perspective) and the only option I found was Xeams ( https://www.xeams.com/Xeams.htm ) - I didn't find anything open-source. I'm using it (the free variant) since 2017 and it works.
Runs on Java, provides all services (smtp, pop3, imap), has greylisting, has embedded user administration, users can have email-aliases, has a web-admin-UI, 1-click-sw-updates, etc... .
I run my own mail server and as long as you use TLS and the one that’s really easy (I can’t remember if it’s DKim or Dmark) most domains accept mail from you.
I really didn’t have a hard time setting mine up and I’ve done it multiple times. Dovecot is a bit of a pain if you really want imap... but overall none of it is harder than most other things.
And this is precisely why I don’t think it’s worth it. I don’t want to have to worry that “most” domains will accept my mail, I want a big email service that has the clout to force domains to accept my mail. I shouldn’t have to solve the mystery of why my mail was rejected (or rather, why some provider thinks I’m sending spam). That’s why I do Fastmail.
What do you use for automated messages (ie, login emails, password resets)?
I asked Fastmail support, and they basically said, "we're not really built for that." However, most of the other services I looked at had arduous terms (arbitration agreements), and even they were talking about delivery rates in ranges of 85-95%.
My takeaway has been that as an app creator, email is really unreliable, and I should try whenever possible not to make it a critical part of anything I build. If 5-10% of the people using your service can't get a password reset email, what do you do?
As someone without much experience with email of that nature, it doesn't seem like the major services actually solve the deliverability problem. If they have Google-style support where I can't get one of them on the phone if my emails aren't sending, then I might even be worse off using them rather than my own setup that I can at least debug and experiment with.
I run my primary mailserver on DigitalOcean.
It seems to work fine. I think you have to have either your account, or the Droplet, active for 30+ days before they'll let you have port 25 open.
Trying to avoid the landmines of blacklisting is really the biggest peril in all of it. I refuse to move a company I support away from GSuite to self-hosted, because I have a high level of confidence that just one person sending a lot of mail to another person outside the company could result in our domain ending up in RBLs, which is very difficult to reverse. Running a mail server in 2019 is not something I need the headache of.
And RBLs aren't that great. I visited this in January of 2018 [1] and in my case (where I do greylisting [2][3]but nothing else for spam) it wouldn't help.
[3] And yes, I do run my own server. It's just for me, so I have no issues with "customer requests" and I've been using the same IP address for about a decade (maybe more). Then again, I've been running my own email sever since 1998.
https://mailinabox.email/ does everything and more. Of course, this doesn't make it easy. You still need to do more things than were you to simply use protonmail/fastmail/etc.
Managed mail server clusters for years with an ISP. Agreed, too many things that can break and cause PITA.
However, ‘Hard’ is a subjective term.
The deeper you are in a trade or the longer you have done it, the easier it comes to feel.
I visited a family farm and found it very very hard to squeeze milk outta buffalo.
My great uncle however has dealt with that buffalo that for years and didn’t sweat it one bit.
In a similar vein, do I really want to tend a buffalo in my backyard, when I can get the milk I need from a supermarket?
However, ‘Hard’ is a subjective term. The deeper you are in a trade or the longer you have done it, the easier it comes to feel.
I agree with this. I don't think mail is particularly harder than other serious Linux and network system administration tasks. But it does require those sysadmin skills, and it's not something I would recommend to a typical software developer unless they were interested in broadening their sysadmin skills or had a particular need that could benefit from a self-hosted mail server.
I've been operating mail servers continuously since 1992, so I guess that's the buffalo I'm milking.
Yak shaving refers to a task, that leads you to perform another related task and so on, and so on — all distracting you from your original goal.
Buffalo milking seems to be variously related to experience (see effortless squeezing) or Buffalo as a service ie. no buffalo on-prem or a backyard required.
This analogy applies to everything and the answer is always the same: if we centralize then we are vulnerable to tyranny. It seems inevitable that we do this though. Human history is a cycle of people investing more and more trust in an authority, then being abused by that authority, followed by a revolt and the installation of a new authority.
Q. Why might a single agent stuck in a sub-optimal Nash equilibrium, along with 100s of millions of others, not make a different choice that might cost them more?
A. Ideology. If the agent wants a different global equilibrium, they might decide that them choosing differently, (1) calms their conscience and/or (2) actually make a material different in the choices of others.
Yes, it is a similar result, but not the same. You give up a lot of control even using a smaller email service provider -- not the least of which is direct control of your own email data. Obviously, since it is email, there is nothing you can do about what others do with emails once they are received on their end, but it is still nice to have direct control of your side.
The counterpoint, however, remains valid. There is some work involved in maintaining your own mail server. If you use a provider, you don't have to deal with any of that -- it's always about tradeoffs. As the article says, however, maintaining your own mail server isn't as hard as a lot of people make it out to be. It takes a little of your time, but once you know the few things you need to do, I find it to not be a big deal at all.
Setting up offlineimap to backup your mail is easy. I run daily backups of my protonmail account so in the even they decide to do something nasty to my account, I wouldn't loose more than a day of mail.
True, and I use offlineimap for a different scenario where I do not have control of the mail server. However, it is still not the same level of control as managing the mail server yourself. I understand why some people don't want to do that, but again, I don't find it to be much trouble at all.
If you're living at the country side, producing your own milk is healthier, can be a good hobby while you're young and a great occupation after you're retired. Manual labor is good exercise and can be fulfilling.
But I'd probably go with goats instead.
I don't think maintaining your own email server can be very fulfilling, but hobbies are a matter of taste.
Growing up in communism, under the Iron Curtain, we weren't finding too much milk in the eighties at the store, so our source for dairy was the country side, friends, family, grandparents, so I grew up with raw milk.
Raw milk from grass fed animals has a very different, much richer taste.
For one you can control what you feed your animals. The taste and quality of milk varies greatly depending on what they eat. You can feed them high quality grass, although in winter some cereals are fine. Anybody that drinks raw milk knows that the taste of what they eat really is reflected in the milk. E.g. if you let them graze on a field with flowers, it will have a flowery taste.
Leave raw milk on the table to turn sour and you get yogurt. It's quite good too. And you can't do that with the pasteurized milk from the store, it doesn't matter if it's whole or not, doesn't work.
Goes without saying that with high quality raw milk it's quite easy to make cheese too, which again, you can't with the milk your can find at the store.
As for why it's more healthy, the pasteurization process reduces the nutritional quality of milk. There are also weak indications that people with a lactose intolerance can tolerate raw milk better than they can tolerate pasteurized milk. It might be that it contains some lactase. Although this claim you should take with a grain of salt.
And in the US at least you can argue that the pasteurized milk you find in stores is ultra-processed. Read the following article:
Note that I'm not saying that you can't find a good source of raw milk from a farm with free range, grass fed animals and great quality control. But it's harder to do so and if you're living in the city, depending on the city, next to impossible.
I do encourage you to try find such a source. The difference in taste is well worth it.
And if you grow it yourself, it's much like anything else. For example tomatoes no longer taste well due to being picked too early. Grow your own tomatoes in season and the difference is night and day.
> I do encourage you to try find such a source. The difference in taste is well worth it.
I'm in Europe.
> the pasteurization process reduces the nutritional quality of milk
It doesn't reduce the nutritional quality substantially though does it?
Europe Food Safety Authority advises to not drink raw unpasteurised milk due to potential health risks (http://www.efsa.europa.eu/en/efsajournal/pub/3940.htm) echoing what many national countries have advised "at a minimum, boil the milk before drinking it to kill potentially harmful bacteria"
Any perceived nutritional gain is marginal and is far outweighed by risk of illness.
There's little evidence that raw milk has higher "nutritional quality". Taste you can make an argument for, and raw milk is definitely required for yogurt & cheese if you don't want to introduce your own cultures.
I'd also note that "pasteurization" can mean different things in different countries. In Europe it's usually more extreme than in the U.S., as the milk is heated ~60 °C higher for a shorter period.
Number of times I have had the problem of "email not working for some reason" since switching to gmail ~20 years ago: zero.
Number of times I had this problem when running my own mail server (and presumably would have today): non-zero.
There's your entire explanation as to why this is a losing battle. Just one incident of an email delivery problem probably outweighs any privacy risk wrt using a centralized provider wrt email. That prior could change if there is a privacy related incident with these providers, but so far their track record has been good.
> Number of times I have had the problem of "email not working for some reason" since switching to gmail ~20 years ago: zero.
This is incorrect. By using Gmail you've lost several legitimate incoming emails due to their horrendous spam filtering. It's just hard to be aware of the emails you've never seen, so you think nothing ever failed.
In addition, if you are relying on @gmail addresses, you run the catastrophic risk of permanently losing access to your email with no-one to call for help. There's so many instances of this happening that you can't even call them isolated instances. All it takes is for Google's ML algorithms to think that you have violated their T&C somehow. And poof! Everything is gone.
It’s true, but it is only half the story. I’ve used a paid provider other than gmail for 18 years. I occasionally had problems starting a couple of years ago, and every single one of them was caused by google not delivering a mail (or marking it spam, which is for almost all practical purposes, not delivering it).
I eventually caved in and switched to fastmail - and everything is good since then.
Google is giving you no trouble because the main instigator of trouble these days is google.
You may find it acceptable to use them with this in mind. I do not.
My point was that running your own server isn't about just getting Hello World working, its about the fact that you basically are now living in a world where you will potentially have delivery failures of your email, or incidental things you need to fix at random. Imagine if this is how your phone worked: on a given day, you may find yourself suddenly having to learn about why your phone calls stopped going through, or in general, 0.01% of the time people can't reach you with no explainable reason. The idea I could wake up on a random Tuesday and find myself debugging ancient erlang code to diagnose my phone calls regressing feels no less disastrous than having to read the source code to my email server if/when it mysteriously stops working.
In any case, this is a argument as to why it's an uphill battle to convince people that running your own server is an easy endeavor. (Which seemed to be the point of this post.) It says nothing about switching off of gmail to another provider as you did, or if the actual burden of running a server is worth the privacy trade-off for an individual (i'd argue for most it isn't, but wasn't really the point i was trying to make.)
Switching from Google to another 3rd party provider doesn't eliminate privacy risk, it shifts around the probability distribution for certain kinds of events to occur. (Arguably, so does hosting your own server, but hosting your own server does rule out large classes of failures.)
Not once. Tbf, I am considering the herculean task of getting off of gmail, but if I were to go with my own server the burden has basically nothing to do with setting up the email server (easy) as this post might suggest, but instead the burden and latent risk involved of having to be personally responsible for operating, maintaining, and being exposed to externialities (like 0days, being blocked by filters, server outages, etc.) Setting up services is trivial compared to the aggregate cost in time, money, and context-switching tax over time of operating them.
That's the part that I really don't understand. If the recipient hired someone to throw away their emails, why would that possibly be my job to rectify? I sent them a legitimate email, they hired someone who promised them to throw away spam, but obviously is incompetent at doing so, so it's up to them to hire someone else!?
I probably would have avoided ever getting dependent on them to that degree in the first place.
But, yeah, sure, I see the point--but my impression is that people automatically see it as their responsibility to figure out why someone else's email provider is throwing away their emails, regardless how much you depend on them.
The "host-your-own-email-server" being "hard" vs "not hard" is an unresolvable discussion because everybody has a different mental threshold of "hard".
I'm still amused that back in October 2017, a commenter (lucb1e) argued[1] that I was exaggerating the difficulties of reliably sending email but a year later in 2019, he confirmed the same difficulties![2]
The discussions in that thread (May 2019) and today's HN thread (September 2019) contradicts author's claim that mail stopped being "hard" 10+ years ago:
>Another reason is because it used to be hard a long time ago. [...], citing the very real difficulties they faced over a decade ago,
Hosting your own email is certainly not impossible and lots of people are successfully doing it. (Or they they're actually not successful because their outgoing emails are sometimes getting silently spamholed without them realizing it.) In my case, I'd rather expend mental energy on other pursuits (machine learning, learning Swift language, etc) than constantly worrying about a personal email server's "reputation health" and then debugging whatever goes wrong.
>There are also different sets of features people want, some are actually pretty difficult to set up.
That's true but I think if we analyze discussions (see this thread and old threads I cited) of one group of HN experts (email is "not hard") debating other HN experts (email is "hard"), the most contested issue isn't the difficulty of advanced SMTP features and enhancements -- it's about outgoing sent emails being reliably accepted.
It's not 'hard', it's 'a gigantic pain in the ass' which is (in normal word use) one form of 'hard'. Which is why most sane people stopped doing it themselves say 15 years ago. Especially for the few cents you can get professional 3rd party email hosting for.
Edit: I also very much like the disingenuity of articles saying 'X isnt' hard! Just execute these 20 commands!' Duh, executing those commands isn't the problem, it's knowing which commands to execute, and the pain of having to figure out the new commands to execute 4 months later when for some reason something broke and now 20 people are twiddling their thumbs until you fix it. Screw that, running email servers is a horrible tedious pain in the ass you should almost never do yourself unless it's the sort of thing you enjoy (I've run Postfix, Qmail and Exchange servers from the mid 1990's until mid 2000's).
I totally agree with this and this is actually something that my Engineer Brain and Product Brain are constantly in conflict about. I ran gitignore.io for 6 years and it was not "hard" to run. It was two commands on a web server. The problem was when I was on vacation or at work and AWS/Heroku crashed and I got flooded with GitHub issues/Tweets telling me the service as down[1][2]. All of a sudden, I have to go take a look.
Martin Fowler said something in a talk a few years ago[3] and I think it speaks to this exact mindset. This post is lacking any economic reasons for people to run their own mail server — tell me why me running my own mail server saves me time and/or money.
As another tangent, I'm actually looking at running my own mail server. For me, it's privacy which coincidentally is not even mentioned once in the article.
There's also a huge difference between setting up a properly configured server and actually getting things delivered. The big ones (MS, Google) mark your mail as spam even with SPF, DMARC, DKIM and the like. Even on the service I use (purelymail.com) the low volume of mail and the fact that it's less than a year old means that even when I email them first, the replies are sometimes marked as spam.
Running my own for about 2 years now. Spf dMarc and dkim set it is accepted by all mail providers. I also get daily reports from Google how well it behaves
Same here, except to Charter, who has apparently just blacklisted swaths of DigitalOcean IPs, probably for some reason. So I just can't email my mom. Other than that, a great experience.
Well I just have to do the one more step of forwarding the SMTP to sendgrid or whatever and then it will be top notch. It hasn't been enough of a pain for me yet to do that next step. I just signal her instead of email.
> Running my own for about 2 years now. Spf dMarc and dkim set it is accepted by all mail providers. I also get daily reports from Google how well it behaves
This is false. The dmarc reports from Google do not actually give you the portion of email that is placed in recipients' spam folder.
The big ones (MS, Google) mark your mail as spam even with SPF, DMARC, DKIM and the like.
This is such a canard. I have almost zero problems delivering to anybody from my closet server, and the only thing that has happened in the last several years was getting notices that AT&T was blocking me. I did a quick relay check and emailed postmaster@. It only took a day or two to get an email saying I was clean to them again, and that was without the DNS security extensions you mention.
Maybe cable modem users are held to a different standard? Dynamic IP addresses? I have DSL with static IPs.
The article points it out, but it used to be harder than it is now.
These days (for a long time now, really), it's hard to set one up that's, say, an open relay by accident. The default configs are pretty much all you need, just replacing a couple of values with those that are custom. Basically, what you're saying is exactly what they're debunking. Also, the article isn't "just execute these 20 commands", it's deliberately non-technical.
I've been running a tiny Postfix mail server since ... 2000 or so, still doing so today. It had grown in to a bit of a monster configuration over 17 years or so, complete with a custom PHP interface to allow me to add user accounts, etc. I learnt a lot doing that, but a couple of years ago I started it all from scratch, with a default config and writing the config into my ansible setup so I don't have to remember it. It was very easy to do. Since then, I've touched that configuration once.
One thing I will mention because I haven't seen it around enough, and didn't know of it until someone said it in passing: if you're getting smap-trapped by gmail a lot, turn on TLS support for outgoing. Made a lot of difference. For postfix that's just adding:
So despite your claim that it's 'very easy to do', in the very next paragraph you are basically saying that you need to be around people who 'mention in passing' obscure details of email server configuration, otherwise it will 'make a lot of difference'? I mean it seems you disagree with me at first, but then in the rest of your post you seem to be making my point for me, so...
"OK, then why is everyone saying it’s hard ?
[...]
Another reason is because it used to be hard a long time ago. People got traumatized by how hard it was to not screw up and never reevaluated the situation. Some people today genuinely discourage other people from running their mail server, citing the very real difficulties they faced over a decade ago, far before some of today’s tools even existed."
How does running my own mail server save me time, money, or give me capabilities that I don’t have by paying someone else? What other criteria should I use?
When you eat home grown vegetables, the experience isn’t objectively worse. When you run your own mail server it is.
If you were running your own restaurant and your home grown vegetables caused a worse customer experience would you do it? Running your own mail server which causes more of your mail to go to spam is worse than using a commercial provider.
"When you eat home grown vegetables, the experience isn’t objectively worse. When you run your own mail server it is."
^ I both run my own mail server and use someone else's service for another purpose. My experience running my own mail server is objectively WAY BETTER than using the commercial provider (and I've been doing this for many years in multiple scenarios).
Just because your experience (even if it is repeated) with certain technology goes one way, that doesn't make it universally true.
It’s not “just my experience”. It’s practically a universal experience that the chance of emails sent from your mail server will be blocked or go into a spam folder a lot more frequently than using a commercial provider.
Of all the questions I have to answer on technology decisions, why would I want to answer why did I choose to run my own bespoke mail server instead of spending a few dollars and making it someone else’s problem and so metaphorically chose IBM - ie “No one ever got fired for buying IBM”.
> - Mail is not hard: people keep repeating that because they read it, not because they tried it
No. Been there, done that. It was a major PITA. It's not even something I would consider again for small organizations as it's an economic nonsense too.
Amen. I ran my own personal mail server for ~15 years (started with qmail+squirrelmail, moved to postfix+dovecot+opendkim+zpush+roundcube at some point). I enjoyed it but eventually just didn't have the time to care anymore.
Moved everything over to Fastmail about a year ago and haven't looked back. My inner geek still feels bad about not doing it myself but my rational brain is happy.
This. Even running your own IRC, while not "hard", is a PITA enough that companies will shell out a chunk of change to have someone else do it for them. And, they may even get better features as a result.
Eh, it used to be pretty bad but the other year the people from the openbsd project wrote a new mail server with a trivially easy to understand config file (I wrote mine from scratch just a couple hours after finding it.)
If you’re already basically competent with the OS you’re using and have a machine with a decent connection that can be used for mail then it actually isn’t hard.
The config file format is the least of your worries (unless you're running Sendmail, ofcourse). It's about havimg to know about DKIM/SPF/dns weirdness/ etc, and having to stay up to date; about blacklists and greylists and the social dynamics around it; about mailbox storage and backups and users complaining about quotas and attachment sizes and that sort of shenanigans; about how will I let my users set auto responders and filters etc (just send them a link to the sieve man page! Lol); about having to worry about being called over the holidays because every custom server has its own peculiarities. And his talk about 'it used to be hard but not any more!' - yeah sure, just like the dozens of times people said that over the last decades? There's a 'year of Linux on the desktop' analogy in here somewhere.
Can't tell if you're serious or not - but if you are, just looking at man sieve is enough to understand why no sane person would give an average end user access to that. I mean I used it for many years but I'm a nerd weirdo.
> just looking at man sieve is enough to understand why no sane person would give an average end user access to that
You don't have to force an average user to hand edit Sieve files. You can put the same kind of GUI or web interface on top of a Sieve file that current mail providers put on top of whatever hand-built custom non-standard filtering rules system they currently have. To the end user it would look the same.
But for an end user like me, who already has a huge number of filter rules expressed in Sieve and would like to be able to upload them to an email provider, not supporting the Internet standard format for filter rules is a showstopper. I don't want to have to re-input all my filter rules into an email provider's web interface one by one. I want them to be able to understand the rules I already have in the Internet standard format I already have them.
" You can put the same kind of GUI or web interface on top of a Sieve file"
From a quick google, there does seem to be a Roundcube plugin for managing Sieve filters, but man yet another dependency (or two, as it seems you need a separate daemon as well) in an already brittle (by the time you get to this point) stack...
Look, I understand where you're coming from, and I used Sieve for many years, but it's 2019 - something is not an 'Internet standard format' when the number of users actually using it can be expressed with 4 or 5 digits at best... This horse is not just dead, we're snorting its ground up bones by now. Nobody caters to power users any more. Back when power users made up a sizeable portion of the internet population, yes, but not in 2019...
> something is not an 'Internet standard format' when the number of users actually using it can be expressed with 4 or 5 digits at best
Well, there is no other standard format at all, so basically you're saying we might as well just give up trying to have an internet standard for email filtering. That doesn't seem like a good outcome to me.
Yeah, that's the folk wisdom on sendmail. But really, since version 8.0, writing your own .cf (or reading it) should have been regarded as bizarre as crafting your own object files with an hex editor. And sendmail has served me well, over the years; its scariest feature (sendmail.cf) is also a strength - as a Turing complete programmiong language, it allows you to tie your incoming/outgoing mail in square knots, if you feel so inclined, or to solve some unforeseen mail routing configuration/problem/whatever. Of course, if you find someone that has gone to the lenght to write a _LOCAL ruleset that does it - and alas that is a creaft I do not possess, and is dying out. When it does, I will probably make the postfix jump.
I cannot claim prescience, because I started ont the trade when sendmail was THE mail program (though ed was no longer THE text editor): but consider, had I listened to the sendmail bashing crowd a couple years later I would have gone qmail (!!!). For not having done that, I thank the almighty weekly.
Config files really are not the problem with running email. The "why the hell is provider/BigCo/... XYZ rejecting our emails again" dance is, with the answer typically being "no clue, they certainly won't tell us, let's hope it works in 48 hours?"...
People also have different ideas of what "hard" is. I've never tried setting up a mail server so I'm talking out of my ass a bit, but after reading the article it sounds like you'd need to do a bit of googling/trial and error changes to get things set up, probably one of those projects that would take up most of a saturday or at least a saturday afternoon and might run into some hiccups every few months that requires more googling/tweaking.
Even if I'm way off base about the mail server set up being this way, some people still consider this type of "saturday afternoon" project hard compared to something that just works out of the box without any fiddling and can be set up in 20 minutes. I'm not trying to be elitist by suggesting these people are dumb or lazy, just that the semantics of what "hard" is varies for people, especially with little projects that require fiddling/tweaking/looking for outside help.
> it's 'a gigantic pain in the ass' which is (in normal word use) one form of 'hard'. Which is why most sane people stopped doing it themselves say 15 years ago
I've been running my own mail server since 2000 or so. It isn't hard, it takes almost no time at all, and I have my own independent mail that nobody reads, and I make the decision of which mail to accept.
I will take the liberty of classifying the above comment as FUD.
Incidentally, it is threads like these that make me regret posting anything on HN. Respond to a vaguely worded FUD with a specific first-person example (of doing something for almost 20 years, no less), and get downvoted into negative territory.
In addition to the other comments about what is meant by “hard”, there is a crucial omission in this article: the failure mode of e-mail deliverability can be both hard to detect, and extremely costly. I wouldn’t mind hosting a blog, because worst case scenario I’m offline for a few minutes, pingdom alerts me, and we’re back. Total cost: some lost blog readers.
If I fail my outbound email setup, it can be months before I realise that most of my emails have been getting dropped. Total cost: I depend on email for work and family, so very high.
I used to host my own email. Tested with gmail: delivery was fine. Half a year later it turns out Hotmail was silently dropping my mails. I felt betrayed, and learned my lesson: email is hard.
Which user demographic is running one's own email server ideal for?
My consultancy handles "all things technology" for multiple small businesses (sized 1-50 people, located mostly in South Asia). Anecdotally:
A lot of these folks are happily using mybusinesname.accounts@gmail.com and so-on, but decide to get their own domain for the added veneer of "professionalism".
Many are happy to use Yandex (1000 users with your own domain for free @10GB/inbox. Probably not appealing for some Americans given US-Russia political issues).
Startups/sole entrepreneurs typically come to us after they've purchased a GoDaddy domain and selected the "Business Email" option upon checkout, so they just continue using it.
There's a small sample of companies who are willing to pay for Google Apps because they like Gmail's web UI, and they want the rest of the Apps Suite too.
We also deal with a lot of local big corporations for project work. They're almost always using some kind of in-house corporate email system running Microsoft Exchange Server with a number of different rules like no attachments over 5 MB and other such silliness.
For us to invest time in setting up a mail server for a customer there would have to be a very rare confluence of:
a) I don't want Yandex
b) I don't want to pay for Google Apps / Zoho / other
c) (Optionally) I want to send 500000 marketing/transactional emails per day but I don't want to pay Sendgrid/Mailgun/Mailchimp/SES
To set it up we'd have to establish pricing and SLA. Would I agree that we do it for less than $5/user/month? Nope!
I can't really see a compelling case to do it unless you're a hobbyist with strong philosophical perspectives on this topic. Maybe my point of view is different because of Geography but I doubt things are very different elsewhere in the world as far as cost/effort is concerned.
Lots of businesses run their own email server. It's not that complicated.
The easiest way to do it is to buy an email server, you can use Exchange or you can use something like iMail or Kerio.
Install that on a server, get a static IP from your ISP, put the server behind a firewall, and point your DNS servers for your domain to it. Then generate an SPF record, and you should be good.
That's how many people do it. As a bonus, you don't need to pay $10/user per month for hosted mail, or deal with Exchange licenses, but you will probably want to back the server up from time to time. You even get webmail with it!
The advantage/disadvantage of this setup is that it's now up to you get to get yourself off spam blacklists and such. When BIG_EMAIL_HOST is having problems, there's nothing you can do but wait. But with your own servers, you are the one who gets to email the spam department of BIG_ISP to request removal, and then there's nothing you can do but wait.
If you know Linux, you can setup Postfix, but I did say 'the easy way'.
Setting these things up is not the hard thing. Building reputation, not getting refused because your subnet neighbour decided to start spamming, having to deal with BIG_EMAIL_HOST not accepting your emails, bla bla bla.
I have run my own mail server for years. It absolutely was complicated for me to set up but I loved the process of learning and got it going without too much trouble. Now I can get mail delivered to nearly everyone, except Charter. It's been NBD from a maintenance point of view. Spam is all tuned nicely and I generally don't have to think of it.
E-mail servers are complicated. People with skills and interest can do them, no problem. It's not overly hard. I agree with that.
But I sure don't go recommending it to my friends who wouldn't appreciate spending a few days learning and setting all that up.
What I can do, however, is offer them e-mail accounts! So I guess as long as every group of 50-100 friends has one person willing to run an e-mail server, then we're all good. Of course, it'd probably be a pain to switch when that person dies or loses interest.
Maybe we just need to make it really easy to have a email@myname.tld that is portable and easy to move around.
To this day none of the mail servers I’ve setup can send to verizon.net.
It’s not “hard”, but there are servers who will not accept a single message from you, rDNS, DKIM/SPF be damned. And there’s fuck-all you can do about it.
Been there, self-hosted my email for more than half a decade, gave up on it a few years ago for a couple for reasons:
If you are self-hosting the most important authentication fallback for your online identity, an ill-timed downtime or deliverability problem really hurts.
Your reputation can be affected by neighboring bad actors you have no control over. For example, SpamRATS punishes full subnets without an actual appeals process.
From the point of view of a major provider, the same email from your tiny server is going to have a lower score than one from their system. Encountered this a lot against Gmail.
And it's a commitment. If you go the self-hosted route, you will need need to keep up on new security and filtering practices, even if you use a turnkey solution. You might get bored with this after a while.
(tagline: "Take back control of your email with this easy-to-deploy mail server in a box.")
I haven't uses it for a while, but last time I wanted to setup a self-hosted server for a domain, this came out the best. (I've handed over the task to somebody else, so no idea how it went.)
Long-term vise I think this is a dubious solution to "set up and forget", or advise people with few resources or limited Linux skills to use.
Mail in a Box makes it seem passably easy to someone with basic skills to get everything up and running, but then don't support in-place upgrades of the underlying operating system. This might be fine, but I really hope people realize what they're getting themselves into. https://discourse.mailinabox.email/t/mail-in-a-box-version-v...
This critique is somewhat shallow, if the backup/restore option is easy enough. But I just have a hunch there's going to be a lot of non-upgraded boxes running fullstack-EOL everything.
Of course, a system that does in-place upgrades and then fails is going to leave no-one any happier either.
I guess that my point really is that appliances requiring "complicated" upgrades by moving to a new OS install aren't all that mature. But I acknowledge I have a hard time determining who this product really is for.
Last time I installed a mail server was a couple of month ago. I consider mysql a pro, I feel pretty familiar on command line, working in this busines as a SysOp and Developer for a couple of decades. But one not just "set up a mail server". Saying "mail is not hard" sounds kind of cynical to me. You need to handle more than one setup, you need to consider compatibilities, you need to make it right, if you want to have a secure and reliable solution.
If your not in the busines of setting up mail servers every day: Mail is hard. As long as there is no download-and-run-solution for a mail-server: Mail is hard.
Fun thought experiment. Let's rephrase hard. If you have to hire a team to manage your email for you, how much would it cost you?
The author of the post just explained how most of his friends think this is hard. I don't think email is very hard. But the ROI of doing this in-house is insanely terrible.
"But the ROI of doing this in-house is insanely terrible."
^ this is far too much of a blanket statement. I've done it, in house, for both personal and work purposes. The cost in my time was not very high, and the ROI was very good.
Obviously there is a much broader range of experiences with managing your own mail server than many on this thread think there is.
I've done this a couple times, and I think it's sufficiently difficult that if in the future I were hiring for a Linux sysadmin, I'd have them set up a mail server (that doesn't get spamfiltered) in less than 24 hours.
I never got spam filtered once DKIM/SPF/rDNS/DMARC were properly configured. I think _this_ is the myth that Big Email spreads. Linux mail servers are hard, but getting past filters is actually not hard. Spam filters are really good today not just in their true positive rate but also their true negative rate.
Given that a lot of services use your email as a 2FA mechanism, I wouldn't want to use a self-hosted mail server as my _only_ personal mail. You have to have a good reason[1] to make the time and maintenance commitment.
While the email experts are in this thread, can I get a guide on how to setup a simple email address @ my own domain?
If you google, most of the top hits are for free mail forwarding using your domain registrar. I tried this and it was pretty terrible with a ton of caveats. For example using namecheap you can only receive but not send from that domain, can't receive attachments, and can't even send an email to yourself for testing purposes.
I think the easiest way is to pay GSuite $6/month?
Don’t have experience with gSuite, but I use fastmail and very happy with them. The idea is you go to your registrar or DNS provider and change MX records to the Fastmail servers (they provide the host names and comprehensive setup help in their documentation).
There’s many options, but it boils down to two approaches:
- pay someone to send and receive your emails (eg. update your MX records to point to gsuite, fast mail etc)
- host your own mail server (something like http://mailu.io/ will work), basically you’re going to need a program that listens for and stores incoming emails, and a program that sends them, plus all the glue inbetween (IMAP, pop3, webmail, storage etc etc)
If you’re going for simplicity, then the first option would be the easiest. I think Zoho offers this for free for one address, and it works just fine.
I use mailgun free-tire and am very happy. Basically you have to set up your MX record and that's it. I use gmail's "send as" feature to send from my domain, and setup a catch-all forwarding rule of "*@mydomain.com -> mygmail@gmail.com" because I like gmail's UI. I even have a "spam@mydomain.com -> myspammailbox@gmail.com" for shady websites.
3. set domain nameservers at registrar to point to fastmail
4. setup domain on fastmail, setup email on fastmail, setup aliases (you need to pay for extra subscriptions to have separate mail users; i think same applies with gsuite).
I disagree. There is next to no code sharing between the server and client side of email--implementing the server side of the protocol is vastly different than the client side, and even the server-server versus server-client sides of SMTP and NNTP are practically different protocols. Furthermore, Thunderbird already struggles to get enough developers to avoid compounding on its technical debt; such a large undertaking is simply not possible with the current manpower.
I'll also point out:
> - Email encryption standards for key acquisitions
This is basically a standardization problem. I've been involved in at least one failed attempt at standardization here.
> - folders/labels stored as imap keywords
The difficulty here is knowing when you can opt-in to this functionality, which is again basically a standardization process.
> - large file sending protocol using third party services
Thunderbird already supports this, and has for... 7 years or so?
> - msg retrieval when email severs experience outages
If I understand this right, this is basically having your MUA duplicate emails locally... which is what most MUAs do these days (at least on desktop).
> - development in mail list servers.
What development are you talking about? Mailman is pretty actively developed, they even had a GSoC project a few years back to add encrypted mailing list support.
Or at least, choose an existing decent one email server software stack, and then focus efforts to make on-boarding/setup of that stack as easy as possible. I think you're onto something because the Thunderbid (and of course the mozilla) brand is respected enough to take seriously. AND, users would flock to it because it is highly respected as an org. Yeah, kudos for this idea!
I would venture a guess that even the most technically savvy users who could set up their own mailsever now, still flock to "Big Mailer Corps" for the easy to use mail client filled with useful AI features, etc. and that even in the presence of an easy alternative to set up their own, will end up forwarding to their "Big Mailer Corp" of choice that offers them the most efficient user experience for their workflow.
IMHO, the most important thing missing right now to promote more decentralzied mail is a cometivtive open source mail (web) app, with a simple UI and a vibrant plugin system for every use case.
Yes, this is a big deal. So many web applications provided by companies as "cloud services" are orders of magnitude better than equivalent web applications you can run on your own damn servers.
The thing I've always worried about is that to my knowledge there's no ACK for outbound mail.
The potential impact of one email I send not being received is absolutely enormous to the extent that certain mails I'd pay a decent amount of money to ensure delivery, probably more than an actual postage stamp.
No mention of RBL? Thats what makes running a mailserver cumbersome - being put on an RBL because of bad neighbours in your subnet and your emails silently failing to deliver because of said RBL.
I think I more understood the ethos of where the author was coming from at the end:
“And by all means, we must not push everyone to use Big Mailer Corps“
which is totally fair. For the rest of the article though, it’s pretty clear that the author does admit caveats where any business would have to spend time & money figuring stuff out. And the fact is there is no real cabal of Big Email, it is such a commoditized industry. As a business you pay a pittance to protect downside risks, it’s a no-brainer.
I don't understand why it has to be so difficult to run a mail server on Linux, BSD, etc. So many different pieces to set up and configure.
I've been running SmarterMail on a Windows VPS for years and it's been rock solid. It's just a single Windows service and all the configuration is done via its web interface.
Why no one has come up with a simple solution like this for Linux/BSD yet?
Email infrastructure is very important for an open and interoperable Internet, way too important for us (techies) to abdicate it to gmail. That alone is a good motivator to run your own email services. Of course, there are other benefits.
"It's way too hard" is a HN meme I've noticed regularly. It's really not, if you have any tech skills or the willingness to learn.
I set up my email server in two days between jobs in 2011 and it's been running with minimal attention ever since. I don't have precise numbers, but to try to give some context I'd say about 2-4 hours a year worth of maintenance.
Over the years I've expanded it to handle a few more domains (consulting companies for self and family members). No issues.
To anyone even slightly interested or curious in the technology, just do it. You'll gain some freedom, gain some experience and help (even if just a grain of sand) decentralize the Internet. Don't fear the naysayers.
Work at a email provider. Mail is not "hard", definitely tricky. Deliverability is a problem, spam is a problem. How do I know? We spend a lot of time working to improve on our services because of these issues.
In general : computers don't solve the issue of trust. End of story.
To all the folks answering basically "yes it's hard" I think the headline is a bit clickbaity but there are some interesting takeaways... To summarize, (a) yes it's harder than setting up Big Corp Mail, but (b) a lot of the horror stories are either old or secondhand, and (c) it's a lot less hard to set up than it used to be.
If the suggestion is, why not give it a try, one thing I feel is missed is, how hard is it not just to set up but to run over the long run, when the technology evolves and you have to keep up, and why would any IT manager or CIO want to gamble their careers on this?
Exactly. I'm curious too. It seems like the author has something up their sleeves -- OpenSMTPD, apparently. But when I look at the homepage[0], I see a very typical linux-y man-page-style homepage... A far cry from what I would consider simple.
There are a bunch of guides floating online for Postfix, Dovecot etc, and some of them are pretty decent. But they go out of date quickly, or if you want to do something slightly different, you're on your own... And even if you end up with a running system, maintaining or enhancing it is a different kettle of fish. You can't just follow the guide again, you're back to man pages and usenet mailing lists with some info inside a thread 23 levels deep.
On redhat derived systems, setting up a basic mailserver was never harder (post 2010) than service "sendmail start & & service dovecot start" + adding a couple of DNS records. Not knowing this means not having done one's homework. Which is also a good indication that one should not be messing with email.
And knowing it is still being long way from running an effective mail server.
The bigger mail providers (such as Gmail) are pretty strict on checking, and feedback is fairly limited. Thus, figuring out why you were flagged as spam is often difficult and frustrating.
There are so many subtle misconfigurations you can make with the way you have your DNS records set up, but really you need DMARC reports to figure out why your email is getting flagged.
It inspired me to build my startup :) We have successfully fixed dozens of email deliverability issues for our customers.
It's a bit of a black box, but how do you get around this? Is it a matter of domain age? IP trustworthiness? Or do a lot of people have to categorize it as "Not spam" first?
My experience with running my own mailservers for the last 20 years: it's definitely not hard. It's not painless, you need to spend a little bit of time now and then, and you need to know a little bit more about email than how to hit "send" in your mail client.
It's annoying that the default settings of almost all mail server software are suboptimal, but once I set it up correctly (yes, that took some time but nowadays there are plenty of tutorials) I had to do almost no maintenance at all. Apart from power failures that were unavoidable, the biggest issue has been spam blocker lists becoming outdated or going out of service. After that, the occasional hickup after a dist-upgrade or moving to a different country. But on average, my mail server is working perfectly fine 364 out of 365.2425 days. And, this is considerably better than the mail servers at the universities and companies I've worked at. Your Gmail might have given you no problems, but it is not working 100% of the time for everyone either.
So I'd say maintaining your own mailserver is as hard as painting rooms yourself, or changing the battery or light bulbs of a car. Not everyone wants to do that, and that's fine. If you're not afraid of it, it's a rewarding experience.
"Mail is not hard: people keep repeating that because they read it, not because they tried it"
Actually I repeat it because I ran mail servers for years, for personal use, for small businesses, for large businesses.
It was hard because spam was difficult to deal with, I don't know if that's gotten easier. It was hard because managing mailboxes, spam boxes, whitelists, webmail, etc for users was non-trivial. It was hard because occasionally you would be blacklisted and it took days to un-blacklist you, either from a spam list, or from a "big provider" because for whatever reason they just didn't want to take your mail. It was hard because you couldn't just run an open relay on most ISPs, you usually had to use your ISP's relay, and thus suffer their own issues; if you used commercial IP space that was less of an issue. It was hard because it had to be secure, and follow standards. It was hard because as your own hosting provider, you had to do all the things providers do: have a stable connection, manage your DNS, do backups, manage configuration, upgrades, patches.
We're not trying to bullshit you. We did it, and it was hard. Maybe it won't be for you, or maybe the "hardness" is just fun for you. But it's not a myth.
In the late 90's I worked for an email server software company. Since then, almost 20 years now, I've been running my own mail server. I wouldn't say it is hard so much as a PITA and very time consuming. I spend quite a bit of time reviewing logs making sure that my server has not gotten hacked or compromised in some new way.
Over the years, a new exploit type comes along, like backscatter, that I then have to figure out how to secure my server against. And I'm also very proactive at reviewing logs and banning IP's against the constant barrage of probing. I must spend a few hours a week dealing with the mail server.
And then there are periodic "someone isn't getting my mail" problems that I have to track down, where Yahoo or pacbell.net or some other large mail system decides that my server is insecure for some reason, despite not being on any DNS RBLs (which are also sometimes a problem for no good reason).
If it weren't for the fact that I have so much more flexibility and ease-of-use on my own mail server for setting up many domains and multiple email addresses, I would move everything to a provider and not manage it myself.
> Let me tell you a secret: Big Mailer Corps are not worried about you but are worried about big senders harassing their users. They do not care about your personal server sending a few mails, even if its in the thousands per months. What they care about is the infected computers or compromised servers flooding their users. What they care about are the marketing companies that are literally shitting over them, sending individually millions of commercial mails per day, trying to work-around spam filters, and that sometimes manage to go for a while without being rejected. Unless you are sending hundreds of thousands of mails to them on a daily basis, quite frankly and without trying to hurt your feelings, you fall waaaaaaaaaaaaaaay below the radars.
This is absolutely not true. A server I had mailed out daily run outputs to local users, and was also configured to send an email to one of the users' gmail accounts. I also used that server for personal email, but eventually had to stop because the daily run outputs got us blocked by gmail.
For me mail is hard because software is convoluted. I need to install for a minimal usable personal mail server: SMTP server (postfix), IMAP server (dovecot), DKIM software (OpenDKIM), Spam filter (Spam Assassin). Also I would likely need to install Procmail for some basic mail sorting.
So that's 3 huge projects and 2 small projects. They need to hook up with each other. I need to learn configuration files for every of those projects.
Add to that dozen of different DNS things (and, recently, HTTPS thing), that you need to configure.
Also TLS certificates.
Well, that's a daunting task for me.
Some kind of simple personal web server software which does it all (may be not for 100k users scale), runs on low end VPS server (10 MB RAM consumption) and requires zero configuration for typical use-cases would help a lot.
And, yeah, don't tell me that delivering is easy. I did everything right, DKIM, SPF, PTR, all those things. No black lists. Gmail still does not deliver my mails sometimes.
A tale of caution for those would-be self hosters: a friend of mine had a Linux system running mail operations and somehow got ransomwared (theory is via supermicro IPMI). All the filesystems were encrypted and the console login banner was changed to an attacker controlled advertisement to a TOR address for getting the system unlocked. He didn't pay. Logs show there was no data ex-filtration. But as usual, his backups were a couple months old so he lost a significant chunk of his email (he used web interface/imap and I don't think he cached his email locally).
I felt vindicated, giving up self-hosting of email and pushing it towards a third party provider because I was concerned about backup/disaster recovery scenarios. When something fundamentally shakes up your life - you might not have the ability to function well and managing a mail server is a stress you could live without.
His assessment on how “Big Mail Corps” accept mail is plain wrong from my experience. When I ran my own mail server, I had SPF, DKIM, rDNS and strict DMARC setup correctly, and got 10/10’s on mail-tester.com.
Admittedly my main problems were with people on Microsoft hosted emails. They seem to run some kind of IP address based blacklist-by-default operation. My emails to Hotmail/Outlook.com users would randomly get either rejected or go straight to spam.
I went through countless online forms, Twitter conversations with clueless CSR’s who kept asking me what Outlook client settings I was using, and finding random people on LinkedIn I could message.
In the end the solution that seemed to work was to reach out separately to everyone I was sending an email to on Hotmail/Outlook.com and getting them to explicitly mark my email as not spam. After a while it seemed to take and stop rejecting/marking my emails.
I run my own, but from time to time large blocks of IP space on Digital Ocean seems to get put into some blacklist.
My solution was to use a fallback delivery through SendGrid and their free tier (happens rare enough, and my volume is low).
Postfix handles this nicely for me.
The only "hard" part I find with running my own mail server for the last 25 years is switching platforms.
Deriving a large part of my income by supporting Microsoft software in small business environments, I have been running Exchange versions all that time. I have enjoyed Outlook as a primary business tool, and looking at the conversion strategies to move over to a *nix mail server that will support that client and the 25 years of horded emails I now store (don't mess with my jokes folder, dude) has been the most daunting part of it.
Even though I have been on Debian releases as my primary workstation OS for over 12 years now, I still have virtual Windows sessions to keep using Outlook. And granted, that also lets me simulate what my clients "see" when I am assisting.
TFA is so wrong on so many levels, I do not even know where to begin with. It surely does not make me want to rush to test the author's MTA.
Unless he's talking of a one mailbox server, accepting outgoing only from localhost and somehow endowed with a non residential ip (or good luck with rdns, rbls, etc.). But
that was always trivial, post UUCP era.
Background:
I've been running mail servers from 1996, several different MTAs, for my company and for customers. Getting the thing up and running is not (very) hard -for a primary MX net facing machine. That's where work begins though.
My primary MTA is still sendmail, and sendmail.cf was never one of my problems - I do not have a degree in compilers, whatever that is, and surely I never read sendmail.cf
I remember a story where a guy had the DNS of his custom mail domain changed after a malicious actor social engineered his domain name provider. The story was coupled with the recommendation to use a mail provider for the added protection that it would in theory be way less likely for their domain used by many accounts to be hijacked in this way. The guy got really screwed because once they got his domain and thus his email they could change passwords all over.
I finally ditched gmail once I realized how politically motivated Google was, and I also realized that I’d rather have a company owe me the service rather than using a free one where they could technically disable my email at any time.
Another voice saying: I've done my own mail, and it is hard. When my messages don't get through to the recipient due to my unusual choice of running my own mail, from their perspective it's as if I never sent the messages.
Just like to offer an alternative view to the "if I move to G. Suite all my problems will go away" myth.
Ive had quite a few clients over the years that have faced deliverability issues on G.Suite with sending IP's being on blocklists. Google isnt immune to this people.
e.g.
http://multirbl.valli.org/lookup/209.85.208.194.html
> We can’t let that happen: allowing e-mail to be fully controlled by a small set of cooperating multi-million users hosts is just accepting to be screwed.
I agree but don't say it isn't hard. It's hard and it's not for everyone but if you can the benefits are worth it.
Now I don't think having more small mail servers will stop big mail servers abusing their power but without a significant number not small servers better just give up.
Had been running my own mail server for the last 10 years, doing mail forwarding for a few friends domain names. Finally just moved all those domains to mailgun, and it's been great. 10,000 forwarded emails a day, for free.
To be honest it wasn't that hard doing it myself, but it broke 2-3 times over that number of years when providers changed their delivery requirements (see DKIM, DMARC etc). It was never something I enjoyed troubleshooting.
Setting up and maintaining a mail server is not hard for the initiated, but I fear is archaic and endangered to be irrelevant as basis for interpersonal communication beyond a work context. It has its place as identity provider, able to receive invoices, but this can be subject to change too - and only newsletters and mailing lists will remain.
Lately I've been having issues sending mail to Linux kernel mailing list, mail was just rejected and not sent through Gmail SMTP server.
After countless attempts to fix that or to understand why Google think I'm spamming I set up my own mail server on DigitalOcean Droplet.
I used mailinabox and the setup was a piece of cake. it works flawlessly now.
I've run my own mailserver since the late 2000s. I've never had much trouble with deliverability. Spam hasn't been a real issue either --- I run with deliberately light spam filtering because I find the occasional ridiculous spam email hilarious. (My favorite spam of all time begins "Yes, I am Bernie Madoff".)
If at some point in the future any of the first three stop being easy in an obvious way, there's a whole team of developers who will work hard to restore easiness.
That's quite different from a blog author shrugging because their configuration-in-a-tweet is obsolete in three years.
The only issue I had setting up a mail server was sending mail to servers using some service which blocks my domain. Places like my university and my work using Proofpoint, for example, seem to only allow mail from large, easily verifiable domains.
This person keeps saying “Big Mailer Corps” which is extremely annoying. That’s a really easy way to blame someone without actually calling out either specific entities or behaviors. I work for what they probably consider a “big mailer corp” and I wholeheartedly and deeply doubt there is some weird conspiracy against people running mailservers, it just actually is difficult and yes some of us really have tried. It’s kind of unfortunate that it’s hard because I don’t think, for example, you can use a personal gmail with custom domains, unless you have a separate Google Apps account. Fastmail and Protonmail do support this, and probably at least a couple other providers. It’s probably worth the paltry $5/mo that Fastmail charges for the saved time and effort debugging, updating servers, and fighting spam filters over years.
Disclosure: Google employee. (I do not work on anything related to mail.)
Is there any reason you can't forward all outgoing mail through AWS SES with a custom MAIL FROM domain so it looks like it's coming from you? That way you would have a high reputation IP pool at nominal cost for a low volume.
I thought the pain of setting up your email server was doing all the IT stuff to make sure that it's always working and you're not going to end up losing all your mail and all that.
Everything is easy to folks that have the time to grok the tech.
I like to write software. I don't like running a Linux server, although I know that i could learn to be an excellent admin.
It's just that every second I spend learning Linux admin, is a second I'm not spending learning Swift.
So I just use shared hosting for my sites; even though I'm perfectly capable of running a VPS.
To add to your point, I like writing software, I know how to set up networks, load balancers, VMs, databases, messaging servers, etc. I have no interest in doing it or being on call. I mercilessly recommend managed alternatives anytime I can. I have a manager who was glad to kill our SFTP server and pay more money for AWS’s managed solution.
He would much rather hand his boss - the founder - a bill than have to explain why our server went down.
That's all great until interruptions in the service you are depending on interferes with your business getting done (and you can't fix or mitigate the problem yourself).
Don't get me wrong, if you choose your partners carefully you can make this work in many (probably most) scenarios, and save yourself some headaches in the process. But even with the bigger players (like AWS), there is a tradeoff to handing such things over to someone else. Services aren't always as reliable and partners aren't always as resposive as they are advertised.
I totally get why people don't want to do such things themselves. I just think it is important for people to realize that doing so has its own set of consequences and risks that they should be aware of. I suspect most of the folks on HN are aware of this, but there does seem to be an awful lot of "it would be dumb to do that yourself" sentiment here.
Don't get me wrong, if you choose your partners carefully you can make this work in many (probably most) scenarios, and save yourself some headaches in the process. But even with the bigger players (like AWS), there is a tradeoff to handing such things over to someone else. Services aren't always as reliable and partners aren't always as resposive as they are advertised.
AWS is a lot more reliable than almost any in house solution. As far as being responsive, if you have a business support plan with AWS, you can always reach live, helpful support. Trust me, being on the Dev side, when I need resources on AWS, it’s just a click, script, or CloudFormation template away. Getting resources provisioned through the infrastructure gatekeepers can literally take weeks.
And just from the CYA standpoint and “no one ever got fired for buying IBM”. If your colo goes down everyone looks at you crazy and you have a lot of questions to answer. If AWS goes down and you took the appropriate steps at least making your infrastructure AZ redundant if not multi region redundant, it’s going to make news and no one is going to question why you chose AWS.
Wow. I'm surprised that this upset anyone. I apologize for that. There was no disrespect or sarcasm meant.
I simply said that I don't run my own server because I'm not comfortable being an admin; mainly because a part-time admin (which is what I would be) is a dangerous thing.
It's bad for a Linux server to be run by an incompetent admin (that would be me), and even worse for an email server to be run by one.
I realize you are likely using hyperbole, but in my experience managing my own mail server is trivial compared to the complexity of building your own electric car.
I think one of the hard things about operating any server is securing it.
If the author is arguing that 'mail is easy now' you can't make that argument without addressing the security aspect. I hate google and I still use gmail because their project zero team is so good.
Big consumer tech companies are finding their own ways to make this argument: apple with privacy, google with security research, even facebook with providing better privacy controls, although this isn't in their DNA and they'll fail. More products are supporting MFA/FIDO now.
If 'mail in a box' etc are going to convince me to self host (which I would love to do), security is the missing piece.
Running your own mail server is like having a pet dog. You have to keep it in good shape, exercise it, feed it well .. chain it up when necessary, clean up after it and so on. Keep the neighbours kids from stealing it and using it to deliver drugs, etc.
If you treat it like a pet, it makes it a lot easier to manager. "Oh, did I forget to check the mail logs this week", and so on..
I don't see self-hosting email as being any different than any other self-hosting service. I mean, take "mail server" out of our your analogy above, and replace it with "web server," and everything still applies.
It looks to be a static site generator. It's surprisingly easy to weather that storm with a simple web server + static sites, it's dynamic sites using Wordpress etc. that have the most trouble.
Heres the point though: I want to be writing email, not configuring email servers.
And if I need a Google/Apple email for usage on my phone, I'd rather just use that one instead of hosting my own email solution.
This is without even considering the fact that configuring email is not as trivial as say, setting up a website. And you're never going to get spam filtering as good as the Big Email corps.
People do not run their own mail server for the same reason people ( including the author of that guide ) do not host jquery and bootstrap on their own servers -- it is not that it is hard, it is that not running their own mail server or not hosting their own jquery and bootstrap code is easier.
Writing applications is hard. Designing a website is hard. Professional work is hard.
But setting up your initial mail server is not hard. Read some guides (ahem: https://flurdy.com/docs/postfix), fix a lot of typos, and you're up an running.
Same with developing the initial version of an application, or designing the first draft. Not rocket science. Not easy but not that hard for an experienced person with simple initial requirements.
But what is hard is keeping it running with more and more users, more requirements and evolving tech and less time to maintain it. Never mind forgotten skills and less priority.
Mail servers have a special knack for running out of disc space or going down for some reason only when you are on holiday and can not get online easily. And you usually only find out after a day or more as you start not receiving messages but don't really notice it for a while.
Or you friends, family or colleagues using it start complaining that some random person they are emailing is not receiving their email and you need to look into it... (Ps. Use DKIM and then DMARC reporting)
If you just want a reliable mail service, just use https://fastmail.com. If you want even more freedom and configurations, do try to set up a mail server. But it is hard. A bit hard.