It remains a constant frustration to me that Postfix, Exim, Cyrus, Dovecot and the like still feel as if they belong very much in the "here be dragons" territories of the Unix world. Configuring these systems is an exercise in constant frustration and bafflement.
They're such a pain to use that since becoming the sysadmin in charge of our work email servers, I gave up running personal ones as well and just pay FastMail to deal with it for me - ain't nobody got time for that.
I often dream of taking a sabbatical and writing modern, user-friendly SMPT and IMAP[0] servers. If only so I could use them at work on my return and save myself a lot of time and stress.
It is very much "here be dragons" but a lot of that is because of legacy compatibility issues from when mail was delivered directly to local users.
If you don't care about local access (IMAP & POP is good enough, no mutt to the mail spool for you) Dovecot is a huge improvement on everything else. You configure it to listen directly for LMTP and use MySQL for user information and Maildir for data it's almost as easy as running a random php+mysql webapp.
Even serverside filtering with sieve "just works".
The inbound SMTP/Spam filtering stack is still a PITA but that's because of security issues (spam).
DBmail (dbmail.org) is one rework that seemed sane but stores all messages in a RDBMS and I didn't want to deal with scaling it at work (little ISP, ~10k mail users, ~1.5 million messages per day including spam) but it was fine for personal use.
I've been playing with homegrown POP & SMTP servers that use a s3 compatible datastore as the backend but that's a side project. People go crazy over email so I really want a simple to operate, sane, zero point of failure mail system...
Re "zero point of failure mail system": I also use Dovecot, and I learnt about DSync recently, although haven't played with it yet. It provides two way synchronisation of mailboxes between two Dovecot servers, so you can store the same mail on two boxes, in completely different locations.
The clever thing is, it can recover from a split brain scenario completely, safely, and without any losses. If your two servers can't see each other for a few hours and you make conflicting changes on both of them, then it apparently is able to recover completely and entirely automatically when the connection comes back up.
You could have one remote server in a DC, and the other locally in your office. Point your mail clients at the one in the DC, but then configure your office router to intercept connections to the DC server and re-route them to the local office server. When you're in the office, you hit the local office server. When you're outside of the office, you hit the DC server.
[edit] Any chance your can elaborate on your S3 test setup? I've considered something similar. I would be interested if there was any mail server software that already does this. The alternative would be to use an S3 based filesystem as your store, but this doesn't seem very efficient. You'd definitely want local caching of messages in this setup.
So, for S3, I cheat and have my own Riak CS cluster so storage is fairly quick and cheap. I should probably use Riak unless the message is too big and then put the body in CS but I was thinking that straight-up S3 is a good lowest common denominator for storing things.
Basically when I started out learning Go I saw a presentation by bradfitz where he made a POP3 server that used twitter for the backend and thought hey, access S3 instead. He had a smtp library on github so I hacked up something that accepts all mail and stores it in a S3 bucket too. I haven't hacked in things like user auth so it's all very proof of concept/useless at this point.
I'll have to take another look at DSync. The SSH batch job per user scared me off last time. I might be able to use it for HA/Failover.
There's probably something wrong with me but I quite enjoy the masochistic rituals of getting postfix/dovecot/opendkim/spamassassin/zarafa humming.
Postfix is actually a beautiful piece of software once you spend some time with it and get to know it. Like most things, once you've done it a few times (and written everything down!) it isn't so bad :)
LDAP on the other hand... that's something that _always_ defeats me.
>> Postfix is actually a beautiful piece of software
Couldn't agree more.
I've also learned a ton from the source code. If anyone has even a passing interest in security, i'd highly recommend browsing the postfix source code.
Designs of much more elegance have been explored (qmail). But as a matter of fact, neither postfix, nor qmail, nor exim, nor (god forbid) sendmail belong into our day and age anymore (I have run 3 of them at scale).
A modern MTA is way overdue. Please keep writing MTAs kids!
In modern languages. Until you get one right. Thanks!
Sorry moe but yes, just yes. It's rock solid, battle hardened, has a fantastic security record, is extremely resource efficient, has gobs of up to date and useful documentation, an active mailing list, a primary author who's stuck with it since conception (unlike DJB) and offers enough flexibility to reliably support virtually any mail setup you care to dream up.
I can't see why in the world anyone would advocate throwing all of that in the bin and adopting something new, flashy and unproven that happens to be written in go/rust/node/other language du jour? I mean why? So we can write configs in JSON and get a 'free batteries included' web management console for monitoring queue lengths?
So we can write configs in JSON and get a 'free batteries included' web management console for monitoring queue lengths?
Yes. Exactly that.
Postfix requires the user to understand pretty much all of its delicate (and often very much ass-backwards) abstractions at once, to accomplish even the most trivial tasks.
Aliases vs virt aliases, transports, the six dozen magic interactions of $mydestination, $myorigin etc. with one another and everything else, ... don't even get me started...
Nothing is easy in postfix. Everything is deeply entangled, counter-intuitive and error-prone. And there is absolutely no reason it has to be that way.
See what elasticsearch did for FTS. This is how software is designed today.
I'm not bashing Postfix, it's still a workhorse. But let's not be sentimental about software. Being better than sendmail simply is no longer the big deal that it was in 1998.
Today I expect my MTA to be all that postfix is, and easy to use.
It's interesting that you brought up ES actually as I've just been toying with it as part of a project I'm working on (I usually use Solr). Unfortunately, I hated it so I guess we just have very different views on what constitutes a good approach to software.
Things I didn't like with ES include:
- Awful documentation (even the newly released guide) and too much marketing speak. I constantly felt like I was reading a brochure rather than a man page.
- I found it opaque and difficult to get clear, concise info on how it's put together and what it's doing under the hood.
- Marvel was nice but it's a shame they charge for it (it's not _that_ nice)
- The install routine (in an effort to be super simple I guess) just felt wrong and needlessly over simplified (I didn't like it making decisions without telling me what it was doing).
And more...
I will concede that working with ES's config files is a lot nicer than dealing with Solr's XML though! And in case you're in any doubt, I am by no means an enormous Solr fan either.
Anyway looks like we'll just have to agree to disagree on this one :)
Edit: also, I am occasionally prone to a bit of sentimentality for good, older software (it took me far too long to give up on apache and embrace nginx for example) - so there's definitely a little bias on my part!
Edit 2: If you could get Salvatore Sanfilippo and Igor Sysoev to team up and build a modern MTA, that would interest me :D
Now that's an interesting comment, thanks for that.
I'll probably come across as arrogant now (sorry), but your critique of ES sounds almost ironic to me.
It's especially curious as you do claim experience with SOLR. Have you ever scaled it beyond a single core?
I'll go on a limb and claim that most of what you perceive as opaque in ES is likely magic that actually works. The rarest kind.
Yes, Marvel is not optional to run a meaningfully sized cluster. But where is Marvel for SOLR again? Where is automatic clustering and balancing? Oh, it doesn't exist?
Sure ES isn't perfect. But it's one generation ahead of SOLR. And Postfix is one generation behind SOLR. That's all I'm saying.
I started out with Exim (3.x or 4 -- can't remember now) and stuck with it mostly for the same reasons that Debian didn't change away -- I wasn't convinced postfix was a significantly cleaner/simpler design -- that it would be worth it to learn how the "other" system worked.
I did have some exposure to qmail before that, and it did feel like it was simpler in some good ways -- but then it became a sort of abandonware, sadly.
[edit: I did recently switch a few vps' over to nullmailer rather than use exim4 configured for smarthost delivery -- once I figured out how to get tls working thanks to this blogpost http://metz.gehn.net/2012/11/nullmailer-with-starttls/)]
Are you talking about the interface to the sysadmin - the fact we see the nuts and bolts in master.cf? This is of course an unfortunate side effect of the power and flexibility on offer.
What i was trying to dig at was more about the code than the user facing side of things.
It beautifully modular. It clearly written - that's a crazy hard achievement for software as committed to security as postfix.
It's dead easy to hack on the code - you could arrive and even without looking at the first class documentation you would be able to tackle implementing a change. The source is that thoughtfully laid out.
I've been running it for a few years now, admittedly with a very basic setup. My config is about ten lines long, written in five minutes or so. I haven't really needed to touch it since.
I love OpenSMTPD. The only issue I had with it was, no filter API (just fixed, as of a month ago or so) and to get things like spam and antivirus filtering, or domainkeys, you have to proxy chain it. Also, the API changed very rapidly, and I had to run nightlies for a while to get around a few issues (I'm on stable now though with no problems.) These things add complexity to the configuration, but I would still rather do this than maintain any other mail server software.
It's on my radar, but to date I've not had (or more accurately made) the time to seriously evaluate it. I'm looking at an architecture overhaul soon, so will make the effort to take a proper look before then.
At some level I've always viewed this as an example of how FOSS can fail to deliver good products at times. It should be embarrassing to the FOSS community that something as fundamental as email services requires super human effort to setup, administer and maintain. I don't know how it is we got here. It's a real head shaker.
I experienced something completely different. I set up a postfix/dovecot/ldap/nginx/roundcube/spam/... for a medium sized office with some special needs last month and was amazed how easy it is. Especially postfix and dovecot are a breeze to work with.
The main problem was, that you can do everything in X different ways and have to think careful which way you want to go.
In addition i am running a very simple private setup for years now without any problems.
I also find them baffling and frustrating. It seems that senior sysadmins are fond of "But email is so easy", and I'm sure it is... once you already have your battle-scars (I'm finding Buildbot lies in the same category). Another sysadmin friend of mine says that setting up your own proper mailserver should be a rite of passage before you can call yourself a sysadmin.
Edit: it's not so much that it's about configuring Application X, but that there are so many moving parts, from local aliases to firewall fun to DNS entries. Then there's the fun of 'deliverability', greylisting and so forth.
Well, you only need two from your list. And I don't really think Postfix+Dovecot is much harder than Nginx+$dynamiclanguage. It's just something fewer people felt the need to do historically.
Setting up mail servers is the one things I truly dread. Its truly barbaric. I cuss and froth at the mouth whenever I have to go down there into that postfix dungeon.
Its the reason I got into Puppet and Ansible - vainly hoping all those barbaric problems could be encapsulated.
Do anyone have thoughts/feedback on Haraka - a mail server used by craigslist http://haraka.github.io/ ( Runs with Node.js + Javascript based plugin system )?
I've used Haraka standalone for a couple of custom-jobs, but I've never used it at any significant volume. That said qpsmtpd was a pleasure to use, deploy, and develop against, so I'd expect to have a similar feeling with it.
On a serious note - you'll find the Haraka developers more friendly and active than any other MTA out there. Just join the #haraka IRC channel on freenode and we're happy to help anyone.
I dread having to set up mail servers. Setting up a typical dovecot + postfix install with a webmail frontend that looks as good as gmail should be as easy and simple as setting up wordpress, but it isn't even close.
Part of the problem is that many years ago certain
people decided it would be a good idea to tightly
couple email to domain names (DNS). Previously email
needed only IP addresses to work.
The result is that now when you are configuring SMTP
you have to also configure DNS. That means more things
that can go wrong, and more things to check as you are
setting things up.
It also means you may need to pay a fee for a domain name.
This is because we all submit to the notion of an ICANN root
and commercial registrars selling (renting) names that
cost nothing to create. Thus email is not solely under your
control. You generally have to play the ICANN DNS game, only
because your email recipients are playing. Nothing stops
anyone from running their own root though. And this is what
is done with private DNS inside organizations.
And then, as if that DNS complication was not already enough
to take control of email away from you, you have various
schemes trying to prevent spam that discriminate for or
against mail you send based on IP address and domain name.
Can you operate email without DNS? Technically yes. There
was a time before DNS, and email worked just fine.
Practically speaking, today you need DNS, whether it's under
ICANN's root or your own.
All this hassle steers you to just accept third party
email hosting. Profiting from this arrangement has become
a career for many a man. And with "the cloud" many are
hoping to cash in yet again, as organizations who once ran
controlled own email feel pressured to let a cloud computing
vendor control it for them.
The fact that all this third party control makes
warrantless search and surveillance so easy is but one side
effect. Centralising hundreds and thousands of accounts in
third parties make the spammer's job easier, too. If you
think about it, there are many unwanted side effects of
centralizing email. When every sender and recipient are
connected directly to each other via a network, why would
you want to prevent them from sending messages to each other
directly?
With the constant connectivity and bandwidth we have today
in many places, the centralisation and outsourcing of email
is baffling to me... it is nonsensical... until you remember
how much of a PITA it is setting up email.:)
It's no wonder we let third parties handle it. Is this PITA by design? Who
cares? Let's just fix it. More of these projects should
exist. Or made public (I imagine many of these are personal
setups now being released for public use). I have my own
that uses qmail.
DNS is not the issue with mail and MTAs. Setting up an MX record is something you can do after googling and reading for about ten minutes. I have only anecdotal evidence to prove this, but that's basically how I set up my own first mail server.
What was a lot more difficult was setting up the actual mailserver itself. Even a simple, two-mailbox-operation was an exercise in frustration when it came to trying to get mail working on a little VPS of mine. Shit, you have to make the sendmail config. How balls-out insane is that?
More recently, there's little working tutorials to get yourself a working dovecot/postfix server, which are relatively easy to understand (thanks, digitalocean!) but I just checked out the first one I found on my google search and it's 2,800 words long. 20 pages if you were to print it out dead-tree style.
I can give you a tutorial on DNS and MX records in much less time than it would take to go through setting up any MTA on linux, and that's the trouble.
But I am actually referring to something different:
hosting your own mailserver.
So when I say "set up DNS" I mean set up a DNS server,
not simply an MX record. This allows you to create your
own domain names and hence email addresses. As I said
above, these email addresses are valid so long as you
and the recpipient use the same DNS root (e.g., ICANN's
root in the case of the public internet).
As bad as things are in terms of the relative difficulty
of setup, I think there are defenders of the status quo
for email and I imagine this explains how I could be
downvoted for my comment.
Don't get me wrong, I love email. It is the reliance on
others to handle 100% of it that troubles me. It is purely
a control issue.
No, you said that. I said that having email so closely
coupled to DNS is part if the problem.
I'm not sure why you would have to remember IP addresses.
We routinely "dial" telephone numbers by selecting from a
list of contacts. IP addresses are approximately the same
length as telephone numbers. The folk wisdom is that people
can remember about 7 digits. But even if you disagree on
all of this, what does that have to do with letting someone
else control our email? The issue here is control, not
whether we use names or numbers or something else when we
enter the address of the recipient.
The nonportability of IP addresses is a problem in its own
right, but I don't see the relevance here. Again, you are
trying to engage me in a debate over domain names versus IP
numbers. Perhaps that is an interesting issue, but here
I am interested only in the issue of control over email (and
because email and DNS have been coupled together, DNS).
And that is what the OP is interested in as well.
I said that email and DNS are closely linked and this makes
email more challenging for any user to control. 1. Because
it complicates the setup and 2. because DNS as we currently
accept it is controlled by third parties.
You are trying to suggest that I am advocating against having email addresses that use names instead of numbers. I am
not.
If email is linked to DNS, and someone else controls DNS,
then you cannot control email.
If you disagree with the preceding statement then please
explain.
They're such a pain to use that since becoming the sysadmin in charge of our work email servers, I gave up running personal ones as well and just pay FastMail to deal with it for me - ain't nobody got time for that.
I often dream of taking a sabbatical and writing modern, user-friendly SMPT and IMAP[0] servers. If only so I could use them at work on my return and save myself a lot of time and stress.
[0] Possibly even with Exchange ActiveSync support - Microsoft licenses the protocol, but I've no idea what it costs http://www.microsoft.com/en-us/legal/intellectualproperty/IP...