
Magma – An encrypted email server daemon - ycmbntrthrwaway
https://magmadaemon.org/
======
SwellJoe
One of the interesting things about this is the work on DMTP/DMAP, which are
"Dark Mail" protocols that have security built in from the ground up
(developed by Zimmerman of PGP fame, some Silent Circle folks, and Ladar
Levison of Lavabit). It's a pretty neat thing that I hope sees widespread
adoption. Fixing email has proven very difficult, due to the incredible
inertia of existing systems...but, at some point we really do need to move
forward on secure email communication.

So, this is really cool stuff.

~~~
ape4
This seems to be a paper about it
[https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NE...](https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2015-09-1/NET-2015-09-1_03.pdf)

------
djsumdog
I see references to ClamAV, and references to soon coming official docker
images. So is this a complete MTA and delivery (IMAP/POP/etc) servers? I was
digging around on github and the about page, but there doesn't seem to be info
on how it does spam detection, or DKIM. Does it support signing messages?

I wish they had a more detailed about page and feature list, but I am still
excited about this.

~~~
mark242
The ClamAV dependency concerns me; messages can only be passed to clamd via a
plaintext stream. This seems to be at odds with the end-to-end encryption.

~~~
matt_wulfeck
Does the "encryption in flight" include interprocess communication? If so
there's not much safety in the world!

~~~
mark242
There's a difference-- point-to-point encryption is what you're talking about.
Anytime a message is transferred, it's done via TLS, but you can't be
guaranteed that at any one point the message is kept encrypted. You can have
this today -- just about any MTA has a configuration to only allow TLS
connections, and then you can set up an IMAP server to only allow encrypted
connections.

End-to-end encryption is the system whereby the moment the message leaves the
sender's hands (phone, desktop, whatever) it's encrypted until it gets into
the receiver's hands. This is your OpenPGP/etc. You could think of that as
"Signal for email". This is apparently not what Magma is doing.

Here's the relevant line from Github:

[https://github.com/lavabit/magma/blob/develop/src/providers/...](https://github.com/lavabit/magma/blob/develop/src/providers/checkers/clamav.c#L387)

You can see that Magma is writing the email message to a file and instructing
ClamAV to scan the file. This is actually worse than writing to a clamd
socket, since (imho) it would be easier for someone to monitor a directory in
a filesystem on a compromised Magma server.

Compare this to how SpamAssassin communicates with ClamAV-- it streams to
either the port or socket:

[https://wiki.apache.org/spamassassin/ClamAVPlugin](https://wiki.apache.org/spamassassin/ClamAVPlugin)

~~~
ladarlevison
Your slightly wrong. Magma uses mmap'ed temp files to hold the messages
because ClamAV requires a file handle, or a file name. In theory the message
only ever hits disk if memory is exhausted. And at least in theory, is never
linked to the file system tree where it can be intercepted. If you know a
better way, please submit a pull request.

------
kijin
If the MTA supports regular SMTP, POP and IMAP, then it cannot be said to
perform end-to-end encryption. When a client accesses mail through any of
these legacy protocols, the server must decrypt all messages before pushing
them down the last hop. This used to be Lavabit's way of offering "encrypted"
email.

If the MTA _also_ supports some other protocol that affords end-to-end
encryption, that's great, but now you need to use special client software that
is either proprietary or seriously lacking in features. This is ProtonMail's
way of offering encrypted email.

ProtonMail is currently working on a proxy called ProtonMail Bridge that runs
on the client and talks to legacy MUAs. All communication between Bridge and
the outside world is still encrypted. That sounds like a good idea to me,
since I have no intention to stop using Thunderbird anytime soon. It's also
probably the only way to bring end-to-end encryption to a large number of
people, since it embraces and accommodates the inertia around email instead of
arrogantly telling people to switch and wondering why they don't. When it
comes to email, backward compatibility is everything.

I'm not sure if Lavavit's Magma can fill the same need; the documentation is
rather sparse. But I would gladly run a local MTA if it means I can have my
cake (end-to-end encryption) and eat it too (compatibility with legacy MUAs).
DIME also looks like a much more open protocol than the vendor-specific stuff
that ProtonMail is using.

~~~
saurik
While you lose a lot of the benefits of IMAP if the server does not have
access to the content of messages, this is absolutely not necessary for POP
and the only information which must be transferred in the clear to use SMTP is
the TO address (and even then, only in the envelope, not in the actual
message). Claiming that these protocols inherently are incompatible with
transferring encrypted content is like claiming that any protocol that sits on
top of TCP or HTTP is incapable of end-to-end encryption. What I think you are
actually trying to say is "compatibility with existing clients without any
additional software or plugins", but that is unrelated to the usage of SMTP,
POP, and IMAP.

~~~
kijin
You're technically correct, but I don't think it's realistic to separate the
payload format (plain unencrypted RFC 2822) from the various transfer
protocols that have grown with and around it.

You mention HTTP, but do you think HTTPS have been as ubiquitous as it is now
if people had to install a browser plugin in order to enable it? That sounds
suspiciously similar to what South Korea has been trying with their infamous
ActiveX plugins for their own national PKI. Nowadays they've moved away from
ActiveX to actual browser plugins for Chrome/Firefox/etc, but nobody likes it
as long as they have to install even a single plugin. "Compatibility with
existing clients" really just means "compatibility with existing clients
without any plugins". People don't like installing plugins.

Maybe a few years from now, every well-known email client from Outlook to
Thunderbird to SquirrelMail will support an E2E encrypted email protocol
(either a modified version of SMTP/POP/IMAP or a brand-new protocol) out of
the box. Then, and only then, I think it might have a chance to become as
successful as STARTTLS has been.

------
HurrdurrHodor
Can somebody tell me what this actually does? All I see is an MTA and some
hand waving.

~~~
myhf
Some history here:
[https://en.wikipedia.org/wiki/Lavabit](https://en.wikipedia.org/wiki/Lavabit)

~~~
HurrdurrHodor
I know of lavabit but I want to know what this software does.

~~~
liopleurodon
Here's the talk Levison gave on it at Defcon a few years ago

[https://www.youtube.com/watch?v=TWzvXaxR6us](https://www.youtube.com/watch?v=TWzvXaxR6us)

------
mike-cardwell
I blogged up some security issues with the Lavabit.com service which uses this
Magma server. Some of you may be interested -
[https://www.grepular.com/Lavabit_Security_Misconfiguration](https://www.grepular.com/Lavabit_Security_Misconfiguration)

------
ianopolous
At first glance it seems like this will still expose who is emailing who to
the network (and potentially other metadata), as well as relying on dns to
function. Is that correct?

------
doomrobo
I guess I'll be the one to ask: Why write an email server with security as
priority #1 in C? I'd understand if it was built off of an existing daemon,
but it seems that this was written from scratch. There are many other options
out there that are reasonably fast and don't have the pitfalls of manual
memory management and undefined behavior that C has.

~~~
jhasse
One reason for this could be that for an _open-source_ application with focus
on security it's very important to have many collaboraters, who look at your
code and contribute. C is still very popular and many people know it.

~~~
tptacek
C limits the pool of contributors. It doesn't expand it.

~~~
devillius
I don't think the Python/Ruby/Node/VB developers are the intended
contributors.

Source: I'm a Python/Ruby/Node/VB/C/C++ dev

~~~
tptacek
Why not?

(I started my career shipping C code, FWIW.)

------
robert_foss
I was hoping to see JMAP support. Is there a reason to prefer DMAP over JMAP?

~~~
SwellJoe
As far as I understand it (which is quite limited, I haven't been able to find
a white paper or whatever about DMAP thus far), there's a fundamental
difference in what layer of the stack security resides in.

JMAP is a vaguely IMAP compatible protocol, in that the mail storage is
probably just Maildirs or whatever (with some indexes to make it fast). It is
not a complete re-design of how email works. Encryption is just in the network
layer between client and server; the server, if compromised, would compromise
the mail (unless other encryption features are layered on top, as in the
current email standards). That's a perfectly reasonable design decision, given
how email has always worked, and the problems inherent in pushing encryption
to the client...particularly if the client is a web browser.

DMAP is not merely "IMAP with JSON over HTTP" (JMAP isn't just that, but
that's a reasonable enough short description). It's a fundamental redesign of
a mail client protocol. Again, I haven't read the formal spec or anything, and
there is no source for DMAP yet that I can find in the repo (DMTP is there,
but no DMAP), but given what it's promising, it can't possibly just be layered
on top of IMAP without significant changes. DMAP seems like a very long-term
project; maybe years away. I don't know if there are any implementations, yet.

And, JMAP is so new as a public thing that I would guess it wasn't on
Levison's radar until recently. It recently began the formal standards
process, which is cool. But, it's also not really got any complete
implementation (just a PoC proxy and an incomplete implementation in Cyrus
with plans for Dovecot to get support at some point, too).

So, I think it boils down to a few things: JMAP doesn't solve the problem DMAP
sets out to solve (which is end-to-end encryption with at-rest encryption on
both ends), JMAP is also very new and has no production-ready implementations
on client or server, and these are both open sourcing of projects developed as
commercial tools (JMAP comes out of Fastmail development, DMAP comes out of
Lavabit and Silent Circle).

Also, this does appear to have an HTTP server...so, probably something quite
similar to JMAP, though certainly not talking the same protocol (since it pre-
dates the publication of JMAP details). It could probably be made to support
JMAP without a huge amount of work, since it already supports IMAP and HTTP.

~~~
lloeki
> JMAP is also very new and has no production-ready implementations on client
> or server

I was under the impression that it was more or less running live at FastMail
but I admittedly never dug into the details.

~~~
SwellJoe
If so, they haven't Open Sourced the implementation they're using (they have
open sourced a JMAP-IMAP proxy that is pretty nice, but I probably wouldn't
want to use a proxy in production). I don't recall what they've said about how
they're using it, and if they're using it or some precursor to JMAP on their
production systems.

~~~
dochtman
Recent versions of the OSS Cyrus IMAP server should include JMAP support.

------
eliaskg
Could someone ELI5 what that actually does and how one would use it?

~~~
thephyber
Magma is an email server, supporting many of the common email protocols,
designed with security in mind. It was designed and used by the guys at
Lavabit (of Edward Snowden fame).

It combines outgoing (SMTP) and incoming (POP, *MAP) functionality, so it
would be like using Dovecot and Postfix (rival mail server products) together.

It requires that you have a relational database (the documents describe MySql)
and a key-value store (the documents describe Memcache) to support it and the
documentation suggests it's designed for Linux (specifically with instructions
for CentOS, which is a Red Hat distro). A mail server is generally installed
on an "always-on" computer connected to the internet-connected network.

Note that even though it's "designed with security in mind", I don't see any
mention of a security audit on either the linked page or the GitHub repo.

------
j88439h84
Is it reasonable to run a secure personal email server with this?

~~~
__jal
I don't mean to be a pain, but before anyone can try to answer that for you,
you need to define 'reasonable' and 'secure' in terms of your environment and
needs.

Are you Just Some Person(tm) with some technical chops, who otherwise isn't
going to attract a lot of unwanted attention?

Are you someone who does things that might annoy others enough to make them
want to mess with you? Who are they and how annoyed might they become?

Are you an aspiring Ed Snowden?

Do you have confidence in your skill set to 'know' that you've configured this
and everything else on the machine securely? If so, do you watch the
disclosure lists for vulnerabilities, enforce configuration, audit and
monitor, and generally stay on top of these things?

Etc. Again, I'm really not trying to be overly pedantic, but there's just no
such thing as 'secure' without qualifying it with a description of 'against
what'. Similarly, what's 'reasonable' for me - someone who's worked with this
stuff my entire career - is absolutely bonkers to suggest to a non-technical
person.

Anyway, all that said, this doesn't look ready for prime time to me. I'm
certainly interested in it, and might play some when I get a chance, but even
after it looks ready for use, I'd likely wait for a while to see what other
people find before deciding to use it in production. My mail system runs
Postfix and Courier with various support systems. Mortals can indeed run this
sort of setup, if they're willing to learn a bit. It isn't rocket surgery, but
it takes a bit of work.

~~~
drdaeman
I don't know about the parent comment's requirements, but I'm:

1\. Someone who has some servers and a little tiny bit of technical knowledge.
Or, to be exactly, I happen to already run Postfix+Dovecot+rspamd for my
email.

2\. Someone who is interested in personal email security/privacy, in
particular in terms of both in-transit and on-storage encryption. I can't
realistically watch for _every_ CVE and patch all the holes on the day 1, so I
would always love something that would let me trust my servers and network
less.

I don't have any specific threat model or a designated adversary. I just see
"hey, there's a some new stuff that looks promising" (this submission) and
wonder if it could improve my life. However, the documentation is sparse, or,
I'd better say, nearly non-existent (or I'm looking at a wrong place). So,
asking if someone had already spent time to evaluate or had actually used that
and can share their experience is a logical choice.

Thus, I just wonder - is Magma for me, can I run a secure personal email
server with it? Like, use it to replace my Postfix+Dovecot system? Can I
expect it to be no less reliable than my current setup (which is just your
average run-off-the-mill configuration, documented in every other "how to set
up your own mail server" article)? Is it correct to expect it to add some
value in terms of security? What the exact those improvements would be?

~~~
__jal
As I (very briefly) mentioned, it looks to me like Magma isn't ready for
prime-time yet. I've seen reports that, for instance, the LDAP server allows
completely insecure configurations (cleartext passwords on the wire) without
so much as a warning. That sort of thing should not be possible with a product
like this, and makes me think they're gluing together pre-existing code, which
is also a little worrisome[1].

Beyond that, I tend to be pretty conservative with things like email. What you
currently run, assuming it is configured correctly, is solid; I'd personally
be hesitant to replace it until other folks have bled over the inevitable
first bugs and it has been in use long enough for people to learn how to
operate it, figure out any weirdness, etc. At the least, you'll be skipping
what will probably be several point releases right after whenever they
actually call it done.

TLDR; if I were you, I'd hold off. (And am personally holding off.) I'm very
interested in this, in theory, but even though I am technical, I'm not a
cryptographer or a security researcher, and even if I were I don't have the
time to audit it. I want those folks to beat it up before I wade in.

[1] Nothing wrong with that in principle, but combined with the jagged edges,
it makes me wonder how much attention any grafted code is actually getting.

------
konschubert
How does it handle Spam?

------
nwmcsween
mm_set in memory.c won't actually do what you want, the asm statement needs
fixing and it can be implemented in terms of memset.

~~~
ladarlevison
mm_set is being used. As I recall that pattern was inspired by looking at a
libc implementation for memset. The combination of using volatile and the
asm("") statements keep the compiler from optimizing away what it thinks are
unnecessary memset (like a wipe operation). It's possible that solution
doesn't work on every system though. So, how would you improve it?

~~~
nwmcsween
[http://www.5starstandard.ca/projects/myosin/myosin/string/me...](http://www.5starstandard.ca/projects/myosin/myosin/string/memsset.c)

Basically just the right inline asm. But the project as a whole is a bit of a
mess, multiple 'memssets' will not do anything to make things more secure than
just one.

