Hacker News new | past | comments | ask | show | jobs | submit login
JMAP is on the home straight (fastmail.blog)
193 points by ac29 on Dec 27, 2018 | hide | past | web | favorite | 50 comments

"The IETF likes separation of concerns" but they're letting Google et al. ram through MTA-STS, which fails to define a standard port number for SMTP-over-TLS and instead requires MTA implementations to also be able to fetch port numbers over https, a completely unrelated protocol.

All because the authors believe mortal men are not capable of understanding DNSSEC. Leaving aside the unholy alliance of Google + Microsoft + Verizon + Comcast, the fact that the IETF made Fastmail ditch authentication from a mail protocol while letting Google subsume DNS and web-based port knocking into another mail protocol definitely tells me there's nobody behind the wheel any more.

I'm happy that the Fastmail team is doing work that excites them, but does anyone know if the IETF has any plans to use ports other than 443 ever again?

> "The IETF likes separation of concerns" but they're letting Google et al. ram through MTA-STS

This is a fundamental misunderstanding of how the IETF works. The IETF codifies industry standards but, as with the W3C, it doesn't have some sort of veto authority over what companies actually implement. If major vendors are using MTA-STS, it's in everyone else's interest that it be described in a public standard. The IETF does get high-quality feedback and I see no reason to question the sincerity of Fastmail's stated reasoning and expressed appreciation.

Similarly, “the authors believe mortal men are not capable of understanding DNSSEC” might be an amusing rhetorical flourish but it's not as convincing as the actual numbers showing that DNSSEC is not reliable on the internet now whereas HTTPS has near-universal deployment _and_ doesn't have the various design flaws which the DNSSEC community is slowly fixing. If you want to protect users, it's usually better to pick something which doesn't require hundreds of parties around the world to make significant changes — and that certainly fits with the “rough consensus and running code” ethos.

You may disagree, but there are other reasons not to rely on DNSSEC besides "mortal men not being capable of understanding it": https://sockpuppet.org/blog/2015/01/15/against-dnssec/

HTTPS does seem like overkill, but on the other hand, it's likely already running (even I, who host my own mail, have an HTTP daemon on the same server), so in the common case, it doesn't increase the attack surface.

The blog you quote is from 2015. Let's look at some recent numbers on DNSSEC/DANE deployment.


These graphs don't address any of the points. Even worse, those numbers don't say anything about how many DNS requests are actually verified... A good guess is that those crypto keys sit idle.

Welcome to the machine. Glad I'm leaving this industry. It's becoming unrecognizable.

I have not tried it or looked at it too closely (since neither my server nor client supports it). But I desperately hope that JMAP succeeds. It's exactly what email needs. IMAP and POP are just so inadequate that locking yourself into gmail/whatever is a reasonable choice.

With JMAP maybe we will actually see some interest in decent desktop and mobile clients for once. And also maybe we will have decent alternatives to the big players for non-techie people too.

I’ve roughly planned a talk entitled “building a fair dinkum email client in half an hour with JMAP” (I’d take an hour if permitted; the result would be much better). The talk is mostly live coding the thing from scratch, with authentication being the only area where I cheat. I submitted it for Strange Loop earlier this year, but it wasn’t accepted, so I haven’t actually written it yet. Anyone got any suggestions of places to submit it to?

That JMAP makes it straightforward to build a rudimentary but fully functional email client gives me hope that people may experiment more than they have been. I believe that a large part of the reason that there are few email clients now is because getting a basically-functioning email client going is a surprisingly large investment of effort, and doing it well is an enormous investment.

I had to lookup what "fair dinkum" meant. My first reaction was that it was probably a technology that I don't use and probably am not interested in. I would think that others may have similar reactions. Maybe if you took it out? "Building an email client in half an hour with JMAP"?

Though that makes it sound like it would work with any email provider. It's not clear that JMAP is a protocol. "Building a JMAP email client in half an hour"?

Maybe there's not enough interest in JMAP yet, and a talk that promotes JMAP over IMAP/SMTP would be better first. Maybe the client could be prebuilt and simply presented as part of the talk as FOSS, showing the small LoC that it took and simply mentioning how little time it took. Maybe having it in multiple programming languages would help, too, to show that its simplicity is not a matter of language of choice.

Just throwing ideas out there.

I'd love to see this talk :-) so if you want to keep up to date with CFPs, the Mozilla TechSpeakers group maintain a Twitter profile at https://twitter.com/mozTechCFPs that tweets timely reminders to submit proposals to conferences. It might help you find places to submit your talk. I hope you do it in London then I can see it live but I'd love to see a recording as well :-D

36C3 (next year's chaos community congress)? I'd love to see that talk, either in real life or online. Coding my own mail client is something I wanted to do for a long time and if I can make decent headway in only 30-60 minutes, I'd definitely watch that and code along.

The only thing I'm afraid of is the programming language, please don't pick some hipster language (e.g. invented in the past five years) or some bloated platform like Electron.

For the base version of the talk I’m just using straight HTML and JavaScript with no external libraries. Variants of the talk using other libraries (e.g. React, Vue, whatever the flavour of the month is) would be reasonable. All it needs is for the JMAP endpoint to be CORS-enabled (probably a bad idea in general), or for a simple pass-through proxy to work around that (I’ve already made a simple such tool for personal use at FastMail). Other than that, it’s just running in the browser with no external dependencies; that’s one of the great things about JMAP, compared to IMAP: it’s web-native.

Anyone got any suggestions of places to submit it to?

FOSDEM? Probably too late for this year, though.

I'm hoping for the same... I started down the rabbit hole of creating a mail server/client once. Regarding IMAP, all I can say is damn, that is a pain and I pretty much gave it all up. Smartermail is probably as close as I've seen to what I'd love to see as a more broad open-source product for mail. I just don't have the patience to wire together a half dozen systems, the maintenance and more to have working email...

I have one address hosted via outlook.com (relic from when they had a free tier for personal domains), and the bulk of email is via gmail. I have mail forwards in google domains for a few really old email addresses (just used it the other day to recover an old account). But for the most part, I would love to see some better personal mail options.

I've roughly looked at some of the mail in a box stuff, but I don't want all of DNS and the world for the domain controlled under my mail server.

> IMAP and POP are just so inadequate that locking yourself into gmail/whatever is a reasonable choice.

Totally agree that email protocols need improvement, but there are other benefits to gmail. I'm always going to want reliable service and storage, good spam filtering (and gmail's scale allows them to do that potentially better than anybody), and the interface is pretty good. I can even use my own domain.

Fastmail does all of that with the addition of great customer support, and I trust them a hell of a lot more with my data and privacy than I do Google.

From current experience, emails starting with - - in the subject line are still bypassing Gmail's spam filter entirely, months after the bug was reported. I rarely get spam at FastMail but it's classification rate has been near perfect.

I don't doubt there was a time Gmail was the absolute best at spam filtering, but I would contend that time is over.

Step 1 is a good backup. With JMAP that could be done for any provider, can you with gmail? The solutions that I've seen all seem like hacks. But I have yet to try it so I could be very wrong. But please let me know if there is a decent alternative, I'm about to migrate a google account.

I'd hope most decent providers let you use your own domain...

It's easy to think gmail spam filtering is good if you never check the spam folder. It has filtered out some very important mails for me and others so I do my very best to disable it. Create a filter for "is:spam" and "Never send it to Spam" is the best solution I know. But despite this google might remove mails anyway.

I've had good success using mbsync to back up a variety of IMAP provided email services. It's not perfect, and does require a little bit of fiddling, but once it's set up, it Just Works™ the way it should. I've even used it to move ~300k mails from Google to Fastmail.

I had a lot of custom spam rules on Gmail because Google takes the opinion of the masses to decide how to filter. Since FastMail's personalized spam filter kicked in, which is based on my opinion, I've never had to make hard rules for FastMail spam handling.

Another big advantage is that their spam data aggregation is optional, unlike Gmail's, and solely uses the content of confirmed spam (aka, you permanently deleted it) to refine their data, so no personal data is getting hoovered up with it.

I did actually shut off offering up my spam data to the masses on FastMail, because I realized I had an edge case where the filters defining my spam as spam for everyone could be harmful.

As far as backup, I have a local Microsoft Outlook which I open every so often which pulls down all the email. I wouldn't mind a Google Takeout-esque download feature from FastMail at some point too though.

The spam filtering isn't perfect (can't think of any false negatives, but some false positives. I hedged with "potentially" above) but I receive very little spam so I just check the folder when I get one. And this is with an old school catch-all domain setting too.

I haven't looked into backing up gmail.

As a Fastmail customer and standards enthusiast (been to a couple of IETFs myself) this makes me extremely happy. Kudos for following the standards path in this world of walled gardens.

I especially liked how they said the process improved the protocol.

> Clients that want to or need to (for example those doing PGP in the client) can still fetch the RFC5322 if needed. The message is represented by a blobId, and the raw bytes can be fetched using the same binary download mechanism as mentioned above.

This bit of JMAP has irked me - encryption is a second-class citizen, in a world rapidly evolving to need it every step of the way.

PGP/MIME / SMIME / Autocrypt and email is a chasm of insanity. You're constantly making choices about security vs usability and the user needs to be aware of them and yet the UX quickly becomes a maelstrom of icons and text and options and ...

Mailing lists are a big problem you have to solve and none of the solutions are very good.

That sounds like the perfect opportunity for a new email protocol like JMAP to solve.

Guess you're miss what JMAP protocol is trying to solve. It is not about content of the emails, but how emails are synchronized.

I haven't missed that.

I just believe, in today's atmosphere, synchronisation where encryption is a second-class citizen is a missed opportunity.

JMAP is for talking to the server. You don't want the server to have access to the plain text, only encrypted blobs, so that's the only thing that JMAP should give you. If JMAP dealt with providing the plain text, that means the server would be able to decrypt and that defeats the whole point of End-to-End Encryption.

Fortunately, Fastmail has a blog post that goes more in depth on the subject: https://fastmail.blog/2016/12/10/why-we-dont-offer-pgp/

What you want is something on top of emails, like autocrypt (https://autocrypt.org/)

When the email has been generated by some far off server, you want JMAP to apply a new layer of encryption on top of it while inside an HTTPS pipe?

That doesn't begin to solve the real issue of how to reliably encrypt from sender to sender across two servers it just makes the process even more needlessly complex.

No. But encrypted blobs are somewhat expected to fall within MIMEFiles.

> Clients that want to or need to (for example those doing PGP in the client) can still fetch the RFC5322 if needed.

Not a case of JMAP saying to the client, "Hey, here's this blob. It might be encrypted." But a case of asking the client to use a message standard they've replaced where possible.

That's what I mean by second-class citizen.

It is just a transport protocol. If email is SMIME/PGP encrypted the server will return for the email that has only one attachment with mime type "application/pkcs7-mime smime-type=enveloped-data". How this blob is processed is managed by the client if support SMIME or PGP. Do not forget email is a federated set of protocols, so the encryption cannot rely on servers, except if they are a walled garden as Proton for example.

I’ve got to say I’ve been very impressed with JMAP using it with FastMail for a few years now. Well done to the teams involved.

Wait, you can use JMAP with Fastmail currently?

As mentioned in the article, 30% of our customers have been switched to the new JMAP-based interface. We’ll resume migrating the rest in the new year.

For clarity, we’ve only had beta users (other than ourselves) on the JMAP-based interface for a few months. The previous interface (which has been in use since the start of 2012) was built on the precursor to JMAP, which works in a broadly similar fashion.

Got it, thanks! Looking forward to the transition (and even more so to 3rd-party JMAP clients!)

I just implemented an IMAP parser/server in Go. I found myself thinking (wishing) a lot for HTTP, JSON, stateless connections, etc. I'm happy to see someone has taken the lead in improving such an entrenched technology.

Hadn't heard of JMAP till just now and now I'm amazed it took this long for it to exist. It seems so obvious, in many ways.

Though I'm sure MIME will always be a bit of a trash fire to deal with.

Are there any other E-mail clients that support JMAP yet?

Doesn't exist yet. Hopefully, once the standard is finalized and has a server to offer public JMAP support will be adopted by email clients. I'm currently working on adding support for JMAP, but to have reference server I need to build and setup latest Cyrus from scratch which was a challenging experience.

It's a chicken-and-egg problem, no clients to support it because currently it'd be only a client for Fastmail, no servers for it as there are no clients.

JMAP client and the protocol impresses a lot. Just 1 to few calls, you can re-sync entire emails state in all folders. With IMAP need to select each folder to inspect its state. Moreover, just a few IMAP servers support fast synchronization extensions like QRESYNC or CONDSTORE.

Where are the JMAP clients?

Where are the JMAP servers? The protocol isn't finished yet.

The protocol isn't finished yet, but there are implementations - I believe the primary one is Cyrus: https://github.com/cyrusimap/cyrus-imapd

Yes, in Cyrus IMAP we take care to follow the latest spec. It's the version that drives the JMAP backend at FastMail.

The JMAP website has a list of other clients and server: https://jmap.io/software.html

Great but:

> The fact that “Message” meant too many things led to it being renamed as “Email”

No. No no no no no. 'Email' means even more things than 'message'.

1. Address: "What's your email?"

2. Synonym for 'send': "I can email you there. "

3. Message body: "Let me know when you've got the email."

I'm trying to understand your objection to this name a bit better, but I can't quite agree with the things you've stated as problems. You seem to be arguing for increasing the ambiguity where there really isn't any:

1. That's an email address, not an email. 2. That's a noun-ed verb - context generally makes it obvious when verbs are used rather than nouns. "I can email you" is a shorthand for "I can send an email to you". 3. That's fine - a mail client / server would never confuse these as the body is very obviously a property of most any representation.

In the same way that I'm not concerned that my postman has never delivered a fence to me, or written on a bulletin board system, I think we can cope with this name being the appropriately sensible one.

Have I misconstrued your argument?

You have, but I don't really know how I could explain it more simply. I'll try though using your analogy.

Your postman doesn't deliver fences to you, he posts post to you. In a similar way, instead of using the word 'post' all the time, a sensible design for an API would want to have a person deliver a message.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact