
JMAP: Like IMAP but Not Really - jasonmunro
https://unencumberedbyfacts.com/2019/01/24/jmap-its-like-imap-but-not-really/
======
bad_user
FastMail is behind this protocol and from what I've read JMAP has evolved out
of their web interface.

I've been a happy customer, even though lately I flirted with going back to
GSuite for my personal email, but after a trial realized that Gmail does many
things well, except for being a good email service. So I went back to FastMail
and renewed for another 2 years.

Seeing this new protocol is exciting, because JMAP is being standardized at
IETF. A breath of fresh air to see a new standard being developed.

Also from what I understand JMAP should be friendly for mobile usage. They
kept notifications out of it, you're supposed to implement notifications using
whatever the mobile platform provides. Interacting via JMAP is via plain HTTP
requests, which is super cool.

I can totally see myself implementing a simple email client for automating
online services. For example if you implement a commenting system for a
website, you might want to do replies by email. That would be a cool project
for me to try out.

I wonder if FastMail exposes JMAP publicly yet. Haven't seen any mentions in
their admin or docs thus far.

~~~
eggsampler
I've been a happy Fastmail customer too, until I was made aware that you can
impersonate other Fastmail customers by just spoofing the email address. Their
servers just happily accept it. SPF and DKIM all pass with flying colours, and
the only way you'd know it's happened is if you have DMARC on and happen to
notice a pass in the report you don't remember sending. Well, that is if the
recipient doesn't reply to the spoofed message - hope the damage wasn't
already done though. It's effectively impossible for the recipient to know
it's been spoofed.

The worst part is I think Fastmail is aware of it and just don't care (believe
that's why they mark their emails with a green tick and text). I understand
that email has never been really authenticated, but this just throws any trust
I had in Fastmail out the window.

I will be evaluating other mail hosts at the end of my subscription.

~~~
navlelo
What is the reason for allowing this? Laziness?

~~~
bad_user
The reason is probably that nothing can stop the successful spoofing of the
From header. DKIM is a signature for authenticating a domain, however that
domain does not have to match the domain in the From header.

Take a look in Gmail at a signed email and you’ll see a “Signed by” field in
its header info, with a domain name as a value.

Also the SPF setting has nothing to do with the From header either.

In other words the “From” value cannot be protected, unless you sign your
email with PGP or S/MIME.

~~~
richardwhiuk
That's tripe.

They know who authenticated to the SMTP server, so they could enforce that the
From address is who it was authenticated by. Otherwise, they basically act as
an open relay.

~~~
sitharus
Sending from multiple From addresses is a common use case. I send from at
least 4 different email addresses all hosted by fastmail in the same account.
Having to create different logins to authenticate each sender would be a huge
pain.

Plus it's not a unique problem to fastmail.

~~~
shawnz
Gmail requires that you prove ownership of an email address by clicking a link
in an email before letting you choose it as a From: address. I think this is a
good compromise.

~~~
regecks
You can also take a blacklisting approach, where it's open-by-default and
users shall be able to restrict any domain to properly authenticated users
only. That way, it is purely a security enhancement for those who want it
(like me).

I demonstrated this behavior to eggsampler after discovering it quite a long
time ago by messing around with HTTP payloads in their web interface - it's
wild to me that FastMail will use the DKIM private keys from an entirely
different FM account to sign your messages.

Unlike eggsampler, I won't be ditching them, but I hope that FM reconsider
their policy eventually. That they have awarded themselves the privilege of a
"green tick" on their own official emails while throwing everybody else to the
wolves is slightly ironic.

------
pedrocr
_> JMAP is a REST API so it uses HTTP requests and responses to issue commands
and get the results. Almost all requests in JMAP are to the same URL using an
HTTP POST to submit a JSON body of “methods”._

Describing this as REST is really strange. Defining your own operations over
an HTTP POST is what SOAP and other RPC style web services do and specifically
what REST isn't. But I guess that a lack of a standard behind REST ended up
with the term being used for everything.

~~~
bradleyjg
At this point I guess any API that runs on top of HTTP is REST? Strange world.

~~~
mbel
I guess REST became catch phrase for all JSON over HTTP protocols.

------
hardwaresofton
I really want to see some innovation in the email space. The landscape is like
a sea of false promises and dashed dreams. SMTP is one of the bread-and-
butters of the internet, yet it just doesn't seem to be moving forward (maybe
that's for the best), and no one's building extensions on top of it.

Maybe I'm naive in thinking it was possible but we could have avoided this
whole "make an account on X messenger so we can talk", if someone had just
jammed XMPP and SMTP together.

Every few years I'm tempted to try and write a SMTP server (MX) myself but
then realize that postfix has been around so long and is the go to choice _for
a reason_. I don't have 20 years of figuring out how X mail server interprets
SMTP to be as reliable as postfix. I settled for trying to wrap it[0].

[0]: [https://gitlab.com/postmgr/postmgr](https://gitlab.com/postmgr/postmgr)

~~~
PaulHoule
Email is "good enough".

The real hole is "instant messaging". Somehow every few years we got from ICQ
to AIM to Paltalk to Skype to Facebook Messenger to Slack to ... (at least
Altassian had the decency to take HipChat behind the woodshed and shoot it)

The strange thing is that these services dry up, blow away, get replaced, but
they don't seem to improve on what came before.

There is a standard, XMPP, but the only people who care about it are
firefighters, cops, and soldiers. For all the anger at internet giants these
days, I can't see for the life of me why people aren't pushing for open
instant messenging.

~~~
kitsunesoba
XMPP’s problem isn’t that people don’t care about it. Its problem is a double
whammy of having zero branding/mindspace presence and being too technical for
many users to bother with.

The second point is especially big. People expect to be pointed to a singular
app/site, not “choose whichever client you like”. Your average Joe just
doesn’t conceptualize services as being separate from clients in the first
place, and even if they did having to choose a client is a dead end.

~~~
yo-nathan
I totally agree with that. I've good experiences recommending Quicksy[1][2]
though. It makes not only account creation easy as pie, but also helps people
finding their peers by their phone numbers.

Give it to your average Joes and Janes and they will just use it like any
other messenger. But since it is pure XMPP, can just use any other Jabber
account to chat with them. :)

I recommended it a lot the last weeks and most of my pals just use it without
any hassle. At least in my bubble most of the people have besides WhatsApp
also Telegram or Viber or WhatEver installed. So it's for them Quicksy is just
yet another app. But this time one, which helps to get out of the walled
garden. :)

[1]
[https://play.google.com/store/apps/details?id=im.quicksy.cli...](https://play.google.com/store/apps/details?id=im.quicksy.client)
[2] [https://quicksy.im](https://quicksy.im)

------
ggm
The main failure in mail standards is a lack of explicit utf8 clean support
lhs@rhs -if this got fixed (it's often called universal acceptance) a lot of
things about mail as an ecology would improve.

I have view on the spam thing. The whole "your idea will not work because"
meme is hugely destructive of innovation in email. It sucks energy and
mindshare. It's classic old timer put down.

What would (imnsho opinion) have fixed spam is sender pays. I've debated this
with a lot of people. We're 50/50 on it. Fifty agree with me, fifty million
don't.

Jmap is utf8 clean btw.

(I used to do email for a living in the eighties when life was simple and
bang!chains!worked)

~~~
gsich
You could pay with computing power.

[https://en.m.wikipedia.org/wiki/Hashcash](https://en.m.wikipedia.org/wiki/Hashcash)

~~~
jannes
That's horrible for the environment.

~~~
bjoli
Depending on the workload it will be a marginal load, and getting rid of spam
would be a pretty big gain.

------
arendtio
To give some context on why JMAP is relevant today, you might want to read
this answer in the conversations FAQ [1]. It basically states that XMPP
powered Apps are having trouble fighting against the battery saving software
because XMPP is stateful. JMAP, on the other hand, is stateless.

As protocols are meant to connect different implementations and bringing more
protocols for the same task quickly hurts the interoperability (before
bringing some improvements in the long-term), I am skeptical to the effects
those new protocols bring to the ecosystem. I am aware that JMAP was born more
out of the RESTful requirements of an HTTP based App, but in general, I am
wondering where this road will lead us.

[1]: [https://github.com/siacs/Conversations#but-why-do-i-need-
a-p...](https://github.com/siacs/Conversations#but-why-do-i-need-a-permanent-
notification-if-i-use-google-push)

~~~
Leace
Note that "RESTful" in JMAP's case means "everything over JSON HTTP POST",
methods to be invoked are embedded in the JSON request.

As for XMPP battery consumption I didn't see major problems, Conversations.im
is always <1% (I just checked and it shows 0%). On the other hand
Conversations can use push to optimize battery usage.

~~~
arendtio
Yes, in fact, there is absolutely no reason to restrict well-built apps like
conversations from consuming battery automatically. Even without Google Push
Notifications the battery consumption is quite good.

So it is pretty obvious that the vendor built battery optimization software
does more bad than good.

------
billpg
Last time I looked at IMAP, I was a little peeved that servers didn't accept
messages added to the "Outbox" folder and send them, instead requiring that
clients talk SMTP and then (or not) save a copy of the same message to the
"Sent" folder.

Could JMAP replace SMTP as well as IMAP?

~~~
akvadrako
Why do you think you need a new protocol for this? Instead what you want is a
different IMAP server.

~~~
billpg
It has been a long time since I looked at IMAP. Are servers that do this now
commonplace? (None I could find used to.)

Can I know from a capability exchange if the remote server will send emails
after adding to the "Outbox" folder?

~~~
amaccuish
No, not as far as I'm aware. But it's an idea I've also had. Upload the
message to drafts, and then when you're ready to send, just move it to Outbox
right?

~~~
billpg
That would be ideal. Keep SMTP as a server-to-server protocol only and IMAP as
server-to-client.

------
akie
The article mentions that JMAP does calendars as well. Anyone have a quick
status update on how it compares to CalDav and if it's supported by any of the
major calendar clients or servers? I did a quick google and looked at the JMAP
site but couldn't find any info.

Background for this is that I implemented a calendar client and a calendar
server (with SabreDAV) last year and I'm wondering if I should be adding
support for JMAP anytime soon.

~~~
nmjenkins
Editor of the JMAP specs here. The JMAP calendar spec is currently just a
draft, but now JMAP core and mail are (more or less) finalised and JSCalendar
([https://tools.ietf.org/html/draft-ietf-calext-
jscalendar-11](https://tools.ietf.org/html/draft-ietf-calext-jscalendar-11))
is in last call too, we will be looking to take the JMAP Calendar spec through
the IETF process in the very near future. Since it is essentially just combing
core with the JSCalendar data format, this should be reasonably straight
forward.

~~~
akie
I see, thank you so much for providing inside information. Looks like it'll be
a while before I can ditch CalDav...

------
aorth
Post is about JMAP (Fastmail's IMAP/SMTP replacement), by the author of a
newish webmail client called Cypht that has just recently implemented support
for JMAP. Cool! Keep going!

[https://cypht.org/](https://cypht.org/)

------
bobm_kite9
Loved this bit:

> Internet: Do you think JMAP will really take off?

> Me: JMAP is an open, smart, modern, and powerful E-mail protocol, so
> probably not.

------
mverwijs
I've started to migrate 18 years of email history to Fastmail, after running
my own mail infra for almost 2 decades.

I am not impressed. Folders moved in the webinterface show up unmoved in
Thunderbird. And vice versa.

Sometimes it works, sometimes it doesn't. They're investigating the problem at
the moment, but the updates they've given me don't give me much hope.

I'm looking at migrating away to another provider that just does plain old
imap.

------
mschuster91
From experience, Exchange's Outlook API ("EWS") is pretty decent. It's XML-
SOAP, sure, but there are libraries for it that can be readily used. The only
thing they _did_ fuck up is the three different kinds of IDs for an object
(esp. confusing when accessing delegated team mailboxes/calendar events) but
once you get it how it works, it's straightforward and allows you access to
anything from email over calendar to address-book.

I just do not know if EWS is an "open standard" that could be replicated by a
third party server or if it is only "open documentation" and licensed for
Exchange only :(

~~~
amaccuish
I've sadly been looking for, and never found, an open source EWS <->
IMAP/(Cal/Card Dav) gateway. I'd love it if my users could get a full
experience using Outlook for Mac for example. Right now I recommend them to
just use Apple Mail/Calendar/Contacts or Thunderbird with the tb-sync
extensions.

~~~
mschuster91
So you'd like for users to keep their familiar Outlook setup, but with a FOSS
server backend?

Not sure if this is possible, I don't know if Outlook uses the EWS protocol,
OWA or something entirely different :/

~~~
amaccuish
Yupp. It works well with Windows Outlook, since they can use ActiveSync. But
Mac Outlook doesn't have ActiveSync (or Card/Cal dav).

------
zAy0LfpBZLC8mAC
> JMAP is not designed around a persistent network socket, so it’s perfect for
> webmail clients that connect, do stuff, then disconnect (which is exactly
> NOT how IMAP is supposed to be used)

WTF? So, webmail was limited by the fact that it was running in a browser and
thus had to use HTTP, which has semantics that don't really fit the needs of
an email access protocol. Now, we do have stuff like websockets that would
make it possible to run a protocol from the webmail client that actually fits
the needs, and instead people invent a new protocol that inherently doesn't
match the needs of the application?

~~~
majewsky
Most mail clients are running on mobile (judging by number of devices). Long-
running connections on mobile are a bad idea because of network reliability
and battery concerns.

~~~
zAy0LfpBZLC8mAC
Hu? Constantly establishing new connections saves power vs. maintaining one
estalished connection? Could you explain how that works?

And could you also explain how constantly making new connections makes things
work more reliably over unreliable links? Like, does that allow you to
transfer data when the network link is down? Does the fact that inside the
TLS/TCP connection data is transferred via HTTP instead of IMAP somehow make
the TCP connection work better over lossy links?

It would seem to me like the exact opposite should be the case, if it has any
effect at all?

~~~
toast0
Ignoring efficiency concerns, modern mobile oses don't allow applications to
keep connections open for long. If you want notifications to work, you need to
use the platform notification system.

The platform argument is roughly that if they hold open one connection, it's
cheaper than each app holding open their own connection. And maybe, if you're
optimistic, the platform will be better at figuring out ping intervals for
that connection that adapt to broken Nat devices per network (ie: on some
networks, wake up once a minute to keep the connection active, and on
reasonable networks wake up once every 30 minutes)

~~~
zAy0LfpBZLC8mAC
Well, yeah, if you are very optimistic and generous. In reality, this is yet
another way of the manufacturers maintaining power over their customers'
property. After all, there is zero reason why any of those things would
require them as a middle-man, they could just invent a protocol for that
and/or provide a library and a system service on the device that allow
applications to maintain a background connection without the need to involve
them in any way on the network/server side. The only useful involvement they
could have might be providing endpoints for measuring and exchanging NAT
timeout information for providers, maybe.

But in any case, none of that is an argument for building a protocol that
forces the problems that result from that on all platforms that don't have
such restrictions, at best that is an argument in favor of workarounds to make
things work as well as possible on such platforms.

Also, I would think that the long-term solution to this problem should be
mobile link layer protocols that allow the mobile station to receive incoming
packets with little delay while still conserving power. Stuff like high-
powered, high SNR alert channels that allow the mobile station to shut down
everything apart from a simple, low power, receiver, that only powers on for a
few microseconds every few hundred milliseconds to listen for a wakeup signal
from the network, so that incoming packets can be delivered with a latency of
a few hundred milliseconds at any time. And then you are stuck with a stupid
poll-based protocol that's adapted to the limitations of some ancient
technology as a replacement for an even more ancient protocol that didn't have
that limitation.

------
andridk
I was really hoping for a GraphQL API standard for mail before reading this.
But this sounds good too.

~~~
thomasfoster96
Reading through the JMAP website [0], I'm actually wondering whether whoever's
behind JMAP has looked at GraphQL. There seem to be a lot of similarities
(especially the "why not REST" section).

[0] [https://jmap.io/index.html](https://jmap.io/index.html)

~~~
ForHackernews
I think Fastmail are the primary ones behind JMAP:
[https://fastmail.blog/2014/12/23/jmap-a-better-way-to-
email/](https://fastmail.blog/2014/12/23/jmap-a-better-way-to-email/)

~~~
jasonmunro
Yep. Also I think they are using it for a subset of customers. While JMAP
support in Cypht is still new, so far it's working just like IMAP, but my
testing is limited since I just wrote the integration :)

------
nothrabannosir
Very interesting:

 _> JMAP is not designed around a persistent network socket, so it’s perfect
for webmail clients that connect, do stuff, then disconnect_

This would allow implementing JMAP in serverless architecture, e.g. self
hosting on AWS lambda. That should be _very_ cheap and significantly lower the
barrier to self hosting (not needing to manage a box, just providing aws
credentials).

Think of what it takes to truly decentralise email. There are technical
hurdles, but also some fundamental ones:

\- other providers mark you as spam because you’re unknown

\- you need an always on server to actually get your incoming mail

This would at least solve the second problem. If we can develop a product like
mailinabox (Which has its flaws but it’s the right idea), but instead of
asking for a fresh vm, it just asks for aws credentials, that could be pretty
solid.

Hopefully one day we can give the people back their control over their means
of communication. This seems like a step in the right direction!

~~~
elcomet
I'm not sure I understand: JMAP is designed for communication between the
client and the mail server. But you still would need a mail server, always on,
to receive the incoming mail from other servers, right ?

Did I miss something about JMAP?

~~~
jasonmunro
Nope, you are exactly correct. JMAP is just a much more modern and efficient
way to connect clients and servers to access E-mail.

------
shittyadmin
So, you say that it doesn't maintain a persistent connection, but that poses a
problem - it needs a way to do push notifications still I assume? Those need
an open socket somewhere - does JMAP allow for it or does it make you rely on
a 3rd party?

~~~
Mailtemi
If you are mention Ios/Android, yes you need the proxy to redirect push calls
from JMAP server to the push services of apple/google.

~~~
inputmice
FWIW this is completely reasonable. Push in XMPP works the same way. They way
FCM (Google) and APNS (Apple) work is that only the app vendor with proper
credentials can trigger push notification for an app. Therefor it has to be
proxied through the app developers server anyway. And of course it cleanly
separates the MDA from the various, proprietary push solutions.

------
youdontknowtho
IMAP is just a way for attackers to perform credential spray's against large
providers these days.

------
busfahrer
I recently thought about email being based on such an old standard and then I
wondered if maybe it didn't matter anymore, because most of the web is moving
to gmail, and my guess is (and somebody please correct me) that when I send an
email from gmail to gmail, that no actual SMTP is involved? I'm assuming that
they have some internal protocol that is just adapter'd to "Email" at the very
end?

~~~
pjc50
> most of the web is moving to gmail

This isn't true; there's a LOT of industry on Exchange of some sort, and
plenty of older institutions running their own email. Especially in IETF land.

And it's a huge risk increase if everyone moves everything to gmail.

