
Making email more modern with JMAP - mfsch
https://fastmail.blog/2019/08/16/jmap-new-email-open-standard/
======
jivings
It will be interesting to see how this develops.

I've spent the last month implementing and testing IMAP support for my app
[1].

It has... not been very fun.

Before we were limited to providers with a REST API, so implementing IMAP
should in theory allow us to support a much wider range of email servers with
a much lower support burden.

Although the protocol spec is tight, it seems like some providers randomly bug
out or hang [2], respond with custom error messages, offer bespoke
functionality [3], or generally don't comply with the spec in breaking ways
[4].

As a result I'm more worried about this going forward than I had hoped.

I'll now also happily support anything that can help modernize email, so I'll
apply to the developer program and will be watching JMAP closely.

[1]: [https://leavemealone.app/](https://leavemealone.app/)

[2]: Yahoo Mail always down ([https://downdetector.com/status/yahoo-
mail](https://downdetector.com/status/yahoo-mail))

[3]: [https://developers.google.com/gmail/imap/imap-
extensions](https://developers.google.com/gmail/imap/imap-extensions)

[4]: [https://github.com/mscdex/node-
imap/issues/775](https://github.com/mscdex/node-imap/issues/775)

~~~
mattmanser
Having worked with IMAP before myself, the spec is very vague a lot. It's a
bad spec.

We had our own IMAP server implementation and had all sorts of problems even
between different versions of outlook.

Each project had an inbox which you could cc in, resulting in huge inboxes
(for the 2000s at least). You could also drag and drop an email you'd received
into the project folder to share it with colleagues. And we had mysterious
disappearing emails, but only at some clients.

One (of the many) problems I remember is that messages could have an ID, but
the spec declared that it could change between sessions, so one version of
outlook stored that in a short uint (max 65535) because why would you need
more for a temporary id. So have fun if you sent anything more than that. And
of course the client had different versions of outlook on different sites so
it took ages to even figure out what was happening.

Another was that you had to make several back and forths on every request,
messages about the messages, to say the payload and number of messages, but
the spec was gleefully vague about the order this should happen in. So back
then what worked for one email client resulted in an empty inbox for another.

I don't think we ever got it working properly with Firebird.

~~~
dboreham
Outlook is known to have a horrible IMAP implementation.

~~~
amaccuish
I've found Outlook's not too bad actually, and at least they support
ActiveSync and MAPI for desktop and mobile. I always had the most problems
with Gmail over IMAP.

------
Sir_Cmpwn
Though I have no complaints about the way they went about implementing it
(open standards hooray!), as a mail client author and someone with lots of
experience with mail in general - I'm NACK to JMAP. I think that it comes with
some questionable design decisions (HTTP? JSON?) which on their face I imagine
look fine to the typical HNer, but in many ways is a regression from IMAP. I
would prefer to see the limitations of IMAP addressed with a few carefully
written extensions to the spec - which is already extensible and has a
thriving set of useful extensions in the wild.

By no means is IMAP a particularly good protocol, but it is extremely broadly
supported and designed more with email in mind than JMAP appears to be, which
seems to focus more on bringing shiny web tech to email regardless of its
utility in such a context. It's also one of many closely interlinked mail-
related specifications, and JMAP fails to integrate well with many related
specs imo.

~~~
tjoff
JMAP is literally the _only_ positive thing that has happened to email in
decades, please don't dismiss it (or do you know of anything else in the works
that has any chance of success?).

HTTP and JSON are such an insignificant price to pay for the sliver of hope
that email can be saved and not being in complete control of google et al.

~~~
Tharkun
The real positive thing that needs to happen to email, is putting an end to
near-monopoly of gmail. Email is great. It's decentralized. You can simply
roll your own. You can use whatever client you like. Apparently even using
HTTP/JSON.

But you can't send mail to anyone using gmail, which is nearly everyone,
unless you play by Google's rules. And even then they'll still put you in
their spam folder, or just randomly blackhole you without any explanation or
recourse.

JMAP vs IMAP vs POP is moot as far as I'm concerned. As long as my mail client
continues to work, I'm good. If this gains any traction, I'm sure dovecot and
the likes will add support for it (if they haven't already). But no amount of
protocol fiddling can fix the fundamental issues of email centralization.

~~~
tjoff
That's what I meant. And we need something better than IMAP/POP to have a
fighting chance in challenging google. Because without it IMAP will, without
doubt, loose market share even more - further cementing the dominance of
larger players.

That is why protocol fiddling is an essential part of it. Not enough, but
without something better than IMAP might as well give up yesterday (as most
already have done).

~~~
Tharkun
> we need something better than IMAP/POP to have a fighting chance in
> challenging google

How's that? I don't see how messing with IMAP/POP/JMAP will make a difference
there. Mail _delivery_ , over SMTP, is what the Gmails and Outlooks of this
world are so nasty about. They claim to protect their users from spam, but
small mail servers get killed in the crossfire. Mail clients downloading mail
over whatever protocol is a trivial problem by comparison, and they have zilch
to do with SMTP abuse.

~~~
tjoff
Yes, but it is all about market share.

If 99.9% of all mail originate from gmail+outlook then they can (and will)
pretty much ignore the rest.

The less dominant they are the more they are forced to be reasonable.

JMAP will help diversify the market as well as simplify the process of
creating competing products. And has the potential to vastly increase demand
for native clients and proper support from webmail vendors.

All of the above will work towards lessen the dominance of gmail etc.

It isn't anything by itself but I believe it is a _requirement_ going forward.

Because IMAP doesn't cut it, so everyone has to create their own crap, which
won't be as "good/easy/cheap" as gmail.

------
inputmice
Pretty cool protocol. I've been following this for a couple of years. I had
already written a library for an earlier version of JMAP (back then references
worked differently / didn’t exist); and it was interesting to see how that
improved in the IETF process. I wrote a library [1] and a POC email client for
Android [2] earlier this year. It takes a moment to fully understand a now
fairly complex protocol but when you get the hang of it it becomes very
powerful.

Sadly the server support isn’t really there yet. The support for Cyrus hasn’t
been released yet (you need git) and some vital functionality like push [3] is
still missing. Also no word from Dovecot yet.

[1]: [https://github.com/iNPUTmice/jmap](https://github.com/iNPUTmice/jmap)

[2]: [https://github.com/iNPUTmice/lttrs-
android](https://github.com/iNPUTmice/lttrs-android)

[3]: [https://github.com/cyrusimap/cyrus-
imapd/issues/2714](https://github.com/cyrusimap/cyrus-imapd/issues/2714)

~~~
Leace
Do you use Lttrs daily? I'm wondering if it works in practice (even when using
non-standard setup) or if it's just a proof-of-concept (that'd mean it may
take some time to mature).

~~~
inputmice
No I don’t use it daily. It can’t even send emails yet and you have to set
your login credentials at compile time. I got frustrated having to deal with
HTML emails and now I’m waiting for the servers to mature a bit. And honestly
also for dovecot to announce support. I mean I love what the fastmail+cyrus
team is doing; but if Dovecot is not going to support it, it is probably not
worth it to build a JMAP-only MUA.

------
psim1
I'm a believer in open protocols, but in recent years, I'm also loving
Exchange and ActiveSync. I know that's not going to be a popular opinion here.
To be honest, I don't know much about how the magic works, but the
configuration is simple, the push is reliable, and everything just works. It
doesn't drain my mobile and it doesn't lose its indexing and download for
hours like IMAP always seems to. And besides, it's mature -- Exchange and EAS
have been around for many years.

So here's to JMAP becoming the open standard that replaces that.

~~~
znpy
Yes, but I can’t tun it on my own low hardware at home for free.

~~~
amaccuish
Sure you can, I use z-push [0] integrated with Kopano which works great. You
can also use z-push with IMAP, CalDAV and CardDAV. The only issue I know of
using IMAP is that the z-push server polls your mailbox, so it's not true
push, but vastly better than the battery drain of IMAP IDLE.

[0] [http://z-push.org](http://z-push.org)

------
jchook
I dislike the details of JMAP’s design.

While I appreciate the value it brings, I can see the 5 years of struggle in
the sheer size of the multiple specs needed to define some pretty rudimentary
mail client/server functionality (retrieve, send, push, pagination, etc).

At some points they use JSON (a data format) to describe procedures... like a
programming language written in JSON. It feels a bit like using a screwdriver
handle as a hammer.

How did we end up with JMAP instead of something simpler or modular,
especially with GraphQL and other similar innovations on the radar?

~~~
zeveb
> At some points they use JSON (a data format) to describe procedures... like
> a programming language written in JSON. It feels a bit like using a
> screwdriver handle as a hammer.

And folks laugh at me for suggesting S-expressions until I'm blue in the face!

Seriously, this is one of the areas in which they _excel_ (ref.: literally
every line of Lisp ever written). They're a pretty nifty data format too (cf.
[https://sites.google.com/site/steveyegge2/the-emacs-
problem](https://sites.google.com/site/steveyegge2/the-emacs-problem)).

------
mfsch
From what I gather it should be possible to use 2FA with JMAP. I‘m looking
forward to no longer having to decide between using third-party email clients
and properly securing the account that’s probably most worth securing.

~~~
Freak_NL
Would WebAuthn support be possible with JMAP? That would really move things
forward.

~~~
tialaramex
One high level problem that jumps out at me is: When do we authenticate?

For web sites WebAuthn drops nicely into the existing login flow. The user
knows what they're trying to do is log in, and so when they're prompted to
touch the button or whatever it makes sense in that context.

But today most email apps just silently update all the time. So are we
authenticating when the app starts and then re-using some credentials obtained
to re-connect as necessary? Is that safe? Even if that's all we do (which
might be safe) it means the user gets prompted for WebAuthn (to press the
button) when the mail app starts, even if they aren't currently interested in
email. If they dismiss it, when do we prompt them again? Because meanwhile
they of course don't get any new email.

~~~
zaarn
OAuth2 can handle this; you authenticate the mail client with an offline token
that's valid for a long time. The mail client can refresh their tokens
regularly and use a short-live token to authenticate their JMAP requests.

OAuth2 doesn't particularly care _how_ the token is obtained, so it can handle
any arbitrary authentication flow, including WebAuthn.

And it's a widely supported protocol; there are libs for C++, Java, Go, JS and
others, so it should be easy to integrate.

------
aquabeagle
Fastmail should throw some engineering resources towards implementing JMAP
support in some mail clients like Mutt.

~~~
stonogo
The fastest route to MUA support is compatibility with existing MDAs like
procmail, maildrop, and the like. At the very least a JMAP client that
populates a local Maildir would get you Thunderbird, Evolution, mutt, and
more.

~~~
namibj
Could a fuse filesystem do the trick for maildir emulation? Assuming that'd be
beneficial to a full maildir, but I guess it'd only be so at large sizes.

~~~
marmaduke
Couldn't à hypothetical fuse fs emulate any file based store for email?

------
rounce
One thing I find weird (probably because I’m new to JMAP and still ignorant)
is having _position_ AND _anchor_ + _anchorOffset_. Why have both? Surely the
latter is sufficient and safer (as any _position_ is only guaranteed to be
valid when the last response was written)? From a glance of the spec I didn’t
see any explanation of why. If anyone could shed more light it’d be much
appreciated.

~~~
chrismorgan
Both are necessary because you may not have the necessary IDs.

anchor + anchorOffset is useful when you use pagination and new entries might
appear at the head of the list but you don’t want them to mess with your
pagination once you’re inside.

position is useful when you have a total count and wish to jump in at the
middle; as a concrete example, when you use lazy loading instead of
pagination. For example, if I have a mailbox of 100,000 emails, and jump to
the bottom: I don’t want to need to fetch all the IDs (probably well over 1MB
to fetch), just so that I can specify an anchor. I want instead to ask “give
me the thirty messages from position 99,970”.

(Note that when I speak of lazy loading I’m _not_ talking of infinite
scrolling, but rather of when you know that you have N messages, and that each
message is allocated M pixels of vertical space, so you can allocate an exact
space, and provide a proper scrollbar. This is how Fastmail does it and how
desktop email clients have historically done it.)

------
wruza
I read imap description long time ago, and I know what it does in principle
(along with issues itt), but can someone explain what value does it serve
today? I have two confusing thoughts.

First, imap seems to be an overkill for a line wider than 64kbps. Texts do not
consume much traffic and an entire mailbox can be served easily to everyone.
There are some huge mailboxes out there, but messengers have this problem too
and they solve it somehow (I suspect trivial &last_date=... from programming
tg bots).

Second, and this may be due to bad clients (haven’t seen good however), imap
is just sloooow. It doesn’t update instantly, but instead I have to enter
every folder and pull down with my finger to update it. It can show outdated
messages which I deleted on a pc _for ten seconds_ before it figures out that
they were deleted. My guess that it has something to do with an infinite
number of roundtrips which this protocol is based on. Occasionally it shows N
unread when there is none. Even push notifications fail to work properly most
of the time (same server-side issues?) And with all these complications, I
still have to download every attachment by finger. How come that heavy
sites/forums/boards/messengers always instantly show new data and mail cannot?
I know that I can simply use webmail, but wtf?

I’m a developer, but when I see software, I always try to evaluate it from
user-only perspective. And from that, mail tech seems simply incompetent. I
wish there was a port smtp->tg and tg->smtp, and special per-subj chats, so I
could leave all that legacy for someone else to deal with and just talk to
them without missing messages or _waiting_ for updates.

(Not even considering mail message formatting, spam mismatches, size
restrictions and non-delivery laws)

~~~
chrismorgan
The slow mail client issue you’re describing is very probably an artefact of
how IMAP IDLE is designed; JMAP fixes this design, as explained at
[https://jmap.io/#push-mechanism](https://jmap.io/#push-mechanism).

------
thelittleone
"Modern", the darling adjective of startup lingo. We thank you for your
service and efficacy. It's time to abdicate the throne. All hail "more modern"
the rightful heir, protector of the realm.

Pre-modern -> modern -> more modern -> post modern?

Jokes aside I enjoyed the experience of Fastmail with the exception of the
migration tools. Very responsive. Kudos for submitting JMAP as an open
standard.

------
gumby
I believe the filtering is for user-initiated search, right?

Would be nice to have some standardized support for sieve-like server side
prefiltering. That's how I sort my mail into mailboxes, send spam (labeled
elsewhere) to the spam trap, and the like.

~~~
eridius
What would this support look like, in the context of JMAP? By the time the
email is in your inbox it's already gone through server-side prefiltering. So
the only integration I can imagine is a way to fetch and store Sieve scripts
so you can edit them from your mail client, but that seems like a very
provider-specific thing.

For example, Fastmail themselves support Sieve, except they don't give you a
single editable script, instead they convert your structured rules and spam
settings into 8 labelled sections in a Sieve script and give you 4 free-form
edit boxes to insert your own code at specific points (so you can't edit the
code that they synthesize for you). I don't know how a protocol like JMAP
would expose this in a generic fashion.

~~~
gumby
Two mechanisms: 1, the ability to write arbitrarily nested if all/any/none of
various parameters (any/specific header w/restricted reflex) for server-side
searches. The Mac mail app has these for the client side, as does my rss
reader. A simpler grammar than needing _inMailboxOtherThan_ yet more powerful.
Could be represented as a json tree. 2 - the ability to run that filter n
incoming mail basically where sieve is run today, with a similar small grammar
for specifying a small set of commands to run like file in mailbox, tag, etc.

------
chiefalchemist
I didn't see anything about security. I understand sending is outside the
scope. But if storage and downloading is to be portable, doesn't encryption
have a place?

~~~
bad_user
The encryption of the connection is orthogonal. That's what HTTPS is for.

And client-side encryption is managed by solutions like PGP. Doing it on the
server-side is incompatible with the email protocol IMO.

~~~
chiefalchemist
So server side is secured how?

~~~
bad_user
Not sure I understand the question. What's your threat model?

~~~
chiefalchemist
On the server, the emails are all encrypted like say passwords? Or are they
stored as text?

~~~
TazeTSchnitzel
Passwords aren't encrypted, they are hashed.

Every email service out there stores email in what by password standards is
plaintext, but hopefully encrypted at rest.

------
verisimilitudes
So, how long until everything is JSON-shit over port 80 and how long until
port 80 is old news and only port 443 is used for anything and everything? I
can't claim to be familiar with IMAP, but it can't be worse than this garbage
that requires an HTTP client and a JSON parser.

To process JSON, you must inspect every character, which is reason enough it's
an awful ''format'' that only an ignorant could hold dear. Douglas Crockford
brags about how smart he is for disallowing comments, but there's whitespace
allowed everywhere. Let's not forget this asinine integer nonsense, because
JavaScript lacks real integers.

I'm sure most people think HTTP clients and JSON parsers are standard-library
affairs, nowadays, though; let's just entirely ignore matters of licensing and
whatnot.

It's disgusting to see this as an IETF standard, but people have been
complaining about how that moves along for decades; take a look at Google's
robots.txt ''standard'' that doesn't do anything useful and effectively makes
every existing file standard, rather than do much of anything worthwhile. The
WWW and JavaScript are nothing more than a continuation of this worse-is-
better filth that UNIX and C previously championed; decades ago, the IETF
revised the RFC for the Finger protocol and warned about security, solely
because some idiot wrote a shitty Finger server in C; now, we have useless and
unnecessary standards such as this that only serve to further this next
generation of idiocy.

I wonder when all of this will finally collapse under its own weight.

~~~
jimbo1qaz
Can you provide some actual arguments as to what's wrong with JSON? It's a
text format that can be inspected by humans, unlike binary formats. (Maybe
IMAP is a text format too, so this isn't an advantage inherently) And JMAP may
have better semantics than IMAP, and there's nothing inherently wrong with
JSON (other than edge cases).

~~~
verisimilitudes
>Can you provide some actual arguments as to what's wrong with JSON?

It's a wasteful, textual format derived from a programming language vomited
forth in under a week. It has asinine integer characteristics for no good
reason; it's not a random-access format; validating it is even more work; hash
table keys that appear multiple times, instead of being in error, instead
overwrite each other, defeating most any kind of optimization; it is thus
wildly inefficient to manipulate, as manipulating it correctly in all cases
requires full and comprehensive parsing with checking. It is an asinine
''format''.

>It's a text format that can be inspected by humans, unlike binary formats.

Even then, there are much better textual formats that could be used, but I'm
of the opinion that numerical (binary) formats are strictly superior,
especially as they're used increasingly often. It's better to have good tools
that will provide a human-readable representation, rather than pretend a
single, inefficient representation for both man and machine is somehow the
best option.

>(Maybe IMAP is a text format too, so this isn't an advantage inherently) And
JMAP may have better semantics than IMAP, and there's nothing inherently wrong
with JSON (other than edge cases).

Computer programs written by anyone competent are built around edge cases;
that's much of what any reliable program is, accounting for edge cases. To
disregarded edge cases, of which there are many in JSON, speaks to me that you
have little concern for writing software correctly, but I imagine you'd deny
this, as who doesn't want to write correct software? Simply understand that
using a format with so many edge cases and other defects as JSON is foolhardy
for much of anything.

~~~
jimbo1qaz
>numerical (binary) formats are strictly superior, especially as they're used
increasingly often. It's better to have good tools that will provide a human-
readable representation, rather than pretend a single, inefficient
representation for both man and machine is somehow the best option.

I'm working on a chiptune music composing program (FamiTracker), which saves
projects in a binary format designed by my predecessor. I once repaired a
corrupted file for someone else. It's borderline impossible to repair binary
formats by hand, since inspecting the data on-disk provides no clues about the
structure, which I don't know since I didn't write the program. I ended up
merging the corrupted file and an older backup in Audacity (an audio editor)
by sliding the two files back and forth relative to each other, because hex
editors are awful at visualizing and aligning binary files (whereas diff tools
are good at visualizing and aligning text files).

Additionally, the on-disk layout changes between versions, and the reader code
is a mess of branching between different on-disk layouts based on the version
number.

Additionally, it's impossible to check in your music project (binary blob)
into Git, then get a meaningful diff between different versions, let alone
being able to merge changes made in 2 branches. (Yes, I use Git for music.
It's actually useful.)

Needless to say, I'm using a text format if I ever write a replacement. A text
format may have trouble encoding binary blobs, but I don't like the idea of
"transcoding .wav into binary files, and throwing away the parameters used".
I'm more inclined to just use pointers to on-disk .wav, and store all metadata
needed to reproduce the binary blobs, and possibly cache the binary blobs in
base64 for speed and convenience.

~~~
verisimilitudes
_I 'm working on a chiptune music composing program (FamiTracker), which saves
projects in a binary format designed by my predecessor. I once repaired a
corrupted file for someone else. It's borderline impossible to repair binary
formats by hand, since inspecting the data on-disk provides no clues about the
structure, which I don't know since I didn't write the program._

That's interesting. If you're interested in discussing that more at some
point, email me. Now, I've designed a numerical format for metadata involving
machine code programs and, while I've not written this piece yet, an important
aspect of this will be a repairing tool. The only reason a textual format
would be easier to repair by hand is because you can use a text editor and
it's more redundant. Now, good design is another quality; ideally, a numerical
format would lack _holes_ that are invalid states and wouldn't be riddled with
pointers or other things if feasible; it varies based on what you're doing,
though.

 _I ended up merging the corrupted file and an older backup in Audacity (an
audio editor) by sliding the two files back and forth relative to each other,
because hex editors are awful at visualizing and aligning binary files
(whereas diff tools are good at visualizing and aligning text files)._

That's again a simple matter of tooling that could be corrected.

 _Additionally, the on-disk layout changes between versions, and the reader
code is a mess of branching between different on-disk layouts based on the
version number._

A runaway format revision like that isn't ideal, I can agree. You can get the
same mess with textual formats, however.

 _Additionally, it 's impossible to check in your music project (binary blob)
into Git, then get a meaningful diff between different versions, let alone
being able to merge changes made in 2 branches. (Yes, I use Git for music.
It's actually useful.)_

I don't use git at all, so this is another difference between us. I'm largely
of the opinion that the focus on _textual formats_ for UNIX-likes was due to
sloth more than anything, as there's generally a lack of coherency between _/
etc/fstab_ to _/ etc/passwd_ and so on; I believe the focus was largely due to
the advantage of ostensibly generic tooling, but reading or writing a
configuration in a text editor, with the manual handy, and getting errors and
other things upon loading it is worse than having a tool write it for you and
show it to you at a high-level.

 _Needless to say, I 'm using a text format if I ever write a replacement. A
text format may have trouble encoding binary blobs, but I don't like the idea
of "transcoding .wav into binary files, and throwing away the parameters
used". I'm more inclined to just use pointers to on-disk .wav, and store all
metadata needed to reproduce the binary blobs, and possibly cache the binary
blobs in base64 for speed and convenience._

I don't know the details of what you're dealing with, but again, if you're
interested, email me and I can perhaps work on a repair tool or help design a
replacement numerical format that will be easier to manipulate.

------
dbg31415
So a few years back, everything was super fast on IMAP on my phone. But now it
just crawls. On the Gmail App it's still pretty fast. So I just assumed it was
slow because Google was being a dick and trying to get people to use their app
instead of Apple Mail or Outlook. Is the protocol slow, or is Google
throttling it?

~~~
jivings
Having just this month implemented an IMAP client I can tell you it's
blazingly fast for Gmail & Outlook.

It's actually much faster and more efficient than using the Gmail API.

~~~
dbg31415
I just clocked it at 16 seconds to check for new mail in Apple Mail checking
against a Gmail Account on an iPhone SE connected to Google Fiber. It feels
pretty slow...

Meanwhile, same update vs same account in Outlook took about 6 seconds.

And in the Gmail App, about 4 seconds.

Not super scientific, just me looking at my watch and opening each app and
seeing how long it took to update.

So maybe something with Apple Mail?

~~~
jivings
The slow part is opening the IMAP connection and authenticating, after that
it's fast.

I guess it might depend on how these clients are maintaining their
connections.

------
cotelletta
Give me the speed of fastmail with the encryption of protonmail, make the two
talk to each other out of the box, and actually make something new instead of
just the same old thing.

Actually, loan some of your devs to protonmail because both their mobile app
and their imap bridge are embarrassing.

------
0x0
They list the wrong RFC number "8261", but the linked RFC is actually "8621".

~~~
brongondwana
Thanks! Have fixed the copy. The link was going to the right place, but
somebody (possibly me) put the digits in the wrong order.

For your efforts, please enjoy a joke:
[https://comic.browserling.com/tag/multithreading](https://comic.browserling.com/tag/multithreading)

(that's my excuse - brain was running multithreaded output to the fingers)

------
Nasreddin_Hodja
[https://jmap.io/#push-mechanism](https://jmap.io/#push-mechanism) JMAP going
to use rfc8030 push for mobile (3rd party service for notifications, really?)
and EventSource (kind of http long polling) for desktop clients, as I
understood. I don't like that idea, it's even worse than IMAP4 IDLE. Why not
just WebSockets for mobile and desktop clients?

Or just building it on top of XMPP protocol (or any other chat protocol).

~~~
nailer
HTML5 Server Sent Events are better than long polling, and they're about as
old as websockets, but simpler, as they maintain the concept of HTTP resources
whereas websockets replace the concept. HTML5 SSE is also easier to implement.

Not hating on websockets, but saying HTML5 SSE is outdated tech is just wrong.
If you were making a real time app that works over HTTP either would be fine.

~~~
Nasreddin_Hodja
> HTML5 Server Sent Events are better than long polling

But it works the same way: infinite http connection.

------
akulbe
Is the protocol itself important to users? or is it mostly important just to
developers?

If it makes a practical difference to a user, how so?

------
z3t4
Lets ignore the biggest issue (spam) and focus on the bike shed.

~~~
dane-pgp
No one's stopping you from coming up with your own anti-spam technology.

In fact, it's possible for individuals and organisations to work towards
multiple goals at once. For example, Fastmail and the IETF have concurrently
been working on RFC 8617, which allows for stricter spam filtering:

[https://dmarc.org/2019/07/arc-protocol-published-as-
rfc-8617...](https://dmarc.org/2019/07/arc-protocol-published-as-rfc-8617/)

~~~
z3t4
The reason people choose Gmail or Office instead of smaller providers, is 1)
They get less spam 2) Less likely get their mail market as spam. This
centralization is killing e-mail. But that's just e-mail. What protocols do
people use to send messages ? Facebook messenger, Whatsapp, etc. More
proprietary centralized solutions. We need a simple decentralized protocol, as
simple as SMTP, bit it need to solve the spam issue, and it need not to be
over-engineered (like XMPP). And no, it should not be built ontop of SMTP,
SMTP has so many issues, like you can send mail pretending to be anyone. There
are solutions like SPF, DKIM etc, but 1) they are built ontop of SMTP 2) They
are not stopping spam. We need a new protocol that replaces SMTP, and have
spam prevention, identify, etc built in.

~~~
dane-pgp
Email could be free from spam if you configured your email client to only
accept emails from people you manually added (and if SPF/DKIM was enforced).
It's not clear how replacing SMTP with a different protocol would make the
situation any better.

The reason why centralised chat systems can limit spam is that one company
gets to decide who can and can't send messages to whom. Recreating that in a
decentralised architecture is precisely the problem that needs to be solved,
and the solution could then be used by email providers.

Now there are no technical reasons not to use SPF/DKIM, and recipients can
enforce it more strictly, the problem seems to be determining which mail
providers (i.e. which domains) can be trusted to stop their users from sending
spam. Spammers have realised this too, of course, which is why a lot of spam
comes from domains that are registered cheaply and quickly abandoned.

One solution that might be worth exploring is requiring that domain
registrants provide a bond to their registrar if they want to send mail from
that domain (grandfathering in all existing domains). The bond could be
refunded after, say, 12 months of sending enough emails that recipients don't
mark as spam.

The difficulty with this is creating a reporting mechanism that can't be
gamed, either to unfairly cost a domain its bond, or to flood the system with
fake positive reports to drown out the true negative ones. Fortunately, with
DKIM, there is at least cryptographic proof that a given email was sent by a
given domain.

~~~
z3t4
Messages should be sent from client to client. The server layer can be
skipped. But the client could of course be a cloud based email solution.
Messages would need double opt in, which would be baked into the protocol.
Id's would be a random hash. Personal messages could be trust based eg. "Do i
know this person" and corporates could use certificates. Once a user has opted
in to receive messages from a id/hash the receiver would generate a challenge
at the start of each session, either using a shared secret or public key
crypto (also baked into the protocol). The protocol should have a standard for
sending simple text. But then allow extension protocols, for example
implementing voice/video. With capabilities eg. Client tells the other client
what formats it accepts.

------
NetOpWibby
FINALLY

------
simplecomplex
Super happy customer of Fastmail of 7 years now.

It’s a joy to use. Fast, responsive, and reliable. No pointless redesigns or
10 second loading screens like gmail.

~~~
craftyguy
.. what does this comment have to do with the actual article other than keying
off of 'fastmail' in the title?

~~~
jhasse
Fastmail's web interfaces uses JMAP under the hood.

~~~
Guest49521
Hmm. I'm a happy FastMail customer of >2 years, but sometimes the web app
loses the connection with the host (especially when on flaky wifi) and fails
to re-establish it. This is really aggrevating when you hit send on an email,
which disables the input text box but then just hangs indefinitely in a
"Sending..." state. I've already reported this to customer support. Symptom of
JMAP, or just a run-of-the-mill bug somewhere?

