
FastMail: Shutting down our XMPP chat service - kevinchen
http://blog.fastmail.com/2015/11/16/shutting-down-our-xmpp-chat-service
======
pquerna
While the FastMail shutting down XMPP is about low usage, the real cause is
that Federated Chat failed to work.

Federation is something that is rarely promoted in modern ecosystems. Before
Twitter/Facebook, we had RSS/Atom, but these have all failed as companies pour
massives investments into "winner take all" approaches to their products. In
the short term, it is easy to see why: Why compromise on iteration of features
and UX by allowing a competitor to interoperate? Why wait for a new field to
be standardized in Atom?

We see it all the time in the consumer space, federation has died, but even
look in the infrastructure space, something like Docker: Yes, Docker Hub, is
"kinda" open, I mean, anyone can create an account, but it's not a federated
scheme where everyone can host their own Images easily. But we use it anyways,
because its a better user experience, and its subsidized by a company not
directly making revenue on it yet, but wanting to control the user experience.

So we drop support for any federated user experience pretty quickly. And now
we are all on Slack, on Facebook, on Twitter.... And it is mostly great. Or we
wouldn't still be there.

But I still wish for a federated future, but I doubt it will happen.

~~~
mseebach
It's also instructive to reflect over the other federated internet
communications protocol: E-mail.

It pretty much works, and that's great, but barely a week goes by when HN
doesn't have an article lamenting the lack of useful spam-control or
encryption/sender verification and many other things. E-mail is still a very
poor medium to facilitate long threads with many participants. E-mail as
notifications (push) is lacking. E-mail as an interface to software systems
(e.g. ticket tracking systems) is lacking.

These deficiencies are all easy to fix in a technical sense, but practically
impossible to address due to the federated nature of the system. The few
changes we do end up getting (HTML bodies, and even MIME before then) are long
and painful, and are largely resolved by "might is right". Others unilaterally
break fundamental features, such as curbing spam by essentially banning
decentralised SMTP. If you rely on being able to send out large quantities of
email, you either have to pay a gatekeeper, or hire a specialist in the dark
arts of actually have your email go through. This dark art consists of keeping
track of the many ways in which the large players in the e-mail world
systematically break the protocol to combat spam.

We have essentially abandoned the federated nature of email and handed over
one-sided control to a small group of companies (Google, Microsoft, Yahoo,
perhaps a few other) in order to effectively curb spam. The alternative would
have been the end of email.

XMPP fails because it tries to, indeed _needs to_ , boil the ocean. There's an
extension, or two or seven, for every conceivable feature and use case, and so
you need to choose which set of features in a n-furcated (where _n_ is
tragically large) set you want/need, and live with the fact the these aren't
perfectly compatible with deployments that has made other choices. And so the
federation and standards compatibility is really only theoretical, and then,
what's the point? If spam killed truly federated email (and the need for
actual working encryption might well finish it off), mobile killed XMPP.

Hopefully, at some point, the IM features of Slack, Twitter, Google Hangouts,
iMessage, HipChat, Yammer, Skype, Whatsapp and friends are going to converge
enough that an open source application can emerge and define a basic set of
features through domination, in much the same way Wordpress ate the blogging
world, and that can then lay the foundation for interoperability.

~~~
pmlnr
> E-mail as notifications (push) is lacking

Let me introduce you to IMAP IDLE.

> E-mail as an interface to software systems (e.g. ticket tracking systems) is
> lacking.

Jira, for example, has a pretty good one.

> We have essentially abandoned the federated nature of email and handed over
> one-sided control to a small group of companies (Google, Microsoft, Yahoo,
> perhaps a few other) in order to effectively curb spam. The alternative
> would have been the end of email.

I've been running my own mail stack for 8+ years. The reason is simple: only I
can mess it up. No 3rd party to lock me out from my communication, no 3rd
party to decide what I can send. If it fails, it's my fault only. And I'm
still using email. I do occasionally get spam, but most of them is already
dropped by blacklisting and a few intelligent rule; the rest is catched by
dspam.

XMPP is overcomplicated, solving nothing IRC can't solve already ( imho, of
course ); that is why it's failing.

What we need for the more open source approach is people to value their
communication and not to trust 3rd party because the look/act cool.

> Hopefully, at some point, the IM features of Slack, Twitter, Google
> Hangouts, iMessage, HipChat, Yammer, Skype, Whatsapp and friends are going
> to converge enough that an open source application can emerge and define a
> basic set of features through domination, in much the same way Wordpress ate
> the blogging world, and that can then lay the foundation for
> interoperability.

They won't. WordPress was open from the start and the current trends are
locking-in APIs, not open standards. There is no profit in openness.

~~~
Zikes
> no 3rd party to decide what I can send

It's my understanding that the biggest hurdle in owning your own email stack
is that a lot of 3rd parties decide what they'll receive, and hold senders to
very strict standards. Have you experienced many issues with recipients on
various platforms not receiving your emails?

~~~
xorcist
I have worked in this area for a long time, and it's very rare that this
causes any problems unless you are a spammer (or "news-letter sender"). You
won't end up on black lists unless there is a mistake. It's an annoyance when
that happens, but that happens for the major players as well.

I often feel the ambivalence to your own mail servers is unfounded. A basic
apt-get for Postfix and Dovecot will give you secure defaults. The only thing
you need to do then is to set up the certificates, the dns records, and keep
up with what happens in the future. It's absolutely comparable to running your
own web server, where you also need to change a cert or two over time.

------
colanderman
What a sad day.

A hearty Fuck You to Google, Facebook, and Apple for tearing apart this open
federated standard. Users don't benefit from being forced to use your shitty
walled-garden proprietary communication apps.

(Yes, forced; if I want to talk to my friends, it usually has to be via
whatever proprietary chat network is in vogue at the moment. There was a brief
golden age of IM during which GChat (federated XMPP) was king, but GChat's
been in its death throes for years, and the current flavors-of-the-week are
AIM and MSN all over again.)

~~~
Camillo
XMPP lacks a fundamental feature: it does not echo your messages back to your
other devices. This made it uniquely unsuitable for the mobile world: if you
start a conversation with someone on your computer, and then move to your
phone, all you see on the phone are the other person's messages, as if they
had been talking to themselves. (Yes, someone did eventually write an
extension for that, which nobody ended up supporting.)

I can't even say that XMPP failed to adapt to mobile. It's more that XMPP was
a terrible protocol with a gigantic flaw that should not have been there in
the first place, and mobile just made the flaw obvious to everyone.

~~~
ge0rg
All these problems have been solved in XMPP years ago, and there are some
clients fully supporting the mobile experience and usage of multiple clients.

On the Desktop, Swift and Gajim come to my mind, on Android Conversations is
getting really awesome (and yaxim, which I am the author of, supports the
basic multi-client/mobile use case you are interested in).

That said, the XMPP protocol is not perfect, and it is lacking some important
elements for a 2010+ mobile chat developer, but considering that it was
initially developed in 1999, that mobile chat is only one of its many use
cases, and that there are extensions covering all you have asked for, the
critique is not appropriate.

[0] [http://swift.im/swift.html](http://swift.im/swift.html)

[1] [http://gajim.org/](http://gajim.org/)

[2] [http://conversations.im/](http://conversations.im/)

[3] [https://yaxim.org/](https://yaxim.org/)

~~~
goffi
Just for the record, we are also working on a next-gen XMPP client, with
decentralized blogging, file sharing, end 2 end encryption and all, and
planing to do an Android port. Other projects like Movim also do similar
things too, XMPP is definitely very alive these days.

Sending message to other devices is managed by message carbons which is in
last call for being a standard XEP, and support is becoming more and more
common.

XMPP is all but deprecated, very active, working well, with a large ecosystem.
But is really badly known by most of people(I have by the way started a series
of article to explain it:
[http://www.goffi.org/tag/talk_xmpp](http://www.goffi.org/tag/talk_xmpp)).

It's really difficult to be heard these days, but there is a new wave in XMPP
world (check our project: [http://salut-a-toi.org](http://salut-a-toi.org) or
movim: [http://movim.eu](http://movim.eu)).

~~~
jessaustin
_XMPP is all but deprecated..._

Just a short English usage note: "all but" means something like "almost".
Judging by the rest of your post, you probably mean "XMPP is _far from_
deprecated" here.

~~~
goffi
yes indeed, I'm not a native speaker as you can guess.

Thanks for the correction :)

~~~
kbenson
I usually read "X is all but Y" as "X is Y in all but name", but there are
subtle distinctions that could be hard for a non-native speakers[1] to grasp
without knowing about them.

1: [http://english.stackexchange.com/questions/9967/all-but-
idio...](http://english.stackexchange.com/questions/9967/all-but-idiom-has-
two-meanings)

------
JoshTriplett
I had a similar experience. I ran my own XMPP server for several years, so
that my email address also worked as an XMPP/Jabber ID. I talked to people
across several different services, including an increasing number of folks on
Google Talk.

Then Google Talk stopped federating properly. Only a bit at first: things like
presence notification didn't quite work right, and adding people didn't quite
work right. And in the meantime, Hangouts popped up, and offered a more
seamless audio/video experience too. Most people I know stopped using Google
Talk entirely in favor of Hangouts; the few that remained used it only while
the gmail web interface more-or-less supported it. Eventually, federation with
Google Talk quite deliberately stopped working.

It didn't take long after that before almost everyone else I talked to on it
stopped showing up, even those who didn't use Google Talk. People had "last
seen" times in the months, and if I wanted to talk to those people in
realtime, I sent them text messages, or Twitter DMs, or Hangout calls.

So when I upgraded my server a couple of months ago, I just didn't bother
setting up the XMPP server again. Not because it didn't work, but just because
it had nobody to talk to.

~~~
stock_toaster

      > So when I upgraded my server a couple of months ago, 
      > I just didn't bother setting up the XMPP server again.
      > Not because it didn't work, but just because it had nobody
      > to talk to.
    

Exactly mirrors my experience. ;_;

------
zanny
Its our fault for letting our parents and cousins and friends start using
imessage / hangouts / etc instead of XMPP. We are the ones to blame for its
death, because the unwashed masses got locked in to regressive proprietary
chat platforms over the past five years and we never pushed the issue.

But it doesn't help that XMPP was always awful. It has OpenGL syndrome - too
many extensions, described too vaguely, and implemented in too many broken
ways to work amongst one another. You get a swarm of mismatched servers and
features, and if it did the same thing that would have saved commoner OpenGL
adoption, it would probably still be king.

That problem is a blessed default implementation. If the foundations that
wrote the specs for these low level pipeline APIs were also burdened to make
sure the blessed implementation supported the entire standard perfectly, there
would be an out of the box open source permissively licensed base for everyone
else to build off, rather than the vague wording of technical documentation.

I'm just hoping Matrix becomes something. In the same way Vulkan can hopefully
save universal graphics programming, Matrix might save IM, but we need to
support it. I'm working on the Telepathy integration for it myself on weekends
in a sketch repo.

~~~
LeoPanthera
I worked hard to get my family all on XMPP and succeeded. The problem is, they
all want to be on iMessage _as well_ , because of course that's what everyone
else in the world uses. Eventually iMessage became the de-facto default even
inside my family, and there was just no reason for XMPP to exist.

------
soupbowl
I feel that [http://matrix.org/](http://matrix.org/) Is the proper open source
replacement for XMPP. I changed over to it from XMPP and haven't looked back.

~~~
hackerboos
Ah push notifications the feature we are all looking for in XMPP which is
there
([http://xmpp.org/extensions/xep-0357.html](http://xmpp.org/extensions/xep-0357.html))
but not implemented in any of the open source servers.

It's missing websockets support though which would be nice.

I found a nice overview of what Matrix.org is doing [2].

[1] - [http://matrix.org/docs/spec/#id33](http://matrix.org/docs/spec/#id33)

[2] - [https://bloggeek.me/matrix-webrtc-
interview/](https://bloggeek.me/matrix-webrtc-interview/)

------
LeoPanthera
Previous discussion about the death of XMPP:
[https://news.ycombinator.com/item?id=10483936](https://news.ycombinator.com/item?id=10483936)

My comment from that thread:

> I've been running my own XMPP server for years, with federation enabled. A
> few years ago, it seemed like the logical successor to AIM and MSN and all
> those other walled garden IM systems. And how easy! My XMPP "name" was the
> same as my email address. One less thing to put on my business card.

> But since then, I have realised a big problem with it - no-one uses it!
> Today I communicate with the world by iMessage, SMS, Twitter, and email.
> "Instant messaging" just seems to have died as a concept entirely, replaced
> by yet more walled gardens like Snapchat.

> My XMPP server is being turned off for good, next week.

(It has since been turned off.)

------
legutierr
My company's dev team is on Slack, so I log in there whenever I need to talk
to them--unless we're doing a video conference, at which point we switch to
Google Hangouts. Most of our clients prefer the phone, email or SMS/iMessage,
but the ones that don't use Skype. If I'm chatting with friends it's probably
on Facebook, unless it's on Hangouts. Any video chat my family does is over
Facetime.

These services are never going to inter-operate, and I hate it. Imagine if we
had to maintain different telephone numbers or different email accounts just
to communicate with different groups of people. It's almost unimaginable, but
that's what we have now with text and video chat.

This is core communications infrastructure. These segregated private networks
have begun to displace the public telephone network in terms of popularity and
importance. These are also the platforms over which video communication is
being rolled out--not the public telephone network, as most people would have
thought twenty-five years ago.

The FCC needs to step in and set some inter-operability rules for the major
players here. I have mixed emotions suggesting this, considering that I'm
generally opposed to regulation of the Internet. But unless the government
steps in, can the situation ever change? Doesn't it need to?

~~~
Atheros
Perhaps we should just wait for one to become a solid monopoly and then our
anti-trust laws can take care of it. That seems to be the good middle-ground
between freedom from regulation and freedom from monopolies. And it worked out
pretty well for our phones.

------
farski
As one of those few hundred people who actively use FastMail's XMPP, I guess
I'm now in the market for an alternative. I need exactly one federated account
on my domain, so a standalone service probably isn't going to make sense
financially.

From a DIY perspective, what are the options? Are Ignite, Prosody, and
ejabberd the only viable options? Are they all being maintained? Is there
anything out there that works at a small scale; like a Node server that could
run on OpenShift, or something else super lightweight?

~~~
brongondwana
We had a bad experience with ejabberd pre-release version 3 (had an intern
spend the entire summer on it, and I put some time in too):

[http://blog.fastmail.com/2012/09/26/one-step-forward-two-
ste...](http://blog.fastmail.com/2012/09/26/one-step-forward-two-steps-back/)

Honestly Prosody looks the best. We were planning to switch to it when async
login support and scalability to thousands of domains was stable... but that
was 2 years ago and it never quite materialised - and now DJabberd is so far
behind in support for "standard XMPP" and the interoperability of the
remaining servers that we just had to give up. I argued strongly against it,
having spent so many bloody years on making this thing work - including tons
of patches to DJabberd and some EJabberd hacking as well - but the numbers
didn't lie.

Better to spend our time on exciting things like the currently-in-testing
cross-domain support for Cyrus IMAPd and reverse ACL lookup that will make the
LIST command blindingly fast even on big servers. I'm hoping to be able to say
two steps forward, one step back this time instead :)

~~~
hackerboos
Did you look into MongooseIM?

------
stock_toaster
In my opinion, and as a customer of Fastmail, I think this is the right call.
xmpp as a _usable_ federated chat protocol is pretty much dead.

For those who wish to keep using it, setting up your own server (prosody is
pretty good!) isn't too hard these days.

------
rekoros
We've turned chat interoperability into a business over at sameroom.io.

What other option is there, really? The other option is no interoperability.

A very relevant post: [https://sameroom.io/blog/announcing-support-for-google-
hango...](https://sameroom.io/blog/announcing-support-for-google-hangouts/)

~~~
brazeon
So how this works with federation? I don't see any open/common API.

~~~
rekoros
It's "federation-as-a-service", basically.

A common use case is federation between two Slack teams (share a channel).

Another common use case is federation between a group in Skype and a channel
on Slack.

Or IRC and Telegram, etc.

Making existing systems work together really well is priority one. API only
makes sense once that's done.

------
verusfossa
This is disappointing, but I don't really blame them. Thankfully, IMAP is too
established to succumb to embrace, extend, extinguish although Google seems to
be trying. Chat protocols haven't been so lucky.

Hopefully federated chat will pick up, but it seems business models around
social networking pretty much preclude any kind of federated communication
between services. If adoption rates for Diaspora, Friendica, Pump.io,
GNUSocial etc. are any indication, XMPP looks doomed. :(

------
lyinsteve
Apple uses XMPP internally as a way to securely communicate.

Doesn't really change much; just thought that'd be an interesting thing to
throw out.

~~~
4ad
What exactly do you mean by that? I'm really curious.

------
rascul
Interestingly enough, more and more of my friends and family are registering
with and using my XMPP server. Jitsi on the laptops and desktops and
ChatSecure on the phones are generally the clients they're using.

------
scrollaway
Two insightful previous discussions about XMPP:

[https://news.ycombinator.com/item?id=10280155](https://news.ycombinator.com/item?id=10280155)

[https://news.ycombinator.com/item?id=10486541](https://news.ycombinator.com/item?id=10486541)

I'd highly recommend people read through them, especially if you're interested
in _fixing_ the issue.

------
duncan_bayne
Ah FFS. It's sad to see that XMPP has come to this. We had the promise of an
open Internet and it's being consciously destroyed by the big players, one
platform at a time.

(By which I mean Google, Facebook, Atlassian, Slack, etc.; all of those
companies who have chosen to put a nail in the coffin of the open Internet).

Oh well, guess I won't be doing online chat any more.

I wonder whether the children even believed the last generation of Roman
Britons when they spoke of writing, under floor heating, goods from half a
world away...

~~~
bachmeier
Can't speak to the others, but Atlassian shouldn't be on that list. They offer
XMPP, I've used it, and it works well.

~~~
duncan_bayne
Federated XMPP? Or is it just that you can hook an XMPP client up to their
walled garden?

------
aikah
All I say want to say is we need open protocols (and standards), period. We
need choice. Those who want to create wall-gardens are free to do so but we
need open protocols and educate users and developers about how important they
are. RSS, XMPP and others are used everyday, even by corporations so eager to
user proprietary ones for their to keep their users prisoners.

------
joshstrange
To all of those moaning about how great XMPP was: It wasn't.

In a multi-device/mobile world XMPP is shit. HN constantly complains about
Slack/Hipchat/etc "Reinventing the wheel". "We have IRC and XMPP" they cry
without stopping for a second to understand that IRC/XMPP as not trivial to
setup correctly, mobile support is complete shit, without things like bouncers
or servers to hold all your messages you can lose things and stuff not be in
sync, clients can't really add features unless all client can support it, and
more.

Does it kind of suck that the dream of federated chat died/is dying? Yes but
if it doesn't support what users really want then it's the standards fault. I
don't blame any of the major players for pulling support, it wasn't going to
move fast enough so they ditched it and built something that would.

~~~
hmans
XMPP (like many other similar protocols) is failing not because it's "not
moving fast enough", but because open standards are required to make a
commitment to simplicity (in order to be inclusive towards as many devices as
possibility) which, in many cases, directly opposes the requirements of a
business (ie. having unique features.)

In the end, it's quite simple: federation and simplicity work _against_ many
business cases. This is why this stuff is being dropped by the big players in
favor of proprietary solutions left, right and center.

~~~
joshstrange
Push notifications, message syncing, read receipts, offline sending (as in I'm
offline, you send a message, I get it when I log in), encrypted chat, mobile
clients, desktop clients, chat room support, direct message support.

These are the things I need out of chat and while XMPP had extensions for some
of those things finding server software and client that supported it reliably
was near impossible.

------
Aoyagi
Well I hate to see any features shut down even if the alternative means
"unsupported operation"...

Opera has been doing that a -lot-, the biggest cases being Unite and soon
Link.

One thing though:

>Meanwhile, our XMPP service is now somewhat behind the cutting edge, and
lacks support for many of the recent XMPP extensions that the dedicated XMPP
users are now beginning to request.

Does that mean if they kept silent the service would just run somewhere in the
corner, forgotten? ^^

------
shmerl
Google really betrayed XMPP effort, and Eric Schmidt should be blamed for it.
Others aren't any better - just greedy gardeners of their walled gardens.

