
Ask HN: What should replace XMPP? - tylorr
Let&#x27;s assume for the sake of discussion that XMPP has died or is at least on its last legs. What could we learn from XMPP that would help us find its replacement?<p>There are a few key questions that need answering. What did XMPP do right and could be copied? Why did so many big players ditch XMPP? What would it take to get big players to adopt it? What are the existing resources for creating protocols similar to XMPP? Is it possible to simplify a chat protocol or is it complex in nature?<p>I briefly tried working on an existing XMPP server implementation but found the protocol and all the extension overbearing and very hard to grok.
======
daurnimator
What XMPP did right:

    
    
      - Any chat protocol must be extendable
      - Federation is good
    
    

What could have been better

    
    
      - Regular users don't want to run their own server
      - Overcomplicated specs with optional bits (MUC, Pubsub)
      - Encryption should be mandatory from the start
      - Logging/history is important
    
    

Other lessons

    
    
      - Device sync is important
      - If you have multiple devices or clients; commit log style is best (see new MIX spec vs MUC spec).
      - In the modern world, people are afraid of XML
          - 95% of people still don't understand XML namespaces
    
    

\-------------------

But XMPP isn't dead (and doesn't have to be). New interesting XEPs are being
working on:

    
    
      - MAM
      - MIX aka MUC2
      - OMEMO

~~~
giancarlostoro
I think the issue of running our own servers would be non-existent if we could
run XMPP in inexpensive $5 a month digitalocean servers. It is my biggest
drawback for XMPP.

~~~
daurnimator
It's really easy to. Just install e.g. prosody

I run my XMPP server, and IRC bouncer (weechat in relay mode) + a few other
misc things all off a $5 VPS.

~~~
giancarlostoro
Where do you host at? Curious what the spec differences are between your VPS
and mine, if you don't mind answering

~~~
daurnimator
I use vultr, here have an affiliate link:
[http://www.vultr.com/?ref=6824304](http://www.vultr.com/?ref=6824304) I feel
like vultr's $5 VPS is what DigitalOcean should have been...

But you don't need good specs. I used to run prosody off a microcontroller at
home; I only moved to a VPS cause my net connection wasn't stable.

------
lwf
[https://matrix.org/](https://matrix.org/) — Matrix is increasingly looking
like the only sensible player; they get so many things right:

* Federated

* Fault-tolerant

* Cryptographic attestations of messages

~~~
Perceptes
I'm a fan of Matrix.org and very slowly working on an implementation of the
client-server API in Rust[1]. In the process of doing so I'm also doing my
best to help them improve and clarify the spec. I'd love to see nice native
clients for Matrix implemented for the various major platforms. Right now
there is vector.im, but I'm personally not a fan of using a web browser for
applications I leave running all the time, like chat.

[1] [https://github.com/ruma](https://github.com/ruma)

------
Todd
As you know, there's a lot of good in XMPP. The part that I think causes the
most difficulty is that it's based on XML. I don't mean this in the typical
knee-jerk XML bashing sense. It's just that XML is ill suited for a stream-
based protocol.

XML is amenable to extensibility, with its innate namespacing features. But
this also made it difficult for beginners to use well and was easy to get
wrong. The XML decision was really just a consequence of the times in which it
was developed. It was a natural choice. If it were done again, it would be
REST-based and use JSON as the data format.

~~~
tylorr
Would it be worth it to try to replace XMPP with something very similar but
using a JSON based protocol and clean up its worts in the process?

~~~
Todd
I absolutely think it would be worth it. The data format change would be one
thing. There are many other factors which are worth considering. For example,
the long lived connections of XMPP don't work well in the mobile environment.
There are XEPs which address things like this.

I think to do it right, you have to build a company around it, and build it in
the open. Also, no one is going to jump on a protocol just because it's open.
It has to be built around a service which is compelling in its own right, and
gains user traction regardless of the protocol. Geeks can get it going, but it
has to be self sustaining.

------
evilotto
I was under the impression that XMPP was alive and well, it's federation
that's dead. Because as it turns out, the big players aren't really interested
in interoperability, they want their own walled gardens because that's where
you can actually make money.

------
qrmn
XML was probably a mistake. Strong, deniable, end-to-end encryption should be
mandatory. The Axolotl ratchet is the current state-of-the-art: maybe it does
asymmetric things we don't need, or maybe that's helpful.

Looking forward: metadata protection. This is a much more difficult-to-solve
problem, but existing tools such as Tor are partially successful.

~~~
JoshTriplett
> Strong, deniable, end-to-end encryption should be mandatory.

Strong end-to-end encryption with perfect forward secrecy should be mandatory.
Deniable authentication
([https://en.wikipedia.org/wiki/Deniable_authentication](https://en.wikipedia.org/wiki/Deniable_authentication)),
however, seems like a potentially interesting option but not one that the
protocol should mandate. Sometimes you _do_ want authentication that remains
valid after the conversation ends, so you can subsequently authenticate the
messages in it.

------
detaro
Encryption would be a massive thing to figure out. More precisely, end-to-end
encryption with proper multi-device support.

That's also something all the other established players haven't solved, so it
would be a great advantage to get users.

------
babebridou
Putting the format and security aside, isn't the main issue that XMPP was so
good that we tried to bite more than it could chew?

\- Local Federation is great, but Global Federation doesn't benefit the big
players

\- Extensibility is great for domain-specific problems, but it leads to more
fragmentation

\- It tried to handle data streaming, but it's inherently suboptimal at it

I mean, it was a great protocol to explore the IM design space, but it was
bound to be replaced once we clarified our needs (for the most part, multiple
devices and media streaming).

As a side note, I often get showerthoughts about mixing IM and blockchain
techs. I'm not sure where that would lead us.

------
modos189
We are developing an open protocol, which take the best of XMPP and improve it
:)

We are for decentralization, the extension of the protocol, but without harm
for the users.

For web and native clients use thin client connecting to the server via
websocket and CBOR. Why not JSON? This allows you not to have problems with
sending binary files unlike XMPP.

Also thinking about using WebRTC for the transmission of all messages, files
and audio/video, but today WebRTC is not supported everywhere.

The server has the ability to transport other protocols.

------
JoshTriplett
Can any of the new secure-text-messaging protocols, such as the one used by
Signal, work in a federated manner? Many of those protocols currently use a
central server; which ones actually require that, and which ones don't?

------
estefan
A few weeks ago I replaced a XMPP chat architecture with socketio. It's so
much better for my needs (website with realtime chat).

There were 2 main reasons in the end:

\- No way to log all messages sent to a database so that users could retrieve
them on subsequent logins \- No way to edit/delete messages, especially in
multi-user chats.

Maybe these things are possible, but I couldn't find how to do it. Now, with
socket IO on nodejs, I have a < 400 line chat server with DB integration that
is much more suitable for my needs.

I guess XMPP's focus was too narrow.

~~~
daurnimator
> \- No way to log all messages sent to a database so that users could
> retrieve them on subsequent logins

This is what MAM does.

~~~
estefan
Perhaps the ejabberd support is bad or poorly documented? Either way I just
couldn't find how to do it. All I could find for MAM was logging to flat
files.

~~~
rakoo
This post [1] seems to indicate that ejabberd's support of MAM is complete.
Note that you also (obviously) need support from the client, which most but
not all clients do. Conversations [2], on mobile, does.

[1] [https://blog.process-one.net/ejabberd-15-06/](https://blog.process-
one.net/ejabberd-15-06/) [2]
[https://play.google.com/store/apps/details?id=eu.siacs.conve...](https://play.google.com/store/apps/details?id=eu.siacs.conversations)

~~~
estefan
That sucks. I searched for ages but found nothing. Oh well, it only took me a
couple of days to replace it, so it's not the end of the world.

------
tylorr
Would something like ProtocolBuffers[0] or FlatBuffers[1] be a good
replacement for XML? The data could then be transmitted in either binary or
JSON formats.

[0] [https://developers.google.com/protocol-
buffers/](https://developers.google.com/protocol-buffers/)

[1]
[https://google.github.io/flatbuffers/](https://google.github.io/flatbuffers/)

------
sbeckeriv
BEEP on TCP - RFC 3081

Any day now...

[https://en.wikipedia.org/wiki/BEEP](https://en.wikipedia.org/wiki/BEEP)

------
pmlnr
IRC - because dead simple & extendable

Tox - because fully decentralized & encrypted

SIP - because rock solid for voice & video

Chat & IM is not the web. It shouldn't be running on HTTP with HTTP APIs.

~~~
jgh
SIP is a signaling protocol (and XMPP can also be used that way). It doesn't
really have much to do with voice & video other than it gets used to
initialize a connection.

Also SIP is horrible.

------
ksec
May not be 100% replacement, but JMAP ?

------
maximmilius_rob
SIP?

~~~
zamalek
The IM protocol that runs over SIP is called SIMPLE and it's anything but.

------
moonshinefe
It's of course possible to simplify a chat protocol. For instance, I'm not
sure why the XMPP designers decided to go with XML; it's much more verbose
than most formats, and typically slower and more complicated to parse in my
experience say versus JSON.

What advantage did it have? Unless someone is sniffing unencrypted XMPP
packets and finds that XML readability handy, I don't really see the point.
And that's an edge case. Not to mention XML's main advantage over many other
formats is having hierarchical relationships--but how does that help in a chat
protocol?

I'm sure someone could very well have counter points to the above, but those
are the things off the top of my head that made me question why XML was the
top choice here for an IM protocol.

~~~
noblethrasher
XMPP predates the first general articulation of JSON by at least 3 years.

