

A new basis for open, distributed, real-time communication - lucdelany
http://matrix.org/blog/

======
Sarien
I would like to know how this is different from jabber/xmpp and how likely it
is that these differences are worthwhile. They say their API is more
"pragmatic" and writing a full-blown XMPP client certainly is very tricky but
I wonder if improving the way XMPP works wouldn't ultimately create a better
result. After all, a lot of thought and experience has by now gone into it.

~~~
oddvar
The main issues with XMPP that drove us in this direction were:

* Not particularly web-friendly – you can’t easily speak XMPP from a web browser.

* Single server per MUC is a single point of control and availability

* History synchronisation is very much a second class citizen feature

* Stanzas aren’t framed or reliably delivered (without extensions)

* Multiple device support is limited

* Baseline feature set is so minimal that good user experience cannot be guaranteed

* No strong identity system

* Not particularly well designed for mobile use cases (push; bandwidth-efficient transports)

disclaimer: I'm involved with this project!

~~~
cursed
I've read through the blog, the comment would have been useful on
differentiating Matrix from XMPP. Also, sort of off topic, but Skype has a
XMPP bridge and WhatsApp is built on XMPP (they just don't support
federation).

~~~
ara4n
Oops - I should have called it out more clearly in the blog.

The biggest difference we have over XMPP is probably that messages in Matrix
get synchronised over all the participants of a conversation... so you get
distributed chat history for free, and no single points of failure on group
chat (as you do with XMPP MUCs). And obviously Matrix is plain HTTP+JSON
rather than messing around with XML.

One of the reasons we built Matrix is because, in practice, pretty much _all_
the current big players started off using XMPP for developing their chat
solutions: Google Talk was originally XMPP; I believe Facebook Messenger was
built out at first on ejabberd; WhatsApp was originally XMPP etc; even APNS
was originally XMPP!

But _ALL_ of them have ended up mutating it to a proprietary closed standard -
and nobody has even tried open federation other than Google's misadventure
with Talk. So, unfortunately, it seems XMPP hasn't ended up being the
interoperable web-for-IM that we all hoped.

Now, I have absolutely no idea if Matrix will be more successful in solving
the problem; the hope is that by keeping it simple and using HTTP APIs there's
more of a chance that players of all sizes will start exposing Matrix APIs for
federation. Only time will tell. It's also worth noting that end-to-end
encryption is a relatively recent potential obstacle: given iMessage and
Telegram etc are all end-to-end encrypted, for them to ever federate with
Matrix we'll need to support the same semantics and crypto. Hopefully we're
going about this the right way (although end-to-end crypto isn't formally
specified or implemented yet), but will be an interesting challenge. And one
that XMPP hasn't tried to solve at all, as far as I know.

(disclaimer: Matrix is my fault)

------
jpfr
The term 'real-time' is used _very_ loosely here. In industrial settings, in
order to claim realtime-properties, you need to make guarantees about the
runtime behavior of server, client and the network infrastructure in between.

IP-based Ethernet -> not realtime capable (without extensions)

Python (garbage collected language) -> not realtime capable

You can at most claim to achieve soft-realtime. As in "sub-500ms but only most
of the time".. o_O

~~~
rpdillon
Different contexts. For weapon systems and industrial systems, real-time
operating systems are needed to guarantee true real-time behavior.

That usage makes no sense in the context of internet messaging for the reasons
you describe. For human-to-human messaging, I think they're referring to a
property more akin to synchronous (versus asynchronous) messaging, i.e. chat
versus email. I speculate that the matrix team thinks this is an acceptable
overloading because, as a practical matter, the two concepts have nothing to
do with one another.

~~~
ara4n
rpdillon is totally right - the term is overloaded. We're using "realtime"
exclusively here to mean synchronous human communication - like the RT in
WebRTC. In other words, IM or VoIP - anything where you expect to get a
response from the other user inmediately rather than waiting asynchronously.

This of course clashes with the RTOS style use of the word, and we aren't
claiming that Matrix has any hard realtime timing guarantees at all, other
than the soft realtime behaviour media stacks like WebRTC give you by means of
high priority threads.

In retrospect it's a confusing term; we should probably avoid using it :)

disclaimer: use of the word realtime in the Matrix.org blurb is all my
fault...

------
rdtsc
On a related note, I discovered that Freeswitch, on its master branch has
mod_verto:

[https://confluence.freeswitch.org/display/FREESWITCH/mod_ver...](https://confluence.freeswitch.org/display/FREESWITCH/mod_verto)

It uses WebRTC for media and JSON-RPC for signalling. On the server side it is
Freeswitch module, so can have access to PSTN calling, VOIP and other
signaling protocol.

------
Paul_S
Can't help but think "jesus, not another protocol I have to support". People
bitch and moan about SIP but the goddamn thing works and is already extensible
if you need your own unique features. Can't you build on that? Instead you're
like phone manufacturers who each come up with their own protocol because hey,
we've got those sweet PBX boxes to sell too!

I love your goal, but I really can't see why you can't use SIP to communicate
and play nice with other endpoints out there. Surely that would be an
advantage.

~~~
oddvar
That would be one way to do it :)

Problems with SIP:

* There are so multiple SIP standards (e.g. rfc 3261 vs rfc 2543)

* It doesn't support NAT (unless you do rfc5626, rfc5627 and ICE)

* Its support for ICE doesn't work well with webrtc which dynamically generates candidates

* SIP's reliability is bad unless you implement rfc3262

* We want to do messaging really well - and SIP's support for that is rather basic

edit: finally, I do agree with your "yet another standard" comment! (as per
the xkcd in [http://matrix.org/blog/2014/09/03/hello-
world/](http://matrix.org/blog/2014/09/03/hello-world/))

Given that XMPP hasn't taken off as it could have done - and the fact that
each large IM or VoIP application seems to end up writing their own version
(and putting that in their own walled garden) - we think a new, well-defined
open standard, together with open source reference server and client codebases
can be the catalyst to create an interoperable and federated "message
transport" solution for a multitude of services!

------
huntaub
This is a very cool project, and one that I am very partial to. I agree that
having a common application layer for structured data would be incredibly
useful for developers and users across the board by freeing data from the big
silos.

Question: Are you looking to expand beyond IM and VoIP, or do you see those as
the main areas where the protocol will function?

~~~
oddvar
as per the FAQ: The initial goal of Matrix is to fix the fragmentation of VoIP
and IM, but Matrix’s real potential and ultimate mission is to be a new and
truly open ecosystem on the Internet enabling services, devices and people to
easily communicate with each other.

I'd say IM and VoIP would be the obvious common cases, but Matrix can and will
be used to solve all kinds of problems!

------
oddvar
Just adding that this is very much the first few steps of this project - if
you think it sounds interesting we would love feedback and comments on
#matrix:matrix.org (via matrix.org) or #matrix on freenode!

------
giggles_giggles
The headline is somewhat misleading -- this is a basis for federated IM, like
xmpp. It's not truly distributed, the way something like tox.im is.

~~~
kaoD
[http://en.wikipedia.org/wiki/Distributed_computing](http://en.wikipedia.org/wiki/Distributed_computing)

P2P protocols are distributed but not all distributed protocols are P2P.

------
cx0
Did any of you (including Geary, I guess he blessed it, or does it go all the
way up to Rami ?) considered the fact that maybe the issue isn't the absense
of open standard (xmpp despite it's quirks does work) but lack of interest
from companies to open up their walled gardens ?

------
tomelliott
How long until you get a reference implementation that supports VoIP?

~~~
oddvar
hi tom!

this is already available! if you click on a user and start a one-to-one
conversation, there's a handy "Voice Call" button - try it in chrome!

------
zAy0LfpBZLC8mAC
"simple HTTP signalling channel" ... seriously?

~~~
huntaub
Can you help explain what you mean by this comment?

~~~
zAy0LfpBZLC8mAC
Well ... HTTP is terribly complicated, and about as bad a fit for the problem
as it can get? It has tons of overhead (at least compared to the mostly tiny
payloads you can expect for IM), it doesn't support push delivery (except
through horrible hacks like long polling), doesn't really support pipelining,
in particular over TLS, none of the mechanisms that HTTP does provide seems to
be of any use whatsoever for the purpose ... and then they complain about SIP!

Overall, it makes the impression on me like it was designed by someone who
doesn't know much more than how to build web apps and therefore is afraid of
anything not HTTP, and so will use HTTP no matter what, even if there are
essentially only disadvantages compared to pretty much any alternative you
could come up with.

(Oh, and also, they claim being "RESTful", without being anything like that
...)

~~~
ara4n
SIP+SIMPLE/MRSP's overheads are just as bad as HTTP/1.x, if not worse. We
decided to build Matrix after 10 years of fun doing commercial SIP and XMPP
work; we have some experience. For instance, the worst scenario we've seen in
a real-life SIP/MSRP user-agent in the wild was 50KB of SIP/MSRP nego to send
the word "Heh" between two contacts (with a delivery report) :) Meanwhile SPDY
and HTTP/2.x improve the overhead situation and pipelining situation
enormously - although frankly we haven't seen any real-life problems using
HTTP/1.1 yet at all...

The "simplicity" of HTTP I mentioned in the Matrix blurb refers to the fact
that RFC2616 is relatively compact and self-contained, whereas SIP/SIMPLE/MSRP
involves a huge number of RFCs, different protocols (SIP,SDP,MSRP...) and
really is a lot more complicated to implement than just doing GETs and PUTs.

Now, the irony is that rather than being "designed by someone who doesn't know
much more than how to build web apps" \- it's more the other way round. Our
experience is mainly with SIP/RTP/STUN/ICE/TURN etc; genuinely RESTful APIs
are relatively new territory for us.

I would genuinely love to know what aspects of the Matrix client-server (or
server-server) API is 'not anything like RESTful' \- it's not too late for us
to change the APIs (they've already been rewritten several times, oscillating
between more or less RESTful purity), and half the point of releasing Matrix
in its current early proto-form is to get detailed feedback from folks on
whether we've Got It All Wrong :)

(disclaimer: Matrix is all my fault)

~~~
zAy0LfpBZLC8mAC
Oh, I certainly didn't mean to imply that SIP was somehow particularly
efficient - I was just surprised that anyone would think HTTP was an
improvement over SIP, in particular given that you just swap the SIP stack for
an HTTP stack and the torture testing of SIP parsers for torture testing of
HTTP parsers, especially so if you consider the history of SIP where people
originally intended SIP to be parsable using HTTP parsers.

As for the huge number of RFCs: Well, those do cover a whole lot more than
passing blobs around? That aspect is fully covered by the SIP RFC, and only a
relatively small part of it, the rest is concerned with session establishment
and teardown, which HTTP obviously isn't concerned with at all.

Which is why I am wondering: What is the point of using HTTP? What is the
functionality that it provides you with that would justify requiring an HTTP
stack in every endpoint, including the security risk from all that complexity?

As for RESTfulness: REST is in conflict with a fixed URI scheme, REST allows
only for a fixed entry point, beyond that, all other relevant URIs should be
discoverable from the returned representations. And no, I am not saying, your
stuff should be RESTful, I'm just saying that it's not.

