Hacker News new | past | comments | ask | show | jobs | submit login

Just about every week now there is a top story on HN about the disastrous state of interoperable messaging in the digital world. Every time it comes up, I implore people to look into Matrix (https://matrix.org), a system with an open specification that checks all the boxes in terms of features (not quite end-to-end encryption yet, but it's being worked on) and is not controlled by a company. I really hope to see it enter people's consciousness as one of the most serious contenders for messaging systems.

Matrix is very usable today via the Vector client (https://vector.im/). You can connect to the public server run by matrix.org, or you can run your own server that federates with all the others if you'd like to maintain ownership of your data. Matrix can also bridge to existing systems like IRC, so there isn't necessarily the need to convince your friends and family to "switch" with you. There are interoperable clients being developed for other operating systems including mobile platforms. Even with client support not all the way there yet, the spec and the system are more promising than anything else out there today.

If you're interested in Rust, you might also want to take a look at my project, Ruma (https://www.ruma.io/), which is a Rust implementation of the Matrix system. It's in very early stages, so don't expect to use it at this point, but it's progressing steadily.

> a system with an open specification that checks all the boxes in terms of features (not quite end-to-end encryption yet, but it's being worked on) and is not controlled by a company

Half of the people reading tune out very quickly around this point.

Most people want something that solves 80% of their problems with 20% reduction in headaches.

The only major problem I still have with Vector is that despite Matrix the protocol specifying ways to register and authenticate accounts using various web apis like oauth[1] Vector is always pushed as the premium Matrix webclient but it doesn't support anything but username / password.

You would be amazed how many people simply refuse to create new username / password pairs in 2016. And its a good thing. We should be past username / password. Support oauth2, openid connect, and for the privacy minded... keep complaining to Mozilla that they shutdown Persona.


Actually, Vector does support alternative auth mechanisms (although it's quite buried) - it implements the web fallback mechanism, both on web, iOS & Android, such that any unrecognised auth mechanisms can be handled by whatever webpage fragment hosted by the server, as per: https://matrix.org/docs/spec/r0.0.1/client_server.html#fallb...

For instance, we have a workplace deployment of Vector knocking around that uses CAS for auth, and the whole CAS flow is handled by the fallback mechanism. (This CAS happens to be user/pass, but it could easily be 2FA or whatever instead).

That said, it's still beta, and not super-documented.

I have no qualms about OpenID Connect itself — it is a sound standard, using JSON Web Tokens is a particularly nice feature — but I dislike encouraging users to identify themselves for every service they use with Facebook or Google accounts. Way too much information about your digital life centralised with a single commercial party.

I'd much rather see the use of password managers become more common, or even better, stimulate user-friendly two-factor authentication standards such as FIDO U2F.

OpenID Connect supports discoverability, which enables a protocol like Mozilla Persona to work over it. That's exactly what some others and I are working on, hopefully we will make some headway and greatly increase the usability of authentication again.

Sounds like a good initiative, be sure to post on HN when you reach significant milestones.

What I worry about with OpenID Connect on the open internet (as opposed to OpenID Connect used in a corporate setting, or to provide single sign-on solutions for specific partnered services) is how it almost always results in a service asking you to use Google or Facebook to authenticate, with huge coloured buttons, followed by a tiny de-emphasized grey link where you can sign up using email and a password.

Is this something we can prevent?

They only provide Google / Facebook buttons because they are by far the most popular option. In practice the vast majority of major online web services support some from of openid / oauth signon, from Github to Steam to Wordpress and more.

As a site developer, it is your job to figure out what buttons your user demographic will use. If it is all facebook / google, then provide those, and most of these sites know providing more obscure options only confuses their potential users.

I think it is, because the "Google" or "Facebook" part isn't static in OpenID Connect, they're just popular defaults. OIDC doesn't care where it goes to authenticate, you can just as well enter your email, foo@example.com and the server will try discovery and authentication against example.com (which can then be running your own OIDC provider).

What we're working on (we will definitely post about it) is basically OIDC with a nicer UX, where the user enters their email, the domain is parsed out, discovery (of OIDC or whatever other protocol) is done, and the user is redirected to either their provider, if found, or to a default method of authentication (currently a link sent over email) to log in.

So what you're saying is that username/password is still the only choice for privacy minded people.

Google Talk did have XMPP functionality, which allowed for SASL. [1] But of course, they killed this off because they wanted to dominate chat and messaging with their own proprietary protocol.

If they had not followed this strategy they would be killing in this market, even whilst supporting XMPP Federation. But they have chosen to release multiple proprietary products, so they aren't the lead in the market. Silly of them really, and it just shows you they aren't really the nimble company they once were.

1. https://developers.google.com/talk/open_communications

I doubt the decision to close up their protocols was taking lightly. You can bet that user lock-in is part of their current strategy.

Did any Google engineers ever reveal Google's true intentions for shutting down their XMPP offerings?

I'm still using XMPP to connect to GChat ... I use it every workday (via Pidgin).

Google stopped supporting XMPP in the sense that they stopped trying to change XMPP to fit their needs. They also started killing off support for arbitrary XMPP servers, e.g. you cannot run your own and expect it to work with Google Chat. (This was the most up to date link I could find.)[https://xmpp.org/2015/03/no-its-not-the-end-of-xmpp-for-goog...]

You can still somewhat use it, but it does not work well. Groupchats do not work at all anymore, and there is no message synchronization over it so any Hangouts client will populate messages an XMPP client never sees, and you cannot request chat history from the remote.

It is definitely not a high quality XMPP experience. If you want to use XMPP, you should probably be using duckduckgo's XMPP offering.

One day it will go away...

They killed it because Microsoft was leaching contact data from them without giving back.


We are pretty much past needing to remember them. Checkout password managers. There are lots of options in that space these days.

Do you mean end users, as in the type of people systems like Slack market to, or are you referring to the failures of previous systems that made open specs and interoperability a goal?

If the former, Matrix the specification and overall system is not what would be sold to the end user. Like you say, most of them don't care how it works or about the philosophical or political reasons why you might choose a system like this. They just want to be able to talk to friends, family, and coworkers and have a good user experience. That's something any company could package and sell in a way that resonated with people, using Matrix as its underlying system.

In other words, Super Awesome Chat, Inc. could create a really slick hosted Matrix service and client with great usability, and the end user wouldn't necessarily know or care about how it worked. In mentioning the open specification, it's not for the benefit of that type of user, but at the HN reader who is likely more informed or interested in the details of the technology, the philosophy, or political/sociological implications.

That was exactly the deal with the old google talk. Most of its users never knew or cared about xmpp and interserver comunication. It just worked.

Haha! I just tried Vector on iOS and apparently moved about in such a way that I accidentally triggered the greatest notification message:

> You seem to be shaking the phone in frustration. Would you like to submit a bug report?

I've seen this UI in other apps, and I absolutely fucking hate it. Google Maps was a particularly egregious offender, where rotating the device or calibrating the compass would prompt for feedback, usually in the middle of trying to use a map. And who shakes a device in annoyance anyway?!

No kidding. Especially annoying when using Google Maps and driving a stick shift.

The stick shift gesture should notify the police of your location and destination.

This is so, so dumb. Who actually shakes their phone in anger? Very few people. How many people accidentally trigger things like this in the course of everyday usage? Everyone.

Google Maps has the same thing and it just strikes me as some cutesy feature a developer thought would be funny, but is actually very annoying.

Having an app developer available to be shaken in anger was deemed impractical.

Wow. I am truly impressed.

A lot of companies use the "shake the phone hard" as a way of sending feedback.


We had jabber for a decade at least. The battle for messaging market share doesn't seem technological.

Heck you even had the option to federate private servers so one could address users across hosts.

But Jabber was XML based. Matrix is JSON Based!

If only we would've chosen the right technology, people would've used it.


edit: it's a joke. I'm an idiot.

You're missing the sarcasm.

If you believe that's the point the OP was making, you did not click the link in the post you're replying to.

Edit: still an idiot.

Please read, comprehend and then post. From the page you didn't read that was linked in the OP:

> NOTICE: This document is Humorous. It MAY provide amusement but SHOULD NOT be taken seriously.

Thank You. Brightened up my evening.

I really really like the ideas of Matrix, but the problem is none of my friends are on it. I had the same problem when I started to use Signal.

WhatsApp actually seems like the lesser of two evils right now - even though it's controlled by Facebook, they've added encryption and some of my friends are on it.

Who'd have thought we'd look back on the days of MSN, AIM and ICQ as the golden age of interoperable messaging? At least I could talk to everyone I needed to through Gaim (Pidgin) or Trillian.

Being able to use Gaim/Pidgin doesn't count as interoperability dude. Those protocols worked only because somebody bothered to reverse engineer them and this "interoperability" only worked half the time. I know because I was using Gaim/Pidgin for Yahoo! Messenger back then.

Today at least they are using web technologies and I get to use Facebook's Messenger, WhatsApp, Hangouts and Slack on Linux without headaches.

For a short period, most of those old proprietary clients started supporting XMPP. That was the golden age of instant messaging. I couldn't help but notice that somewhere around that time is also when Apple released iMessage and FaceTime. Suddenly, everybody ran back to walled gardens and we de-evolved to a point where people are suddenly okay paying for a service that they can only sort of integrate with. Thats how low the bar has gotten now.

Personally, if I were a place like Google where the content is more important than the product. I would be having very pointed discussions with the people in charge of things like Hangouts and Allo. There seems to be a lack of vision compared to what is easily doable in this space. Knowing enough of the XMPP spec to see what was never done, I have to agree that the author is right. Google never understood what they had with GChat.

To put it another way. Imagine if everybody ran around trying to make their own SMTP or HTTP protocols? Thats what everybody is trying to do with chat right now.

Oh, I totally get what you're saying, except that I don't find usage of XMPP clients to be that useful. Yes, I know that Facebook supported at some point XMPP clients. I never cared about it.

Real interoperability is federation. If we had federation, the problem of using your favorite client would be moot. My argument is that there never was a "golden age" of chat, unless you refer to IRC back in the nineties and even that would be debatable.

And out of the big boys I can only name Google's Talk as supporting federation, only to be killed and replaced by Hangouts, that doesn't support federation because, wouldn't you know, they needed to innovate and couldn't do that while supporting XMPP, allegedly. And even if Hangouts pushed WebRTC forward, I'm glad it's not used and I'm glad that it suffers a slow death, because this proves that the innovation argument is bullshit. And I'm fairly certain that Google Talk was more popular than Hangouts, even if Hangouts got pushed down the throats of every Android user.

> I really really like the ideas of Matrix, but the problem is none of my friends are on it.

Build a whatsapp bridge for matrix? I mean, it's built for the exact case where people aren't "on" it.

Precisely. There's already the beginnings of a WhatsApp bridge for Matrix (and indeed anything that libpurple can speak) at https://github.com/matrix-org/node-purple/tree/master/appser.... It just needs tuits for people to work on and finish off. Meanwhile, the IRC & Slack bridges are live and work relatively well today (although a lot more work still to be done).

The problem is Whatsapp is actively hostile to any kind of attempts to creating 3rd party clients or APIs.

Exactly, and this is the problem with the state of messaging in 2016. The companies have built their walled gardens and are now actively defending them against "outsiders". It's to the point where I've considered buying an iDevice just to communicate with a few friends on iMessage.

Strange thing is this was always the case and just recently started to change. MSN API was a joke, you needed a desktop environment on the server to run the actual client to use SOME of the functionality. Nearly the same with Skype. Gchat removed Jabber pretty fast. Facebook chat also removed the outside functionality of their XMPP clone. Whatsapp is not even able to provide a web client, but only a weired bridge because they crippled to xmpp they are using way to much. No experience with the Apple services but i assume they are similar.

Next to that we always had open protocols, that worked perfectly, had plenty of clients and apps and nobody used them.

It is a stupid market.

Convince them to get Telegram. It works on most devices, and has extensibility built in:



I just installed the Telegram app. Why is it asking for my phone number?

They use phone number as account name rather than email on mobile.

What if I don't want to use it on a phone?

The web and desktop versions can send an SMS to authenticate if you don't want to or can't use the phone app.


I see. I don't really want to tie it to a phone number, though.

Already did with the in-laws. :-)

Wasn't really a problem with ICQ, even though they had actively struggled and tried to change protocols frequently (and broke unofficial clients once in few months).

The community was large, and there was a significant demand for good clients (official one was shit). I guess now the difference must be that either official client is good enough for most users, or that users' attitude to how they want things had changed and they learned to submit to whatever service providers dictate.

So was AIM, though. That didn't stop the third party clients.

AOL worked actively to stop third-party AIM clients in the same ways as mentioned above from icq through WhatsApp -- changing protocols and central auth mechanisms.



Right, the point is "whatsapp is hostile to third-party clients" makes no sense as an explanation of why we don't have third-party clients for whatsapp, because all third-party clients for everything have had to deal with hostility from the first party.

I have had some success with getting people on signal, though now that Whatsapp is encrypted by the same technology there doesn't seem to be a big benefit anymore. Generally it seems to me people are not so adverse to having several messanging sollutions installed these days.

"Decentralized persistent communication", as long as you either use a centralized homeserver you don't control, or run a hugely resource-hungry and crash-prone homeserver yourself.

I tried running my own for a few months, and resource demands for even a single user in a few moderately popular channels brought down my small ec2 instance.

Eventually I had to shut it down, but now I've lost my ability to authenticate as the user I identified as during that time, until I set up my homeserver again.

For anyone else reading, it's worth noting that these issues are with Synapse, the "reference implementation" homeserver built by the team developing the Matrix spec. Performance issues are a focus for its development now. There are also other projects (like my project Ruma) that provide alternative implementations that won't necessarily have these performance issues (hopefully!) The nice part about having a spec is that anyone has the opportunity to provide a superior implementation.

That's interesting, I'll check it out!

Yup. the Synapse server implementation is still beta, and we are working away on the performance issues (yay, Python/Twisted). In the last few months we've reduced the memory footprint by about 2-3x (by string interning, better caching, and letting the cache size be configurable). The only stability issue we're aware of currently is the risk of being OOMed on small VPSes.

That said, we're really hoping that having almost finished fleshing out the whole ecosystem now, the future will be much tighter and svelter homeservers like Perceptes' ruma implementation.

Problem is the greatest system in the world is useless unless a quorum of people in your circle are using it. I've been trying to get my friends to use Wire for a year but still have to run skype

That's where bridging with other services comes in, e.g. using Matrix on your side to talk to contacts on IRC or whatever else without the people on the other side having to change what system or software they use.

Does this require control of the server being bridged to implement, or could I do something like, say, set up an instance that lets me be on both Rizon and Freenode at the same time on one client connection?

My understanding is that bridges are implemented via application services (the Matrix term for privileged plugins that extend the capability of the server,) so a user's ability to communicate over non-Matrix protocols is dependent on the homeserver where their user account is based. The best known example of this is that the public matrix.org homeserver is bridged to the Freenode IRC network, and certain channels are synced to each other, e.g. (#matrix on Freenode is the same thing as #matrix:matrix.org on matrix.org.)

I'm not sure how this works for non-public communication, like private messages on IRC or services like Facebook where it wouldn't be safe for the Matrix homeserver to have access to a user's Facebook account.

At this point I'm not super well-versed on the details of application services so hopefully one of the Matrix team members will reply to you directly.

It generally doesn't require input from the target network being bridged, unless you're running a large public bridge like the ones we host on matrix.org. You do however currently have to run your own homeserver in order to run your own bridge (although in future we're looking at ways of getting around this for 'personal' bridging purposes).

If you run your own homeserver, and your own bridge to Rhizome/wherever, you can think of the end result being a bit like bitlbee or a pidgin-in-the-cloud... but decentralised, and with the richness of Matrix's conversation semantics (e.g. arbitrary data types) rather than being limited to IRC.

> If you run your own homeserver, and your own bridge to Rhizome/wherever, you can think of the end result being a bit like bitlbee or a pidgin-in-the-cloud... but decentralised

Wouldn't this still be centralized to your home server?

Wasn't open chat already solved by Jabber decades ago?

Moxie Marlinspike recently published an interesting essay that adressed this question.

”XMPP is an example of a federated protocol that advertises itself as a ’living standard.’ Despite its capacity for protocol ’extensions,’ however, it's undeniable that XMPP still largely resembles a synchronous protocol with limited support for rich media, which can't realistically be deployed on mobile devices. If XMPP is so extensible, why haven't those extensions quickly brought it up to speed with the modern world?

Like any federated protocol, extensions don't mean much unless everyone applies them, and that's an almost impossible task in a truly federated landscape. What we have instead is a complicated morass of XEPs that aren't consistently applied anywhere. The implications of that are severe, because someone's choice to use an XMPP client or server that doesn't support video or some other arbitrary feature doesn't only effect them, it effects everyone who tries to communicate with them. It creates a climate of uncertainty, never knowing whether things will work or not. In the consumer space, fractured client support is often worse than no client support at all, because consistency is incredibly important for creating a compelling user experience.”


I am hoping the Matrix team will write a blog post about their viewpoint on Moxie's post, given that they are out to do something Moxie doesn't believe can succeed. I'm also curious to hear how they respond to the specific section you quoted: how is Matrix going to avoid the same fate XMPP suffered here?

We haven't had a chance to write a formal response yet; too busy getting Matrix out of beta. I responded at the time at https://news.ycombinator.com/item?id=11669681 though.

A quick specific answer w.r.t. XMPP is that Matrix aims for a very coarser granularity of modularity, published as a single consistent spec, and mandates feature profiles to determine which modules should get used where. In other words, there is only one official set of endpoints specified for Matrix for a given use case (e.g. mobile IM+VoIP). So it ends up being under tighter control (whilst still very much taking proposals and PRs from the overall community), and is easier to swap out the higher level layers to rapidly evolve functionality - rather than suffering the proliferation of XEPs which is XMPP's double-edged sword.

This is always going to move slower than a centralised service, but we think the freedom of being able to select your clients/servers/services is well worth that cost in agility - and does not have to compromise privacy if engineered correctly.

Thanks for the response and the link to the previous response. I'd missed that! Both are great answers. :}

> Like any federated protocol, extensions don't mean much unless everyone applies them, and that's an almost impossible task in a truly federated landscape. What we have instead is a complicated morass of XEPs that aren't consistently applied anywhere.

The problem is, without actual numbers - how many clients support what - this statement is just a personal opinion, not a rule.

We can say exactly the same about the web. It's also federated, it also has fractured client support (although, sadly, core engines can be counted on fingers) and it's also full of extensions. While I won't praise the web, and I'll surely admit that the trends for siloing are strong (for many, a single big social network site of their choice is the "web"), it still manages to provide somehow decent user experience.

There is the w3 and other standards comitties joined by industrie leaders and a lot of experts and public eyeballs. Chat hardly compares. Perhaps the W3 should concern itself with chat. The IETF released SIP. The w3c offers IRC channels.

A broken browser is a pain, new standars improve the user experience (e.g. html video). For the public user there is no discernable difference in any basic chat's protocol and implemetation.

IETF has also standartized XMPP.

Jabber/XMPP has a number of problems that make it unsuitable for modern chat environments where clients (read: smartphones) have poor connections and frequently want to sleep to conserve power.

For example, XMPP wants an always-on TCP connection between the client and the server for the client to appear "online". There's an extension which allows for sleeping, but having async mobile connections in an extension means that you need the right pair of server and client to enable this feature. Plus, there's no handling of syncing messages between multiple clients. If you've got a smartphone and a desktop client, to compete with Facebook Messenger you need detailed conversation transcripts on both clients -- and an intelligent way to ping your clients and tell which should receive sound notifications.

see more at https://op-co.de/blog/posts/mobile_xmpp_in_2014/

Emphasis on in 2014, that was forever ago in Internet time. XMPP is steadily moving forward and the things mentioned in that post have improved a lot. Check out https://conversations.im/ for example.

That was not only XMPPs problem. It is what sunk most of the desktop IM systems, making way for the mobile ones.

Jabber doesn't do offline messaging well at all, nor push particularly well (not out of the box, so to speak).

The "standard library" of messaging has expanded a lot lately.

This is cool. But IMHO the point of the article is about Google losing its first-mover advantage in chat apps due to wrong branding and prediction for GChat, not interoperability.

I force my friends to use Gchat when the chat is turning into long conversation and I want to type on my PC keyboard instead of my cell.

But I'm not sad Google missed out with Gchat. One less product Google dominates with means some other company can innovate in that area and compete.

True. My comment is not a response to the particulars of this article, but more about the area of software of which GChat was a member. Apologies for turning it into my own soapbox.

I have a computer which is always on and always connected.

It's my smartphone.

Can I just run a 'homeserver' on that and be connected?

Sure. Homeserver software doesn't care where it's hosted. If you want it to federate with other homeservers on the Internet, it will need to be accessible from outside your local network, of course.

Is anyone working on it that I could help?

I'm not sure exactly what you're asking. The only practically usable homeserver software right now is Synapse (https://github.com/matrix-org/synapse), an implementation in Python built by the Matrix core developer team.

Synapse only runs serverside. In future we'd love to evolve things to be able to run clientside - e.g. our P2P Matrix GSoC project: https://github.com/matrix-org/GSoC/blob/master/IDEAS.md#peer...

So, why did you answer "Sure" to his question if the answer is actually "No, you can not run a homeserver right now on your smartphone" ?

I mean, a python app for posix systems is not really close to being run on unrooted android and iOS.

I somehow didn't process the "it's a smartphone part" and completely misunderstood his question as a result. I'll edit my original reply to correct that. Edit: Apparently I can't edit it anymore. :{

That's fine. I am really interested in the possibility of running a 'homeserver' directly on a phone.

Do you think that would be possible?

Yes, it is possible. Ultimately, a homeserver is just a program that implements the client-server, and optionally the federation, APIs. The only feature-complete homeserver right now is Synapse, which is written in Python, and as of this moment has performance issues even running on a server, so it's not realistic that it could be modified to run on a phone. My homeserver Ruma is written in Rust, which is a systems language that actually is suited for mobile and embedded systems development, although I am currently not targeting that use case, nor is Ruma mature enough to be practically useful yet. If you're interested in the possibility of developing your own homeserver implementation specifically targeting your use case, you should take a look through the Matrix specification documents: https://matrix.org/docs/spec/

It was a "sure, in principle".

Always connected? I really envy you.

I mean, can you even take the subway or go on an overseas travel and still be sure that your smartphone is your single point of failure?

If you are on Android or iOS, it’s not always connected – and you can’t actually run things in background all the time anymore either.

I really like the ideas behind Matrix. Sure, the existing (open) clients aren't all that sexy, and synapse can be a bit slow sometimes. But implementing your own client is a breeze and bridging other networks through application services makes it incredibly powerful.

My experience with the matrix.org community has been very pleasant so far. Keep up the good work, folks!

Matrix looks interesting. But, damn, that animation on the homepage slows my browser to a crawl!

How is the feature set any different than XMPP?

They have a good explanation of how Matrix compares to XMPP in their FAQ: https://matrix.org/docs/guides/faq.html#what-is-the-differen...

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact