Just about every week now there is a top story on HN about the disastrous state of interoperable messaging in the digital world. Every time it comes up, I implore people to look into Matrix (https://matrix.org), a system with an open specification that checks all the boxes in terms of features (not quite end-to-end encryption yet, but it's being worked on) and is not controlled by a company. I really hope to see it enter people's consciousness as one of the most serious contenders for messaging systems.
Matrix is very usable today via the Vector client (https://vector.im/). You can connect to the public server run by matrix.org, or you can run your own server that federates with all the others if you'd like to maintain ownership of your data. Matrix can also bridge to existing systems like IRC, so there isn't necessarily the need to convince your friends and family to "switch" with you. There are interoperable clients being developed for other operating systems including mobile platforms. Even with client support not all the way there yet, the spec and the system are more promising than anything else out there today.
If you're interested in Rust, you might also want to take a look at my project, Ruma (https://www.ruma.io/), which is a Rust implementation of the Matrix system. It's in very early stages, so don't expect to use it at this point, but it's progressing steadily.
> a system with an open specification that checks all the boxes in terms of features (not quite end-to-end encryption yet, but it's being worked on) and is not controlled by a company
Half of the people reading tune out very quickly around this point.
Most people want something that solves 80% of their problems with 20% reduction in headaches.
The only major problem I still have with Vector is that despite Matrix the protocol specifying ways to register and authenticate accounts using various web apis like oauth[1] Vector is always pushed as the premium Matrix webclient but it doesn't support anything but username / password.
You would be amazed how many people simply refuse to create new username / password pairs in 2016. And its a good thing. We should be past username / password. Support oauth2, openid connect, and for the privacy minded... keep complaining to Mozilla that they shutdown Persona.
Actually, Vector does support alternative auth mechanisms (although it's quite buried) - it implements the web fallback mechanism, both on web, iOS & Android, such that any unrecognised auth mechanisms can be handled by whatever webpage fragment hosted by the server, as per: https://matrix.org/docs/spec/r0.0.1/client_server.html#fallb...
For instance, we have a workplace deployment of Vector knocking around that uses CAS for auth, and the whole CAS flow is handled by the fallback mechanism. (This CAS happens to be user/pass, but it could easily be 2FA or whatever instead).
That said, it's still beta, and not super-documented.
I have no qualms about OpenID Connect itself — it is a sound standard, using JSON Web Tokens is a particularly nice feature — but I dislike encouraging users to identify themselves for every service they use with Facebook or Google accounts. Way too much information about your digital life centralised with a single commercial party.
I'd much rather see the use of password managers become more common, or even better, stimulate user-friendly two-factor authentication standards such as FIDO U2F.
OpenID Connect supports discoverability, which enables a protocol like Mozilla Persona to work over it. That's exactly what some others and I are working on, hopefully we will make some headway and greatly increase the usability of authentication again.
Sounds like a good initiative, be sure to post on HN when you reach significant milestones.
What I worry about with OpenID Connect on the open internet (as opposed to OpenID Connect used in a corporate setting, or to provide single sign-on solutions for specific partnered services) is how it almost always results in a service asking you to use Google or Facebook to authenticate, with huge coloured buttons, followed by a tiny de-emphasized grey link where you can sign up using email and a password.
They only provide Google / Facebook buttons because they are by far the most popular option. In practice the vast majority of major online web services support some from of openid / oauth signon, from Github to Steam to Wordpress and more.
As a site developer, it is your job to figure out what buttons your user demographic will use. If it is all facebook / google, then provide those, and most of these sites know providing more obscure options only confuses their potential users.
I think it is, because the "Google" or "Facebook" part isn't static in OpenID Connect, they're just popular defaults. OIDC doesn't care where it goes to authenticate, you can just as well enter your email, foo@example.com and the server will try discovery and authentication against example.com (which can then be running your own OIDC provider).
What we're working on (we will definitely post about it) is basically OIDC with a nicer UX, where the user enters their email, the domain is parsed out, discovery (of OIDC or whatever other protocol) is done, and the user is redirected to either their provider, if found, or to a default method of authentication (currently a link sent over email) to log in.
Google Talk did have XMPP functionality, which allowed for SASL. [1] But of course, they killed this off because they wanted to dominate chat and messaging with their own proprietary protocol.
If they had not followed this strategy they would be killing in this market, even whilst supporting XMPP Federation. But they have chosen to release multiple proprietary products, so they aren't the lead in the market. Silly of them really, and it just shows you they aren't really the nimble company they once were.
Google stopped supporting XMPP in the sense that they stopped trying to change XMPP to fit their needs. They also started killing off support for arbitrary XMPP servers, e.g. you cannot run your own and expect it to work with Google Chat. (This was the most up to date link I could find.)[https://xmpp.org/2015/03/no-its-not-the-end-of-xmpp-for-goog...]
You can still somewhat use it, but it does not work well. Groupchats do not work at all anymore, and there is no message synchronization over it so any Hangouts client will populate messages an XMPP client never sees, and you cannot request chat history from the remote.
It is definitely not a high quality XMPP experience. If you want to use XMPP, you should probably be using duckduckgo's XMPP offering.
Do you mean end users, as in the type of people systems like Slack market to, or are you referring to the failures of previous systems that made open specs and interoperability a goal?
If the former, Matrix the specification and overall system is not what would be sold to the end user. Like you say, most of them don't care how it works or about the philosophical or political reasons why you might choose a system like this. They just want to be able to talk to friends, family, and coworkers and have a good user experience. That's something any company could package and sell in a way that resonated with people, using Matrix as its underlying system.
In other words, Super Awesome Chat, Inc. could create a really slick hosted Matrix service and client with great usability, and the end user wouldn't necessarily know or care about how it worked. In mentioning the open specification, it's not for the benefit of that type of user, but at the HN reader who is likely more informed or interested in the details of the technology, the philosophy, or political/sociological implications.
I've seen this UI in other apps, and I absolutely fucking hate it. Google Maps was a particularly egregious offender, where rotating the device or calibrating the compass would prompt for feedback, usually in the middle of trying to use a map. And who shakes a device in annoyance anyway?!
This is so, so dumb. Who actually shakes their phone in anger? Very few people. How many people accidentally trigger things like this in the course of everyday usage? Everyone.
Google Maps has the same thing and it just strikes me as some cutesy feature a developer thought would be funny, but is actually very annoying.
I really really like the ideas of Matrix, but the problem is none of my friends are on it. I had the same problem when I started to use Signal.
WhatsApp actually seems like the lesser of two evils right now - even though it's controlled by Facebook, they've added encryption and some of my friends are on it.
Who'd have thought we'd look back on the days of MSN, AIM and ICQ as the golden age of interoperable messaging? At least I could talk to everyone I needed to through Gaim (Pidgin) or Trillian.
Being able to use Gaim/Pidgin doesn't count as interoperability dude.
Those protocols worked only because somebody bothered to reverse engineer them and this "interoperability" only worked half the time. I know because I was using Gaim/Pidgin for Yahoo! Messenger back then.
Today at least they are using web technologies and I get to use Facebook's Messenger, WhatsApp, Hangouts and Slack on Linux without headaches.
For a short period, most of those old proprietary clients started supporting XMPP. That was the golden age of instant messaging. I couldn't help but notice that somewhere around that time is also when Apple released iMessage and FaceTime. Suddenly, everybody ran back to walled gardens and we de-evolved to a point where people are suddenly okay paying for a service that they can only sort of integrate with. Thats how low the bar has gotten now.
Personally, if I were a place like Google where the content is more important than the product. I would be having very pointed discussions with the people in charge of things like Hangouts and Allo. There seems to be a lack of vision compared to what is easily doable in this space. Knowing enough of the XMPP spec to see what was never done, I have to agree that the author is right. Google never understood what they had with GChat.
To put it another way. Imagine if everybody ran around trying to make their own SMTP or HTTP protocols? Thats what everybody is trying to do with chat right now.
Oh, I totally get what you're saying, except that I don't find usage of XMPP clients to be that useful. Yes, I know that Facebook supported at some point XMPP clients. I never cared about it.
Real interoperability is federation. If we had federation, the problem of using your favorite client would be moot. My argument is that there never was a "golden age" of chat, unless you refer to IRC back in the nineties and even that would be debatable.
And out of the big boys I can only name Google's Talk as supporting federation, only to be killed and replaced by Hangouts, that doesn't support federation because, wouldn't you know, they needed to innovate and couldn't do that while supporting XMPP, allegedly. And even if Hangouts pushed WebRTC forward, I'm glad it's not used and I'm glad that it suffers a slow death, because this proves that the innovation argument is bullshit. And I'm fairly certain that Google Talk was more popular than Hangouts, even if Hangouts got pushed down the throats of every Android user.
Precisely. There's already the beginnings of a WhatsApp bridge for Matrix (and indeed anything that libpurple can speak) at https://github.com/matrix-org/node-purple/tree/master/appser.... It just needs tuits for people to work on and finish off. Meanwhile, the IRC & Slack bridges are live and work relatively well today (although a lot more work still to be done).
Exactly, and this is the problem with the state of messaging in 2016. The companies have built their walled gardens and are now actively defending them against "outsiders". It's to the point where I've considered buying an iDevice just to communicate with a few friends on iMessage.
Strange thing is this was always the case and just recently started to change. MSN API was a joke, you needed a desktop environment on the server to run the actual client to use SOME of the functionality. Nearly the same with Skype. Gchat removed Jabber pretty fast. Facebook chat also removed the outside functionality of their XMPP clone. Whatsapp is not even able to provide a web client, but only a weired bridge because they crippled to xmpp they are using way to much. No experience with the Apple services but i assume they are similar.
Next to that we always had open protocols, that worked perfectly, had plenty of clients and apps and nobody used them.
Wasn't really a problem with ICQ, even though they had actively struggled and tried to change protocols frequently (and broke unofficial clients once in few months).
The community was large, and there was a significant demand for good clients (official one was shit). I guess now the difference must be that either official client is good enough for most users, or that users' attitude to how they want things had changed and they learned to submit to whatever service providers dictate.
AOL worked actively to stop third-party AIM clients in the same ways as mentioned above from icq through WhatsApp -- changing protocols and central auth mechanisms.
Right, the point is "whatsapp is hostile to third-party clients" makes no sense as an explanation of why we don't have third-party clients for whatsapp, because all third-party clients for everything have had to deal with hostility from the first party.
I have had some success with getting people on signal, though now that Whatsapp is encrypted by the same technology there doesn't seem to be a big benefit anymore. Generally it seems to me people are not so adverse to having several messanging sollutions installed these days.
"Decentralized persistent communication", as long as you either use a centralized homeserver you don't control, or run a hugely resource-hungry and crash-prone homeserver yourself.
I tried running my own for a few months, and resource demands for even a single user in a few moderately popular channels brought down my small ec2 instance.
Eventually I had to shut it down, but now I've lost my ability to authenticate as the user I identified as during that time, until I set up my homeserver again.
For anyone else reading, it's worth noting that these issues are with Synapse, the "reference implementation" homeserver built by the team developing the Matrix spec. Performance issues are a focus for its development now. There are also other projects (like my project Ruma) that provide alternative implementations that won't necessarily have these performance issues (hopefully!) The nice part about having a spec is that anyone has the opportunity to provide a superior implementation.
Yup. the Synapse server implementation is still beta, and we are working away on the performance issues (yay, Python/Twisted). In the last few months we've reduced the memory footprint by about 2-3x (by string interning, better caching, and letting the cache size be configurable). The only stability issue we're aware of currently is the risk of being OOMed on small VPSes.
That said, we're really hoping that having almost finished fleshing out the whole ecosystem now, the future will be much tighter and svelter homeservers like Perceptes' ruma implementation.
Problem is the greatest system in the world is useless unless a quorum of people in your circle are using it. I've been trying to get my friends to use Wire for a year but still have to run skype
That's where bridging with other services comes in, e.g. using Matrix on your side to talk to contacts on IRC or whatever else without the people on the other side having to change what system or software they use.
Does this require control of the server being bridged to implement, or could I do something like, say, set up an instance that lets me be on both Rizon and Freenode at the same time on one client connection?
My understanding is that bridges are implemented via application services (the Matrix term for privileged plugins that extend the capability of the server,) so a user's ability to communicate over non-Matrix protocols is dependent on the homeserver where their user account is based. The best known example of this is that the public matrix.org homeserver is bridged to the Freenode IRC network, and certain channels are synced to each other, e.g. (#matrix on Freenode is the same thing as #matrix:matrix.org on matrix.org.)
I'm not sure how this works for non-public communication, like private messages on IRC or services like Facebook where it wouldn't be safe for the Matrix homeserver to have access to a user's Facebook account.
At this point I'm not super well-versed on the details of application services so hopefully one of the Matrix team members will reply to you directly.
It generally doesn't require input from the target network being bridged, unless you're running a large public bridge like the ones we host on matrix.org. You do however currently have to run your own homeserver in order to run your own bridge (although in future we're looking at ways of getting around this for 'personal' bridging purposes).
If you run your own homeserver, and your own bridge to Rhizome/wherever, you can think of the end result being a bit like bitlbee or a pidgin-in-the-cloud... but decentralised, and with the richness of Matrix's conversation semantics (e.g. arbitrary data types) rather than being limited to IRC.
> If you run your own homeserver, and your own bridge to Rhizome/wherever, you can think of the end result being a bit like bitlbee or a pidgin-in-the-cloud... but decentralised
Wouldn't this still be centralized to your home server?
Moxie Marlinspike recently published an interesting essay that adressed this question.
”XMPP is an example of a federated protocol that advertises itself as a ’living standard.’ Despite its capacity for protocol ’extensions,’ however, it's undeniable that XMPP still largely resembles a synchronous protocol with limited support for rich media, which can't realistically be deployed on mobile devices. If XMPP is so extensible, why haven't those extensions quickly brought it up to speed with the modern world?
Like any federated protocol, extensions don't mean much unless everyone applies them, and that's an almost impossible task in a truly federated landscape. What we have instead is a complicated morass of XEPs that aren't consistently applied anywhere. The implications of that are severe, because someone's choice to use an XMPP client or server that doesn't support video or some other arbitrary feature doesn't only effect them, it effects everyone who tries to communicate with them. It creates a climate of uncertainty, never knowing whether things will work or not. In the consumer space, fractured client support is often worse than no client support at all, because consistency is incredibly important for creating a compelling user experience.”
I am hoping the Matrix team will write a blog post about their viewpoint on Moxie's post, given that they are out to do something Moxie doesn't believe can succeed. I'm also curious to hear how they respond to the specific section you quoted: how is Matrix going to avoid the same fate XMPP suffered here?
A quick specific answer w.r.t. XMPP is that Matrix aims for a very coarser granularity of modularity, published as a single consistent spec, and mandates feature profiles to determine which modules should get used where. In other words, there is only one official set of endpoints specified for Matrix for a given use case (e.g. mobile IM+VoIP). So it ends up being under tighter control (whilst still very much taking proposals and PRs from the overall community), and is easier to swap out the higher level layers to rapidly evolve functionality - rather than suffering the proliferation of XEPs which is XMPP's double-edged sword.
This is always going to move slower than a centralised service, but we think the freedom of being able to select your clients/servers/services is well worth that cost in agility - and does not have to compromise privacy if engineered correctly.
> Like any federated protocol, extensions don't mean much unless everyone applies them, and that's an almost impossible task in a truly federated landscape. What we have instead is a complicated morass of XEPs that aren't consistently applied anywhere.
The problem is, without actual numbers - how many clients support what - this statement is just a personal opinion, not a rule.
We can say exactly the same about the web. It's also federated, it also has fractured client support (although, sadly, core engines can be counted on fingers) and it's also full of extensions. While I won't praise the web, and I'll surely admit that the trends for siloing are strong (for many, a single big social network site of their choice is the "web"), it still manages to provide somehow decent user experience.
There is the w3 and other standards comitties joined by industrie leaders and a lot of experts and public eyeballs. Chat hardly compares. Perhaps the W3 should concern itself with chat. The IETF released SIP. The w3c offers IRC channels.
A broken browser is a pain, new standars improve the user experience (e.g. html video). For the public user there is no discernable difference in any basic chat's protocol and implemetation.
Jabber/XMPP has a number of problems that make it unsuitable for modern chat environments where clients (read: smartphones) have poor connections and frequently want to sleep to conserve power.
For example, XMPP wants an always-on TCP connection between the client and the server for the client to appear "online". There's an extension which allows for sleeping, but having async mobile connections in an extension means that you need the right pair of server and client to enable this feature. Plus, there's no handling of syncing messages between multiple clients. If you've got a smartphone and a desktop client, to compete with Facebook Messenger you need detailed conversation transcripts on both clients -- and an intelligent way to ping your clients and tell which should receive sound notifications.
Emphasis on in 2014, that was forever ago in Internet time. XMPP is steadily moving forward and the things mentioned in that post have improved a lot. Check out https://conversations.im/ for example.
This is cool. But IMHO the point of the article is about Google losing its first-mover advantage in chat apps due to wrong branding and prediction for GChat, not interoperability.
True. My comment is not a response to the particulars of this article, but more about the area of software of which GChat was a member. Apologies for turning it into my own soapbox.
Sure. Homeserver software doesn't care where it's hosted. If you want it to federate with other homeservers on the Internet, it will need to be accessible from outside your local network, of course.
I'm not sure exactly what you're asking. The only practically usable homeserver software right now is Synapse (https://github.com/matrix-org/synapse), an implementation in Python built by the Matrix core developer team.
I somehow didn't process the "it's a smartphone part" and completely misunderstood his question as a result. I'll edit my original reply to correct that. Edit: Apparently I can't edit it anymore. :{
Yes, it is possible. Ultimately, a homeserver is just a program that implements the client-server, and optionally the federation, APIs. The only feature-complete homeserver right now is Synapse, which is written in Python, and as of this moment has performance issues even running on a server, so it's not realistic that it could be modified to run on a phone. My homeserver Ruma is written in Rust, which is a systems language that actually is suited for mobile and embedded systems development, although I am currently not targeting that use case, nor is Ruma mature enough to be practically useful yet. If you're interested in the possibility of developing your own homeserver implementation specifically targeting your use case, you should take a look through the Matrix specification documents: https://matrix.org/docs/spec/
I really like the ideas behind Matrix. Sure, the existing (open) clients aren't all that sexy, and synapse can be a bit slow sometimes. But implementing your own client is a breeze and bridging other networks through application services makes it incredibly powerful.
My experience with the matrix.org community has been very pleasant so far. Keep up the good work, folks!
Matrix is very usable today via the Vector client (https://vector.im/). You can connect to the public server run by matrix.org, or you can run your own server that federates with all the others if you'd like to maintain ownership of your data. Matrix can also bridge to existing systems like IRC, so there isn't necessarily the need to convince your friends and family to "switch" with you. There are interoperable clients being developed for other operating systems including mobile platforms. Even with client support not all the way there yet, the spec and the system are more promising than anything else out there today.
If you're interested in Rust, you might also want to take a look at my project, Ruma (https://www.ruma.io/), which is a Rust implementation of the Matrix system. It's in very early stages, so don't expect to use it at this point, but it's progressing steadily.