Hacker News new | comments | show | ask | jobs | submit login
Easy XMPP: What are we doing here? (jabber.org)
378 points by psiconaut on Jan 16, 2017 | hide | past | web | favorite | 165 comments

These would be my concerns about potential differences between Signal and an 'Easy XMPP' client; would someone who knows Signal say whether these are accurate?:

* Signal users are not anonymous; Signal requires users' phone numbers.

* Signal is centralized. Is there a way to run your own Signal server?

* Signal uses Google Chrome on the desktop (and Android?) and Google Play Services (or some part of them) on Android (I don't know about iOS). Whatever you think of Google's intentions, they are one of the leading surveillance organizations in the world. Signal users must trust Google.

* Is Signal's system (not user data) fully open and transparent, end-to-end?

* Would I need to trust Signal more than I would need to trust Jabber?


EDIT: I came across this comment from Moxie in late 2015 which addresses some of these issues and provides a broader view:


If we were going to rank our priorities, they would be in this order:

1) Make mass surveillance impossible.

2) Stop targeted attacks against crypto nerds.

It's not that we don't find #2 laudable, but optimizing for #1 takes precedence when we're making decisions.

If you don't want to use your phone number, don't use it. You can register with any GV, Twilio, Voicepulse, or other throwaway VoIP number.

If you don't want to run Chrome, use Chromium instead.

If you don't want to use Google Play Services, use GcmCore.

The world you want this software for is not the world that everyone else lives in. You can certainly make it work in that world with a little effort, but because of how we've prioritized our objectives, that's not the default experience.

Signal requires users' phone numbers.

True for Signal. XMPP is using JIDs which are functionally comparable to email addresses, and similarly anonymous (or not).

Is there a way to run your own Signal server?

The Signal client is open source, the server mostly so. You could create your own semi-Signal community, but it would not be able to talk to official-Signal users.

XMPP is designed for federation by default, and you can run your own server, akin to email. Though some server operators decide to disable or cripple the federation feature (Facebook, Gmail).

Would I need to trust Signal more than I would need to trust Jabber?

This is a complicated matter. You need to trust the server operators to handle metadata in both scenarios, but you can run your own server with XMPP. You need to trust the client developer, which is Signal in one case, or the developer of your choice (or yourself) in the other. You need to trust the device manufacturer/Google in both cases.

What is this constant rumor about that Google killed XMPP federation? I think they killed the XMPP support in their own clients, but in general they still provide the service. I've at least three active @gmail.com XMPP accounts in my roster on a private server. I've read some rumor that they do not do TLS for server2server traffic, but in the end if you use XMPP on your desktop you most likely use OTR.

In 2013, the xmpp community decided to enforce TLS on all server-to-server links [0]. Google still doesn't support this, essentially preventing non-gmail users to communicate with them.

While OTR is a possible solution, it has its own can of worms (multi-device, offline use).

Besides of this, Gmail xmpp silently fails if the other party upgraded to hangouts.

[0] https://github.com/stpeter/manifesto/blob/master/manifesto.t...

Apologies in advance for pedantry, but OTR and TLS aren't comparable security solutions. Unlike TLS, OTR is end-to-end. You can't trust the security of an XMPP conversation that occurs over TLS connections without also trusting all the servers in between.

It's not a rumor. Yes, TLS doesn't work, but that's not a big deal, because you can still allow unsecure connections.

The issue is that you see 3 people are online, you can also receive their messages, but they can't see anything you write[1]. This behavior is worse than outright blocking, because it creates confusion at first seems like everything works and why people are complaining, then makes you suspicious that perhaps Jabber is broken, and in order to talk to your friends you might start using gmail.com.

[1] Initially this happened to anyone who converted to hangouts. I don't think there's option anymore to opt out from hangouts and get back to GTalk.

I think it's more than a rumor. As far as I know, Google stopped allowing new XMPP servers to federate, because almost all of the new servers that were trying to were sending spam.

It's possible that they grandfathered you in, if you were already federated when they put on the breaks, and your server wasn't guilty of sending spam.

Or it's possible that I'm wrong and they just got really aggressive about banning new servers if anything even remotely spammy came out of them, and it's really the spammers complaining that "Google won't let anyone new federate."

Google is moving away from XMPP everything. They recently deprecated all XMPP related services in their cloud suite. [1] The XMPP API will be permanently disabled October 31, 2017. Previously they've deprecated their Google Talk windows client, and said that they plan to focus their efforts on Hangouts. [2]

[1] https://cloud.google.com/appengine/docs/deprecations/xmpp

[2] https://plus.google.com/+MayurKamat/posts/ETBvtp9VYav

Phone numbers are not always anonymous. In India and I am sure some other countries as well, you have to give a photograph and a government ID to get a SIM card. And that is why I am hesitant to use Signal: I want to talk to you but I am not sure if I should give you my number?

For a number of years in Norway it used to be the case that you didn't need to show any ID or similar when buying a prepaid sim. Thus it was popular among criminals, and the related scratch cards circulated as virtually as cash. The police raised a stink about this situation, and soon enough you need ID to get them.

Wow, really? I'm so used to just buying SIM cards on street corners, I didn't realise some people had to go through documentation to get them.

Also, in some countries like the US, you can buy 'burner' phones without any need for identification--no contract, no ID.

In Australia you need to show ID to get a SIM card, even just a $2 SIM from the grocery store.

> Is Signal's system (not user data) fully open and transparent, end-to-end?

I believe that it is not transparent in the usual sense, because the server-side component is has no source available, but also that it does not need to be transparent in a cryptographic sense: the amount of information known by the Signal servers is minimal, and doesn't include messages, contact graphs, or even which users are communicating with each other. It does include what users they have and what phone numbers, and when they log in / where from: I assume they could also learn whether someone is using Signal at a given time and how much (from bytes transferred) but nothing more interesting about the conversation. I believe the use of Google Cloud Messaging is the same way.

I haven't looked into the protocol myself and I'm not sure if all of this is true. In particular, while OWS says they don't have any "records" of who a user is communicating with (https://whispersystems.org/bigbrother/eastern-virginia-grand...), I'm not actually sure if this means they're just not logging it, and a system could be built to record this.

So how about comparing rather than with Signal, instead with Conversations (https://conversations.im/)

It's XMPP based and offers many Signal-like features. Seems very similar to the goals of this.

You're fighting the wrong battle here in order to get people to use better tools. Most people don't care if it's Free Software or not. They want something that's simple to use, and offers reasonable security. Right now, XMPP is not, due to the awful clients out there, and the attitudes towards any sort of comparisons with products that they are less philosophically aligned to.

This is perhaps the biggest problem Free Software programs have when you start treading away from things like a browser. There is a huge hostility to ease of use and building things that are Powerful Enough. People don't care if you have a lisp scripting language embedded in your app, they care about how long it takes to get started doing the stuff they care about with your software. Until usability becomes more important, then XMPP isn't going to gain traction.

I think Google is only used to send empty notifications to android and chrome devices, right? IOS too, maybe? Not that this nullifies your point about Google. There's just not much for them to collect if what I asked is correct.

This is my understanding as well. However even this requires you to have various google-parts installed on your phone. These can be fairly obtrusive with permissions and such.

My understanding is that Play Services sends information about the user to Google periodically.

Correct. And Signal’s .apk includes a Google library for cloud messaging, but that library also collects information about the user and sends it to Google, even if the user does not have Google Services installed.

Well you still need to trust that Google/your device manufacturer isn't simply reading everything on your device's screen.

I recently tried riot.im and I'm realy blown away by the good UI they have and how easy it is to get started to develop your own stuff.

It is a shame that nothing by the likes exists in the XMPP-sphere.

EDIT: As a side note: Daniel Gultsch from the conversations-fame is doing a great job by providing/developing a realy awesome XMPP client for android and pushing the standard forward.

I registered when it was called called Vector and recently started using it more seriously. It is freaking fantastic. Voice/video calls actually works and the Android app is very good. It was long since I had such as strong feeling of "this is it".

When it comes to protocols (as opposed to "services"), they're seldom replaced. I mean, we're still using IRC for open source projects. Matrix/Riot gives me a glimpse of the future.

I recommend reading this: http://www.titus-stahl.de/blog/2016/12/21/encrypted-messenge...

> I mean, we're still using IRC for open source projects. Matrix/Riot gives me a glimpse of the future.

Matrix/Riot does tend to break down in channels with 30k users all chatting, sending hundredthousands of messages per minute. (Which is a real use case of IRC).

I guess you're talking about something like Twitch? That's certainly a use case but a rather extreme and rare one. Among public IRC networks I believe QuakeNet has the record with a peak of ~240k users in total on the entire network (that was a long time ago). Channel record on QuakeNet was ~10k users. These days it looks like only two IRC networks have >30k active users in total (according to http://irc.netsplit.de/networks/) - freenode and IRCnet.

Quakenet frequently reaches 8-11k users multiple times a year, for example in #eurovision during the song contest, or in #election last november.

Other networks have similar events.

And yes, most of the problems I describe only appear on Twitch, or other livestream chats.

Right. I do share your concerns about performance, but I'm sure it will improve considerably. A few months ago on reddit, one of the Matrix core devs (ara4n) answered a question about how much they can improve performance:

"enormously. much of synapse's algorithms and DB schema are still unoptimised, plus python is not exactly renowned for being super-fast. we're effectively going through rewriting chunks of synapse - e.g. the new state storage representation in 0.18, and meanwhile Ruma is going through writing a cleanroom impl in Rust that should be a bajillion times faster :)"


Ruma: https://github.com/ruma/ruma

I'm ignorant on the subject, how can hundred/thousands messages per minute be a real use case? I would expect such massive amount to be parsed by robots, not humans, and in that case why use IRC at all instead of a queue system like RabbitMQ? Maybe because it's easier to share an IRC channel than an open queue?

Well, take QuakeNet's #eurovision during the ESC. Tenthousand users, all spamming as fast as they can type.

If that is a meaningful discussion is another topic, but many chats for live evenys, especially on Twitch, have these numbers of users and messages.

Think of it as a crowd cheering. You can't make out any single voice, but there's a coherency to the noise. In the same way, mass-chats conform around smileys and memes in response to whatever the crowd is viewing together.

How does it break down? The largest Matrix rooms we've seen have got ~50K users in them with very high traffic, and Synapse is good for at least 10 msgs/s currently (with orders of magnitude improvements on the horizon).

Try bridging a twitch chat into matrix, and you'll see message latency go up massively, and performance go down to be unusable.

There is a reimplementation of the Python based server in Rust going on right now. The Python version has proven really slow in places, and Rust is probably a great answer to shortcomings in Python performance without having to change semantics much.

A native server with a socket-based and not longpolling based transport system would indeed solve many of the issues.

But even then, Matrix will likely not scale well enough to replace Twitch's backend, or to be able to nicely handle QuakeNet's event channels.

You'll also have to have a native client at that point, because the webclient also breaks down at those amounts. And you'll need to use a binary protocol or simplistic text protocol between client and server, because otherwise the user's mobile connection won't be able to handle that traffic.

Matrix will have to change a lot of things before that scaling is easy.

Would that rust implementation be https://www.ruma.io/?

Why is there a lack of screenshots of the clients on the Riot.im website? sigh

Maybe because you can try it directly online without registration?


not the android app

The UI is almost the same.

no, the web app / desktop app UI is terrible. The android and iOS apps are pretty usable.

Have also been looking at Riot and the underlying Matrix protocol (http://matrix.org). Also impressed.

Have you tried XMPP's client kaiwa ?


This looks great. I've often thought the biggest issue holding back xmpp adoption is the lack of web clients. I'm convinced e-mail would be dead if it wasn't for webmail clients. People will eventually graduate to a native client that suits their needs but finding a client and installing it as well as finding a xmpp service provider is a much bigger initial hurdle than just trying out xmpp because a friend recommended you a service. I've been thinking about hosting my own xmpp server and I will definitely keep this project in mind if I ever make accounts for friends.

> blown away by the good UI

I can only imagine an engineer could see it this way. I found it rather confusing. I tried joining #debian and I kept getting "invitation" messages from some IRC bridge.

Reflecting a bit more, I think that if I stayed away from IRC it would have been less weird. Now I'm just wondering how the web client works. Do I trust my encryption to JavaScript?

And now after actually trying it out, photos and videos work pretty well. I can see this is a work in progress. So far so good, they just gotta hide all those public groups away somewhere. They could use a harsh critique from Moxie about real users I think.

Riot looks nice as far as I can tell. Is there any documentation, proper list of features, what kinds of bridges exist, how to use them etc? The info on riot.im seems really sparse.

See https://matrix.org/ for all that. Riot is the flagship Matrix client, but there are many other alternatives in various states of development.

For bridges that you can run on your on "homeserver" (if you indeed want to run your own server!), see https://matrix.org/docs/projects/try-matrix-now.html#applica....

If you use the matrix.org homeserver, they bridge with a number of IRC networks, including freenode and OFTC. The closest thing to an official list that I've seen is https://github.com/matrix-org/matrix-appservice-irc/issues/2....

There's also Zom (zom.im), which like Conversations is derived from the Guardian Project's ChatSecure.

Conversations isn't derived from ChatSecure. Newer versions of ChatSecure were (are?) going to be derived from Conversations.

> A single decision by Moxie or a single court order in some country can make Signal unavailable to a large part of its user base.

This is a great point made later on the thread.

And already happened in some countries, for example, with whatsapp.

Social pressure was big enough to revert it in a reasonable time, but time is very relative on how much you or your business relies on it.

But even if Moxie does something weird or some country breaks Signal. You and your contacts can easier switch to another system.

While when your XMPP server admin does something weird or a country breaks it you are locked in.

For Signal you use an ID which is not owned by Signal. So for Signal and alike the decentralisation doesn't come from the service but from your own social graph which is your phone book on your smartphone.

Moxie explained it much better I think. https://whispersystems.org/blog/contact-discovery/

Is mobile number portability a thing in every country?

I suspect that relying on IDs owned by another system can have its drawbacks. Maybe it's not likely that an entire country is forced to change phone numbers. It's still possible to lose a number for various reasons (most entirely benign). An oppressive government could definitely disrupt communication of suspects by forcing the phone company(-ies) to terminate their contracts and unlist their numbers.

(But IDs being piggy-backed to a different system which is not easily disrupted on a mass scale does have some advantages, too.)

Good question, also it was only a remark that the phone number or email address is not owned by the IM system. They can change anyway.

I got my mobile number in 1999 and already switched twice with the number. Sadly it is kinda expensive to take the number with you, 25 to 30 euros.

> A single decision by Moxie

I'm sure that if Moxie turned evil it would be difficult for the project to survive, but that's also one of the reasons they've been successful. They have control over what they're building, and they can move fast and make breaking changes. It's very difficult to build something beautiful when you can't make breaking changes.

Genuine question--doesn't domain fronting make it difficult for states to censor Signal? Or is the idea that it's a cat-and-mouse game and the state will find a way to censor even with domain fronting?

Domain fronting helps prevent network-level blocking. It does nothing in the face of a court order directly to Signal.

People who care more about convenience than freedom and privacy can and do use Skype, iMessage, and Snapchat. If you give up freedom and privacy to make a more convenient client, you're not improving the freedom and privacy situation, you're just making more of the miserable proprietary software that we're trying to get away from.

If it's not free software, you have neither freedom nor privacy. If it's not decentralized or federated, you have neither freedom nor privacy. The only contenders for freedom and privacy are XMPP and Matrix. All the others are contenders for money and popularity, but not for freedom and privacy.

Popularity and money are useful and not inherently incompatible with freedom and privacy, but they are secondary. Creating a new application that is not federated or decentralized does not help, no matter how much more popular or convenient it is.

Except that messaging services are useless if the people you want to talk to don't use them. If all the people you want to talk to share your prioritization of freedom & privacy above everything else, then you're probably ok using a federated, self-run XMPP style messaging system.

Unfortunately, for the vast majority of people, that just isn't the case. I would love it if I could use XMPP-based services to talk to everyone I care to talk to, but, in practice, I use 8 different messaging services, and all of them are walled-garden non-XMPP services.

In the end, the people I know who care about privacy and security communicate with me using Signal (can count that number on one hand), and the rest use one of the others.

You can say and wish all you want that popularity and money are secondary, but in the world we actually live in, they're #1. I expect that won't change until and unless something horrible happens and "regular people" have their privacy compromised in a way that causes them real, tangible, extreme harm.

> Except that messaging services are useless if the people you want to talk to don't use them.

This applies to centralised/proprietary systems as well, including those 8 walled gardens that you use.

When Slack was created, nobody was using it. When Signal came out, nobody was using it. Same with Telegram, Skype, Twitter and all the other centralised/proprietary systems.

Some of those did manage to gain enough users for positive network effects to kick in, so "nobody uses it" isn't specific to FOSS/decentralised/federated systems, it applies to messaging systems in general. The fact that so many messaging systems keep being created, and so many users hop from one to another, shows that it's not a particularly strong argument either.

The real problems for systems like XMPP are the levels of investment required in marketing, UI design, etc. to compete with the proprietary systems.

Right, I think the point I was -- poorly -- trying to make is that your average user doesn't care about federation (or openness). It comes down to which service has the best marketing and can capture the most market share.

I was under the impression iMessage has end-to-end encryption?

Yes, but there is no way to verify fingerprints, so trust is still very much client/server based.

The real issue is federated IM vs non-federated IM. Anyone can make a IM system work off a single server. People do it all the time and we have had a series of incompatible IM systems in the past. Signal and Whatsapp are just the current flavours. Soon they will be gone and there will be new hotness. At this point I consider any non-federating IM system to be part of the problem.

As mentioned by someone in the linked thread, part of the user problem is that users can't even conceive of a federated IM system and don't know what one might be like. Asking someone to get a XMPP client so they can communicate with you normally just ends in confusion. There is no download button on the xmpp.com site.

* https://mail.jabber.org/pipermail/standards/2017-January/031...

I'd argue the future isn't just federation but bridging. We have already seen repeated attempts on the client side to abstract IM platform - Pidgin being the largest, but a ton of effort was invested into the Telepathy framework for about a decade.

The adoption rates show a general failure of the model.

Server-side transparent interoperability like Riot has with several services, however, seems extremely promising as a way to break that cycle.

How does it differ from XMPP server-to-server gateways?


Bridging is the problematic issue that gave birth to federation, so I'm pretty sure you're mistaken here.

I said isn't just, meaning you need both.

XMPP was a tower built too tall and wide and it crumbled under its own optiomal extension weight.

Just bridges mean you are still selling your soul to a primary operator, which prevents trust.

Federation and bridges without extension hell means you can use whatever homeserver you want, bridge to whatever you need to, and finally have working interoperable IM.

  > Asking someone to get a XMPP client so they can 
  > communicate with you normally just ends in confusion. 
That's not entirely valid. I mean, look at bittorrent, they had an ^official^ client and then there were other clients and people who did not know anything about configuring their clients still managed to use them.

Asking someone to used an XMPP client, ideally should be the same as asking someone to use a torrent client.

> Asking someone to used an XMPP client, ideally should be the same as asking someone to use a torrent client

It is the same, and the majority of people can't manage to do either.

The context of the discussion ("Easy XMPP") is an attempt to fix the UX of XMPP clients, which starts with a set of small steps documented in the pages linked from https://wiki.xmpp.org/web/Easy_XMPP

I think the issue with XMPP is the myriad of XEPs (XMPP extensions) which are optional, causing excessive fragmentation.

Some clients are excellent, like conversations.im. The problem is accidental complexity. Perhaps a solution would be to have a meta XEP that aggregates a few basic XEPs and defines a minimum common denominator.

There are attempts at creating an XMPP compliance suite[0], but the biggest problem is actually a lack of volunteers / money to improve the clients and servers. Conversations is driven by one full time developer, most other applications are hobby projects.

[0] http://xmpp.org/extensions/xep-0375.html

It was a real sad day when Facebook and Google decided that proprietary was the way to go...

The writing was on the wall before that, though. WhatsApp was eating everyone's lunch and Facebook / Google wanted to move faster and own the messaging space.

WhatsApp didn't exist yet

WhatsApp has existed since 2009, which preceded Google Hangouts by 4 years.

Afaik facebook was designed to be a closed walled garden from the get go and never offered interoperability despite using xmpp at first.

Google gtalk predates facebook and happened at different times, IIRC the change to a closed chat service was a reaction due to facebook abusing the openness to scrape contacts.

What kinds of skills would a volunteer need to be able to contribute?

What skills you have are laregely irrelavant; just find something that annoys you personally about [your favorite client here], dig in, fix it, and send in a patch. Or, if you're not a developer, use whatever skills you have, mock up a way that a UI component could work better and send it in, or write documentation, or simply file bug reports. Everything helps :)

Also, most people are introduced to XMPP in ways that don't let group chats work very well, so they think that's an XMPP problem, not the service's.

How is that not an XMPP problem?

I meant that "users perceive it as something XMPP can't do", so it's not a protocol-level issue, as the feature exists, but a branding and compatibility issue. Those, depending on how you define XMPP, are part of XMPP's problem.

Because it works fine with the right client.

I'd really appreciate more detail, here.

If everyone is using Conversations as their client, group chats work fine. Issues arrise when you introduce other clients into the mix ...

So Conversations has implemented a feature that isn't in the protocol, but important enough that when the feature doesn't work, users are turned off?

Or it's part of the protocol, but design decisions lead to client interoperability problems?

Maybe there's a third answer, but both of these are problems with XMPP.

It's part of the protocol, but many clients don't implement it correctly.

Multi user chats/chatrooms are part of the protocol though.

I'm not sure "popular clients" is a good thing to look at; Skype UI isn't great (and really bad if you use the IM part of it).

On this topic; I wouldn't mind hosting my own XMPP service, but most clients aren't good enough (specially on Android). I can't advocate for a service (protocol?) that can't offer some basics that are covered and seamless everywhere else (eg, sending files, showing media files in the client itself, etc).

I understand standards are slow, but when I think how difficult was to get avatar support everywhere... it makes me sad.

After many years, I gave up on XMPP recently for "internal family" communications. I'm using Telegram now, although it really bothers me that I need a phone to start using it.

I thought Android is one of the best-supported platforms with conversations.im?

Yes, you're correct; but I found the UI too ugly and at that point I was already too frustrated. It's probably me, but I had put already too much time on something that shouldn't be an issue.

Fair enough, and not good if that is the feedback about a well-supported plattform. Still looking for a more-than-basic desktop client.

That's interesting; what Android apps do you like the UI of, out of curiosity? Conversations basically follows the Android design guidelines to the letter, so I'm curious if it's all Android apps or just something about Conversations. The only problem I ever have with the UI is the unreadably small font size (but at least there's a setting for that).

I have 0 friends using signal so it's hard to give it a try: Did they fix the multiple devices end-to-end problem? Can I continue a conversation from my smartphone on my desktop? (should be possible by mirroring the conversations and having 2 keys). This was what killed OTR for me (and that it fails on unstable mobile connections, and the setup).

Yes, you can carry the conversation from phone to several desktop platforms. It uses a Chrome extension to achieve that.

You can't have two phones synchronizing yet, though.

Yes, you can't have two phones, or a phone or a tablet, or two computers synchronizing. It also won't synchronize SMS conversations, it'll hijack the SMS datastore if you want to use the seamless mode and do a bunch of other annoying stuff :/

You can workaround it somewhat by using groups and join both of your devices. The cryptographic primitives for multi user messaging are there.

Your crypto-enthusiast friends will be like "why is our conversation CC'ed to another number?" and you'll have to explain to every single one of them that it is a multi-device workaround.

And then when the NSA decided that they need their number CC'ed, your friends won't ask you...

There is no way to use signal on a desktop unless you are willing to give in to google and use google chrome which is far fetched for an app supposed to protect privacy.

But I guess requiring a smartphone for a privacy anything is quite far fetched too.

I'm not sure when the last time you tried it was, but that feature has been working since I first started using it, well over a year ago.

It _could_ be better in that the "desktop" client is just a Chrome app. Personally, I would find even a standalone Electron-based app to be a better choice UI-wise.

Ugh, yes, I don't like these chrome apps. Even hangouts randomly clogged my CPU or died silently. Also firefox is my home. Bummer :/

I think there is still place for an end to end encrypted Jabber app with an option to self host the server. This would avoid the single point of failure issue that Signal has (Signal servers being blocked in certain countries).

Also for companies where we want to self host and not have our communications routed through a third party.

What federated / decentralized protocols have been actual successes? For what protocols can I set up my own server in my house or in some cloud service, and have a comparable experience to using a major provider's service (modulo scalability and personal sysadmin effort)?

I'm worried that the only ones seem to be email and the web, both of which came into existence when the internet was small and academic and it was natural for universities to decentralize. And running email on your own is getting increasingly hard because of spam and IP reputation. (We seem to have more-or-less won the war on spam, but at the cost of making email much less decentralized than it used to be.)

There's a mention of BitTorrent elsewhere in these comments, and there might be an argument for Bitcoin. But even for IRC, people tend not to run their own servers (although they could); there are a very small number of IRC networks, run by random people.

I would love to see a decentralized and federated team chat app along the lines of Slack or Discord, but I'm having trouble believing that such a thing would have a chance of success in a post-1995 internet.

Correct me if I am wrong, but https://riot.im/ seems to fit the description of a decentralised and federated team chat app.

> running email on your own is getting increasingly hard because of spam and IP reputation.

Are you sure ? https://mailinabox.email/

I am not sure, but that page doesn't talk about deliverability at all, as far as I can see. Is it generally true that spawning a mail server on a random cloud instance will work for deliverability? I would have assumed that any given cloud IP has a nonzero chance of having been used by a spammer in the past.

(Or, in other words, if this does work, why do people who have small- to medium-scale deliverability needs use a dedicated email-sending provider instead of just using a t2.nano running an AMI with Postfix?)

The technical merits and drawbacks of XMPP aside, deployment only works if there's an appetite from deployers. For high-visibility consumer chat that average people use, this appetite has vanished.

Around the mid-2000s, IM networks started getting tired of constantly changing their protocols to thwart third-party reverse engineering efforts like Microsoft logging into AIM, libpurple (Pidgin), or Trillian. But then Google Talk appeared [1] in 2006 inside the coveted invite-only Gmail, supporting XMPP, and significantly raised the bar.

So interoperability became a tool to maintain market share. The underdogs WLM and Yahoo started seamless interop [1] in July 2006, while Google Talk and AIM started a limited interop [1] in 2007. AIM briefly dabbled with XMPP it in 2008 [2] (great source -- see comments for AIM's response).

In the meantime, Facebook opened up for everyone, introduced Chat and rapidly lured away the myspace/AIM generation, becoming a major player in chat. Facebook introduced XMPP in February 2010 [3] but discontinued it [4] in 2015 after having deprecated it the year prior. This neatly coincided with their announcement to monetize the Messenger ecosystem, in ways that require a captive client [5].

Other vendors are similarly pursuing monetization within the client -- Snapchat and Kik as a content portal [6][7][8][9], Google as a context-aware assistant, Microsoft is lost at sea, Whatsapp as a Facebook data mining scheme, the Asian apps as a combination of all other techniques and microtranactions -- when anyone can bring a third-party client, their monetization strategy suffers. This makes XMPP's deployment future exceedingly bleak, perhaps restricted solely to corporate deployments.

[1] https://news.ycombinator.com/item?id=11114518 [2] https://web.archive.org/web/20080120143857/http://florianjen... [3] http://web.archive.org/web/20100318030410/http://developers.... [4] https://developers.facebook.com/docs/chat [5] https://developers.facebook.com/blog/post/2015/03/25/introdu... [6] https://news.ycombinator.com/item?id=11935956#11941090 [7] https://news.ycombinator.com/item?id=12000854#12002773 [8] https://news.ycombinator.com/item?id=12206846#12207459 [9] https://news.ycombinator.com/item?id=12272973#12273447

Great footnoted history....I will reference this when younger developers ask me the "WTF messaging" question, which happens a couple of times a year. I suspect it will be useful for many years to come.

> Microsoft is lost at sea

Nice sum up of events, though you missed that Microsoft bought Skype.

I think the real issue here is that too many people conflate XMPP the protocol with the various XMPP based services (most of the smaller free public services do this). If we tell people to "go sign up for an XMPP account" it's obviously going to be too complicated; they're going to search, find the XSF websites or a bunch of random libraries and protocol information, and give up. Meanwhile, companies can use XMPP and build their own brand around chat products based on it (while never mentioning it, because their users don't care) and regardless of whether they choose to federate or not they can reap the benefits of a community of protocol developers, the plethora of libraries, and maybe even existing client and server code.

Didn't OpenID have this exact same problem?

Interestingly, email didn't, despite acronyms like POP3, IMAP, SMTP being a frequent occurrence once you try to supply an email client of your choice. So what was different about email, once internet users started escaping the walled gardens of old?

Users were all registered by default to an email account at their ISP. All of them could send emails back and forth to people inside and outside their ISP. When they started to suck, they registered new addresses on specialized email providers, and they kept their contact list with them, so emails could still be used. Because emails was pretty much the first way of communicating and because of how common it is it won't disappear soon.

I don't know the history of OpenID at all, and don't know the history of email well enough to comment, but I suspect email didn't become a common word overnight, and I doubt most people would know how to use a client that asks them for SMTP settings these days (they expect to just visit gmail.com and see messages, or maybe at the most complicated install Thunderbird, enter their username and password, and wait while it says "autoconfiguring" for a bit).

It would (will..?) be a crying shame to see XMPP grind to a halt - but I suspect, sadly, that the honest answer to the question is "not one heck of a lot really". I don't know if it got started too early, or moved too slowly, or what - but in the end, it missed its boat.

XMPP is great for back-end applications and internal messaging for businesses, and is not going away, the issue is the "consumer space" where every big vendor has its own XMPP like protocol but slightly broken to be incompatible with generic clients.

what's the universal alternative for XMPP?

In the U.S.? SMS is as close to universal as one can get (which is to say, not very).

XMPP is still pretty relevant in the server to server space and in large companies requiring a scalable API which offers async, full duplex comms over a single socket connection and offers libraries and message brokers in pretty much every language.

That said I do believe some of our XMPP customers are starting to look around and ask for REST/Websocket based APIs as their new dev team hires look to reinvent the wheel ;)

Seriously speaking though - I think XMPP missed the boat as far as messaging goes. Smart phone apps took everyone unawares and XMPP didn't move fast enough to provide a solution for low power devices on high latency networks.

That said I wonder if there is a future in which XMPP does prove to be a compelling solution. I wouldn't put money on it, but even today a full duplex API which is highly secure, async and which offers schema enforced validation of your messages sounds pretty damn compelling. That said there is a lot of progress with things like STOMP, RabbitMQ etc. providing the pieces necessary to replace XMPP.

It's a pity since XMPP makes it so damn easy to build reliable, realtime apps.

I wanted to like XMPP, but I never did. It drowns developers in complexity so they never get around to solving anything interesting or useful.

Bonus: I once discovered I had two of the main culprits for XMPP getting one of its worst misfeatures (lack of sensible framing) in the same room so I got to yell at them for it.

XMPP was form over function. And it wasn't even pleasant form.

In a sense the problem with XMPP is that it is too simple. If you are willing to type some XML you can log into a server and have a basic chat session over a telnet session. Hence the requirement for extensions to do anything else.

The alternative is to try to anticipate everything anyone would want to do with a protocol. That has problems as well. The XMPP approach is to accept everything and aggressively ignore everything not supported. XML is quite compatible with this approach.

XMPP uses the XML as the framing. An approach that could actually be considered clever if one is striving for simplicity in a protocol.

Good protocols have a layered design where you deal with different concerns in a fashion that promotes simplicity, robustness and performance. For instance first the complexity in framing individual messages, then the way you represent the payload (on at least 2 levels), and then the semantics you can layer on top of that.

Using XML framing in XMPP was the opposite of simplicity. (Sure it was simple in the sense that it required zero experience with actual implementation of protocols to arrive at the conclusion, but the result was something that was harder to implement properly).

It is "simple" for moronic, bad, implementations of the protocol, but it only complicates the situation when you need performance, quality and efficiency. It complicates it greatly. In essence, you end up having to write two parsers: a shallow framing parser and a deep parser.

And if you think you are going to get any help from the fact that there are lots of XML parsers: I've got bad news for you. There's lots of XML parsers that are meant to parse documents. Not millions of simultaneous, "endless" streams of data from dodgy clients.

XMPP is not good protocol design. It is brutally stupid protocol design.

(An irony is that right now there are several areas where you would want to use a messaging protocol for small devices. This ought to be the moment where a messaging standard had a chance to shine. And XMPP ends up being one of the least desirable protocols because so little care was taken in designing it)

> There's lots of XML parsers that are meant to parse documents.

You obviously wouldn't use such a parser for XMPP. You would use a parser designed for the purpose. Very few people would ever have to even think about the issue, there are XMPP libraries available for pretty much any environment in existence.

This is true for people who write applications. It is not true for people who would like to write servers and XMPP libraries that are somewhat efficient.

XMPP also isn't usable for many new IoT applications where it should have had its opportunity to shine, but where there just isn't enough memory to deal with the ridiculous bulk.

But I do get that most developers don't really try all that hard. I mean, look at the resource usage of Slack. It is an application that does _nothing_ that requires CPU, yet it gobbles it up. It tells you something about the kinds of people that write chat applications today. Not exactly the sharpest tools in the shed.

>...the ridiculous bulk.

Well presumably this hypothetical IoT application would have to support TCP/IP. Against that, a few hundred bytes of state table for the limited subset of XML required for basic XMPP would count for much.

You presume. I write code that has to deal with it. That's the difference.

The fundamental problem with decentralized messaging these days is push notifications. Apple, at least, makes it very hard for you to deliver APNS notifications without running a centralized server. XMPP and IRC will never work as well as a centralized service until the notification architecture changes.

Actually, XMPP has a solution for that for a year now [0]. The principal idea is that a client developer runs a push proxy component that forwards XMPP push events to GCM / APNS / whatever. The client tells its server which push component should be contacted and disconnects.

[0] http://xmpp.org/extensions/xep-0357.html

These proxy components have many of the same problems as centralized servers. If the proxy component goes down, your client stops working. You can't just download any random client and have it work forever. You have to trust that the developer won't shut it down. Paying the developer can give you this trust (see IRCCloud), but most people aren't willing to pay for something they can get for free elsewhere.

The advantage of slack over jabber is not only that the client doesn't suck (it's just electron) but that the server logs and creates a continuous experience for the user no matter if hes on mobile, in a browser or both.

The same can be provided by IRC with Quassel or IRCCloud, though.

It can be argued that there's no advantage of slack over anything because it is cloud based[1], is not fully cross-platform (no linux client), is proprietary closed source centralized service, and so on.

[1]: http://www.wired.com/2015/03/slack-admits-hacked-enables-2-f...

There's an XEP for the server to keep a message archive and provide the same experience. http://xmpp.org/extensions/xep-0136.html

What servers, and what clients implement it though? No idea...

Most things use Message Archive Management these days: https://xmpp.org/extensions/xep-0313.html

It's still experimental, but it should replace message archiving "soon" once a few final kinks have been worked out (that being said, most servers and clients implement it already and in my mind it's ready for production).

Discord also has this advantage.

> A single decision by Moxie or a single court order in some country can make Signal unavailable to a large part of its user base.

I think the solution here would be a "lightweight" federation of 3 or 4 entities following a common charter and covering a wide geographical area. If one drops out for some reason, the others will still ensure the service remains alive.

Then it is a single court order and 3 people agreeing on the same decision which is quite easy to make when it has to made at gunpoint.

The actual solution has been known for ages: decentralization, federation, peer-to-peer. pretty much what the internet was designed to be.

> Then it is a single court order and 3 people agreeing on the same decision which is quite easy to make when it has to made at gunpoint.

Not if the 3 entities reside in different countries/jurisdictions.

> decentralization, federation, peer-to-peer. pretty much what the internet was designed to be.

While I agree with you from the theoretical standpoint, it's been already shown that neither users nor providers care about that. A federation is not by any means easy to maintain (just look at the amount of work spent in keeping e-mail relatively free from spam and usable). So, either the federation in question is small enough that you can keep it under control or you're better off researching a fully decentralized system that is resilient to all the problems all other services have (doesn't sound easy to me).

Ricochet.im uses tor hidden services for messaging. No signup, no choosing your own name, no logging, since it uses tor hidden services, whatever security improvments tor gets, it should get. A while ago when there was an explosion in the number of hidden services running, I think it was this program. Thoughts?

We don't need another federated network. Administrators of those nodes are putting themselves into danger. What we need to develop is a purely decentralized p2p network that is resistant to censorship and eavesdropping.

Well it would be enough to have one proper client for each platform. On Android Conversations, iOS Chatsecure now with OMEMO as well. Desktop.. not sure. But to be honest, I think I'll prefer Matrix with Riot right now.

Has anyone tried https://coy.im/ ?

The idea looks cool but not that they explicitly don't want to support OMEMO sticking with old OTR: https://github.com/twstrike/coyim/issues/233

competition is healthy

Except when you've got so many clients and so many servers that the typical user gives up instead

Email exists.

When Slack is bought by Google and merged with Hangouts, and WhatsApp changes over to a new all-messages-delivered-by-the-NSA protocol, email will still exist.

> When Slack is bought by Google and merged with Hangouts

Hah, Google, have multiple messaging systems and bother to merge them?

And XMPP has always tried to be the email of IM. Similar model, address format, and so on. A sea of independent, autonomous domains openly connecting to each other.

Snail mail too exists. And will exist for quite a while still. About as relevant for personal communications as email.

Email survived because it is adopted as main identity provider, similar for SMS

I've recently been thinking about xmpp making for a great federated identity provider for multiplayer video games. With communication already built in, initialization of peer-to-peer sessions already a common use-case, and the right XEPs it could replace centralized services like steamworks, psn or xbox-live.

And the web of trust still won't work.

Email that is delivered via a network of NSA-managed servers?

particularly with the network effect.

It gets tiring to convince people to install Signal, Wire, etc.

argument to install Signal is pretty easy - I just ask Android users if they use Facebook Messenger to manage SMS. if answer is negative I propose why not use different nice SMS client called Signal which can also do encrypted online instant messaging

though many people are just too lazy to replace even ugly default Messaging app, so wet just use it with my wife and parents since I boycott Messenger/Whatsapp, just in case for video calls I have Skype Mingo (it's sad that out of billion people in India it's difficult to find one person which would upload always newest build to apkmirror)

That argument would not hold if someone knowledgeable is in the room as the same time[1]. Silence[2] would be the fork that actually does encrypted SMS and took it when Signal dropped it.

Why give away your SMS to a messaging app that purposely dropped their encryption to focus on online chat which is useless to me as I don't have a data plan?

[1]: https://whispersystems.org/blog/goodbye-encrypted-sms/ [2]: https://silence.im/

IMO, users in an organization engaging in debate and discussion about the best system based on the merits of the system to use is productive.

Signal is a centralized point of failure and surveillance.

Why can't I run my own signal server? Why doesn't the client support this?

We need something like Signal that that isn't under the control of a single party.

There was a piece recently about the proliferation of central server based chat services by young people and newcomers with total disregard that this was just adding to the issue of those not being to talk to each other.

For some reason the people are reinventing the wheel and the problems that come with it that have already been addressed.

We actually need signal to go away and disappear for the very reason that is a walled garden casting shadows on what we actually need to grow in the long run which would be matrix and xmpp at the moment.

Awhile ago I built a little hack that allows you to run your own Signal server:


That very much sounds like Matrix. Not only it provides a secure, federated WhatsApp/Signal-like experience, but it can replace Slack as well.

It also sounds very much like XMPP.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact