
Google defends dropping chat federation with inaccurate comments - AndrewDucker
https://plus.google.com/116276248303121270590/posts/ggNwh9eLYWu
======
kkowalczyk
Unfortunately, the guy responds with his own misinformations.

Google said: "For example, mobile has several requirements around bandwidth
and battery that are simply not part of the standard"

He says: "This glances over the the fact that the X in XMPP stands for
eXtensible, which still results in proposals for new protocol extensions every
month."

His misinformation is this: XMPP is XML based. The bandwidth that Google most
likely refers to is a result of verbosity of XML. You can't eliminate that by
extending XMPP, something that a guy who serves on XMPP Council should know.

The only way to fix that specific problem is to change the protocol. Which is
what Google did.

This is an example a bad advocacy: badmouthing companies that decided to no
longer use something you work on.

I also find troubling the pounding Google gets for every little mis-step.

When it comes to using and promoting open technologies Google is seemingly
from another planet compared to other tech giants (Apple, Microsoft, Facebook,
Amazon, you name it).

And yet the furor is not about Microsoft using completely secret protocol
(Skype), not about Apple promising to publish their secret protocol (Facetime)
and then not doing it, not about Apple preventing fair browser competition in
iOS etc. but about Google no longer using XMPP.

A lack of perspective.

~~~
magic_haze
Really? There are plenty of valid reasons why XMPP is not suitable for mobile
usage (most notably, the proprietary push notification systems that all
platforms insist you have to use), but XML isn't one of them. If you use TLS
with XMPP (as gtalk does [1]), you get the compression protocol for free, but
even if you can't, there's XEP-0138 [2], which specifies the use of zlib and
has been in development for over 12 years now, and finalized since 2009. XML
is fugly, but it works. At the scale Google operates, it is laughable that the
use of XML in chat protocols could possibly impact their bandwidth accounting.

[1]
[https://developers.google.com/talk/open_communications#devel...](https://developers.google.com/talk/open_communications#developer)

[2] <http://xmpp.org/extensions/xep-0138.html>

~~~
dmayle
Disclaimer: I am a Google employee, but I do not speak for Google.

I'm not offering an opinion for or against the decision to drop XMPP, but I
wanted to offer some corrections to what is being said.

I think you've misunderstood how scale works. At scale, problems do not
shrink, they grow.

I do not work on anything even closely related to Hangouts, but I used to run
an ejabberd server myself, and I've worked with SIP a bit.

XML does have problems, both with bandwidth, (even compressed), high CPU
utilization, parsing code is more complex (which leads to more security
issues). XMPP also has issues, and I'm not just talking about how hard it is
to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources,
the incredible number of failure states, etc.)

Cheers, Doug

#1 At least early versions of iChat used XMPP with SIP. For those who don't
know SIP, it stands for Session Initiation Protocol, which means that it
doesn't transfer the audio/video, it's just a language for making connections.
You have to negotiate a transport layer (which was often but not always RTP or
SRTP). You have to exchange connection ports via SDP. If you're lucky and
manage to get by all firewalls at this point, you now have to hope you're able
to use compatible codecs. Interoperability was a nightmare.

~~~
greggman
And yet google wave ran on xmpp which was arguably a far far more
sophisticated app doing way more communication than just chat

~~~
josephg
Google wave only used XMPP for federation. It was a nightmare to deploy and we
never got it working reliably. Different jabber servers all had different
quirks or implemented subsets of the XMPP extension spec.

The client-server protocol was simply JSON. XMPP was not involved. (Well, we
translated protobuf messages to JSON, so it was ugly JSON, but still JSON).

~~~
mh-
_(if you can't comment on this, no worries. thanks)_

    
    
      we translated protobuf messages to JSON
    

just curious- is this the same 'protojson' I see exchanged in the current crop
of APIs?

aside, any opinions on others choosing that as a standard way of communicating
between internal services on disparate runtimes? (golang<->python, for
example)

    
    
      --
    

_edit: offtopic, thanks for ShareJS._

------
dvt
Really? Can you so harshly judge Google on abandoning a (lets face it)
terrible protocol?

Maybe I could get over the fact that it's so complex -- which puts an
unnecessary strain on both servers and clients -- and maybe I could get over
the fact that (on average) servers consume 40% more memory than their IRC
counterparts. Maybe I could even get over the fact that the spec could change
at any moment (the eXtensible nature of it isn't necessarily a good thing, you
know).

But what I couldn't possibly in a million years get over is that... it's _XML_
(shudder).

\-- To the downvoters:

Here's some previous discussion about why XMPP sucks:
<https://news.ycombinator.com/item?id=2069810>

I'm currently working on a project that deals with real time communication
(think IRC for the new web) and after fidgeting around with XMPP for a couple
of weeks, I gave up on it completely. It's awful. I doubt many people have
implemented an XMPP client (or even worse.. a server). Try it some time.

~~~
aimatt
I understand the frustration of working with a fat protocol. What is a viable,
open alternative though?

~~~
dvt
IRC! :P Seriously though, there isn't a light one afaik.

I ended up making my own -- I based it on IRC but wrapped it in JSON. It's
stupid simple because chat should be easy.

~~~
dublinben
IRC is so lightweight, it's what NASA uses for their antarctic weather
flights.

[http://arstechnica.com/information-
technology/2013/01/antarc...](http://arstechnica.com/information-
technology/2013/01/antarctic-irc-how-nasas-flying-lab-stays-connected/)

~~~
mh-
antarctic satellite IRC?

best i:line ever.

------
davorak
> All that talk about Client Choice, Service Choice and Platform Choice has
> been replaced with "if the other big players don't play, why should we?".

It seems like the impact of the other big players not playing is down played
here. Did it not allow facebook users to start conversations with google talk
users but not vice versa?

Playing the cooperation card first is what google did, but in the iterated
prisoners dilemma you often need to play the non-cooperative card if that is
what your peers are doing.

That really seemed to be the core of the issue. Now 5-25 years from now maybe
google will have an entrenched culture of playing the non-cooperative card and
will not reach for the cooperative card first but I do not judge this to be
the case yet.

~~~
ralphm
I believe I get to downplay that because the federation they joined helped
grow their own network, and allowed them to do Google Talk for other domains.
I know many companies that have moved their XMPP stuff to Google for that.

Now those outside contacts are cut off, without any warning. Worse, they see
their inside contacts online, can send them messages that never show up in
Hangouts!

Those big players are hardly affected, though. Also, Google could simply block
those entities that don't fully federate (only).

~~~
BarkMore
_I believe I get to downplay that because the federation they joined helped
grow their own network_

Do you have any evidence that federation helped Google grow the network in a
significant way?

 _allowed them to do Google Talk for other domains_

Are you referring to Google Talk integration with Google Apps For Your Domain?
Outside evidence suggests that federation is not used between these domains.
For example XMPP SRV records are not required and ignored if present.

------
hayksaakian
It was ultimately a business decision - the same kind that leads to product
shutdowns and all the other "evil" things that new Google does.

~~~
steveklabnik
Oh, a _business_ decision. I guess we can just let them continue to embrace,
extend, and extinguish the commons we've built over the last decades, then. No
complaints necessary.

(I'm not a GOOG shareholder, so their business needs don't matter to me. I
_am_ now going to be cut off from my contacts, though. Part of markets is the
feedback that comes with mis-steps, and saying "it's just business" does a
dis-service to those of us who feel that Google has been over-stepping their
bounds.)

~~~
icebraining

      The [EEE] strategy's three phases are:
    
        Embrace: Development of software substantially compatible with a competing product,
                 or implementing a public standard.
        Extend:  Addition and promotion of features not supported by the competing product
                 or part of the standard, creating interoperability problems for customers
                 who try to use the 'simple' standard.
        Extinguish: When extensions become a de facto standard because of their dominant
                 market share, they marginalize competitors that do not or cannot support
                 the new extensions.
    

How does this fit in with what Google is doing? Where are their non-standard
extensions to XMPP (or RSS, etc) that weren't supported by the competition?

XMPP was essentially a non-player before Google used it, and it'll return to
being a non-player after they leave. There's no big strategy needed to
accomplish this.

This is not to say I support Google's recent decisions, but then again, I had
already moved off their applications (Reader and Gmail) way before the
shutdowns started, mostly due to the methods used to push Chrome.

~~~
steveklabnik
They embraced standards via XMPP. They then extended them in proprietary ways,
with Hangouts rather than gtalk. They then are deprecating gtalk in favor of
Hangouts, and will finally extinguish it.

~~~
jmillikin
Hangouts is not an extension of XMPP, it is a replacement.

Google's extensions to XMPP took place via the XEP process, and were published
so other vendors could implement them.

~~~
zobzu
That's exactly the same thing. Once you own most of the implementations
running "the standard" you modify it and close it so that nobody can follow
your path.

It does not matter if the said modifications are called "G-XMPP" or
'Hangouts'. The result and logic are the same.

The rest is a useless fight on semantics.

~~~
jmillikin
The Hangouts protocol is not a modified or closed version of XMPP. They have
nothing in common except that they're both used to implement IM clients.

SSH is not an extended version of Telnet.

~~~
zobzu
And then again.. you didn't get the point. SSH and Telnet do the same thing,
but nobody killed off telnet by getting everyone to use their implementation
of it, then kill it and propose SSH as alternative.

Thats what Hangout does. Name 5 of your friends without a google account who
use internet & mobile phones. See what I mean? They were using XMPP. Everyone
could talk to them using XMPP. Not anymore. Wanna talk to them? Gotta use
hangout, no way around it.

It does not matter if its based on the same _code_ what matter is that it
offers the same functionality AND kills off the previous product by using
their market share.

That's the freaking concept.

------
jjuliano
The problem with XMPP is that it requires several open ports (5222, 5223,
5269, 5298, 8010) on the client, on the technical and practical standpoint the
reason why XMPP is unpopular to big players because many businesses won't be
bothered to open their ports, now if you are a solutions provider creating
software for XMPP solutions for those companies, the most viable solution is
to use BOSH to direct XMPP traffic to port 80, but there's another big
problem, the only available BOSH library is a GPL BOSH library that you would
license in order for you to not to ship your software along with it's source-
code. Now if you can do the same thing over HTTP 1.1 stream, why bother with
XMPP?

~~~
BarkMore
* The problem with XMPP is that it requires several open ports (5222, 5223, 5269, 5298, 8010) on the client.*

XMPP does not require any open ports on a client.

~~~
jjuliano
On the server, I mean.

source: I'm an ex-cofounder of a now very successful syncing solution startup
who uses XMPP and I wrote the initial server and client code.

------
rakeshpai
I think I might have 2c to clarify the OP's stand, having worked on a mobile
XMPP client before. This is with regards to his point about bandwidth and
battery usage on mobile.

XMPP is a connection-oriented stateful protocol. The connection setup process
is very expensive: Connect, authenticate, advertise the list of supported
client features, get the roster (contacts), and IIRC presence exchange, etc.
Therefore, once you've established a connection, you really don't want it to
break. Even just reconnecting is very expensive.

However, in the mobile scenario, regular connection breakage is the norm. What
most mobile based clients will then do is proxy the XMPP connection on the
server-side and connect to the client over HTTP instead. This is well defined
in the BoSH spec (XEP:124). BoSH was originally designed for web-based
clients, but happens to meet the need of the mobile use-case.

However, when it comes down to implementation, there are several optimizations
that can be applied:

1\. The roster exchange at the connection time is very expensive, and usually
only sends data to the client that the client probably has already stored
locally from a pervious session. Connection overhead can be reduced
drastically if there was some way of versioning the roster and only
transferring deltas. This was the OP was referring to. (Roster versioning,
XEP:273).

2\. Presence packets are one of the most chatty aspects of the protocol (x
went offline, x is now online, etc). I've heard someone say that it
constitutes about 80% of the packets on a typical connection, but this is
anecdotal. In cases where the user isn't actively interacting with the client,
having presence packets being sent all the time seems wasteful. One way to
tackle this is to buffer up and de-duplicate the presence packets on the
server, and send them in batches only when necessary. (For example, if x goes
online, then offline, then online again, and then goes 'busy', the client only
needs to know that x is busy.) I think this is what the OP was referring to
when he mentioned the google:queue for delayed presence delivery. This alone
can be a huge win for battery and bandwidth, since transferring data and
processing it can be reduced drastically.

Aside: Presence is so chatty, that some mobile clients simply don't handle
presence at all. Take WhatsApp for example. It doesn't show whether every
person in your list of contacts is online. The assumption is that every user
is online by the nature of the device, and this assumption works in the common
cases. In the edge cases is where you'd need to buffer up messages. This
reduces chattiness drastically, making the protocol seem much more
lightweight. (WhatsApp is based on XMPP, but have spun it off into their own
thing.)

Just thought I'd add these points to the discussion here, since there seemed
to be some lack of understanding of the protocol. IMHO, the XML, as verbose as
it is, isn't really the problem. The protocol itself is verbose and whether
the transport uses XML or whatever else is of comparatively little
consequence. This verbosity of the protocol (and not so much the format of
data transfer) has its effect on battery life and bandwidth usage.

~~~
mh-
>Aside: Presence is so chatty, that some mobile clients simply don't handle
presence at all. Take WhatsApp for example.

One could also take Hangouts for example. :)

anyways, very informative post and good anecdotes, thank you for sharing.

my understanding of the XMPP spec is that _(assuming the server follows the
server-client spec, anyways)_ there's no way to unsubscribe from your
contacts' presence notifications and keep them on your roster [and therefore
able to be contacted without reauthorization, usually.]

can you confirm/enlighten me? thanks!

~~~
ralphm
Presence subscriptions don't have to be symmetrical in XMPP, and indeed you
can even have contacts on your roster that don't share presence in either
direction. As for exchanging messages, as far as I know Google's
implementation is the only one that doesn't allow that without presence
subscription. All others do.

In fact, this caused presence subscription requests to be the only attack
vector in terms of spam in Google talk, which some argue makes it _harder_ to
combat. Mostly because there is less data available to detect malicious
behaviour.

------
newman314
If the issue is with mobile, why not build a MQTT <-> XMPP bridge or just
support MQTT as an alternate protocol?

<http://en.wikipedia.org/wiki/MQ_Telemetry_Transport>

------
fakeer
1\. Microsoft/Facebook (and maybe more corporates) didn't play by the rules.
They don't believe in "give and take", they just do "take, take,..". Google
didn't like it so walked out. Instead of showing the sportsman's spirit and
instead d of carried on despite of all the follies they simply left having
pissed off. Good decision.

2\. Universities, organisations? The number is too small to be bothered about,
esp when compared with the likes of the two above^. They would rather innovate
and try to lock-in users into their own platforms rather than hoping someday
Cuck might turn the benevolent saint.

3\. Whatever you say but unless you are short of manpower and talent it's
actually easier difficult to innovate in-house sth proprietary and not when
you've to conform to a "standard" and that too when the rewards are [1].

4\. Even I don't like the move but it was a business decision and this one
made a lot more sense than killing off GR.

The surprising fact here is, we hardly see anyone criticizing the providers
that chose not to share back. We, keep on saying "no problems with XMPP" and
"even though MS and Fb were not giving back, Google should have just let them
free-load after all _it's a do no eveil_ corp". Huh.

~~~
onedev
I'm interested in how you think MSFT and FB just "took, took, took..."

~~~
fakeer
Thank you for asking.

Seems you have not been paying attention recently. Certainly not in the contxt
of _microsoft+facebook+gtalk+xmpp+google+talk one way_ , just google it; but
that's my assumption. This is what I meant by "took...3 times".

If it was sth else you wanted to know or were referring to then I obviously
failed to grok that and you'll have to help me understand.

------
kmasters
Isn't there a better question to be asked here? Federation is supposed to
enable, well, federation. Google is not preventing XMPP federation in any way.
Google is not "shutting down" XMPP or killing it.

Why is Google the company important to the XMPP community? I scarcely see that
it should matter.

Open protocols are not exclusive clubs or infinite universes, anyone can opt
in or out at any time, the rest of us will live.

I think its great when big companies such as Google or whoever participate in
open standards, but I also appreciate that they are in the business of
providing the best product for their users (in their eyes) and are free to do
what they want.

When they do participate, we are getting a gift from them. Its not their
obligation, right or even necessarily to their benefit. When they find other
priorities fine, maybe someone more committed will move the project forward.
And in other cases their NON-participation can be a gift as well.

So whats the big deal?

~~~
Macha
Google Talk was the largest federated XMPP server, and it provided an easy way
for people using XMPP to talk to non-technical users. "I'll add you on XMPP" =
blank stares, "I'll add you on Google Talk, just use your Gmail account"
works, even if you yourself are not using Google Talk.

