
WebSub: Open protocol for distributed pub–sub communication on the internet - dgellow
https://www.w3.org/TR/websub/
======
edhelas
"Hub deliver content to each subscriber with a POST"

It's quite common on some social networks to have several thousands/millions
subscribers for some pages/communities/accounts.

Is it really wise to build a publish-subscribe delivery system on top of HTTP?
This seems to be a huge overhead.

In the meantime XMPP is already offering similar features (XEP-0060: Publish-
Subscribe
[https://xmpp.org/extensions/xep-0060.html](https://xmpp.org/extensions/xep-0060.html))
for more than 10 years. It's implemented in several servers and can handle
huge loads without problems (everything is handled in real-time through
encrypted TCP sockets accross the network).

We are building social networks on top of XMPP for several years now, you can
check Movim ([https://movim.eu](https://movim.eu)) and Salut à Toi
([https://salut-a-toi.org/](https://salut-a-toi.org/)) :)

~~~
icebraining
What's the overhead of HTTP vs XMPP? A few headers? Doesn't seem that
different.

~~~
KaiserPro
"a few headers" if 50% of your payload is headers, thats a big deal.

Mind you XMPP isn't all that efficient either. as its all based on XML.

~~~
edhelas
I'm inviting you to read this page
[https://xmpp.org/about/myths.html](https://xmpp.org/about/myths.html) :)

~~~
KaiserPro
I didn't say slow, I said efficient. Its hilariously verbose compared to a
decent binary protocol. Failing that something based on protobuffers.

In the embedded world a JSON/XML parser eats a tonne of resources.

One could of course use it over satcomm as it says, but its hilariously
expensive when you are paying by the byte. But, compared to a massive JSON
goop with embedded pictures that twitter uses, its a paragon of speed.

------
Communitivity
XMPP has already been mentioned (thanks edhelas). What I didn't see mentioned
was ActivityPub
([https://www.w3.org/TR/activitypub/](https://www.w3.org/TR/activitypub/)), or
some of the advanced messaging features users have come to expect (e.g., store
and forward, forms, automated route handlers).

What I am curious about are the following questions?

1) What differentiates WebSub from XMPP?

2) What differentiates WebSub from ActivityPub?

3) How are you handling the N-squared delivery problem, if you are delivering
content directly to each subscriber with HTTP POSTs?

4) Does WebSub currently support store and forward? If not, is that on a
roadmap for a future version?

5) Same as 4, except for support for forms and form responses? Examples are a
builtin Yes or No reply, or a builtin poll vote.

6) Same as 4, for automated message routing.

7) Why not have it be transport-agnostic, instead of mandating HTTP? And why
HTTP? The growing trend is towards more decentralized.

8) How does this compare to Sir Tim Berners-Lee's SOLID
([https://solid.mit.edu/](https://solid.mit.edu/))?

------
antpls
So I read the comments but I still don't understand what problem this
specification solves. Does it enable new use cases? Does it enable new trust
models? Since it's only a server protocol, how do final users actually read
the content?

Edit : I don't understand the downvote. I use RSS daily to fetch news and I
don't have problem with it. It's simple and deliver news to the edge (my
mobile phone). I'm not saying WebSub is useless, I would like to understand
what a 3 entities model brings to the table compared to a simple server-client
delivery. What's more, the Subscriber entity cannot be a mobile device with
the current network, because mobile internet providers block incoming GET
requests. Therefore to fetch news, it has to be a pull model.

Why not add a simple rational in the header of the spec, explaining the
problem, the existing solutions, and why this new solution? Does it solve a
security issue, a scalability issue or a trust issue?

~~~
rakoo
WebSub is the successor to PubSubHubBub, which was designed to make
distribution of RSS items more efficient (ie receive items as they are
published on the host):

\- publisher would send updates, instead of having everyone poll \- publisher
would be protected from thundering herd if a content suddenly becomes popular
\- publisher and subscribers wouldn't need to exchange a full "page" of items
when only one is needed

~~~
saurik
This sounds great in principle, but this is 2018... virtually none of the use
cases for RSS can be replaced by something which requires the consumer to be
online and capable of receiving an HTTP POST (which isn't possible on the web
platform at all, so this shouldn't be called WebSub). To the extent to which a
"thundering herd" ever existed for RSS, it was from large numbers end users
with normal computers behind firewalls constantly checking the feed, which is
an environment where this protocol has no hope of ever being applicable.

So far, no one on this entire thread has descried a single use case where
WebSub actually provides value. Someone mentioned that theoretically a service
like Facebook could use this for Facebook, but then they themselves linked to
a page on Quora with an explanation from someone at Facebook for why they
tested out PubSubHubbub and then gave up on it which stated that he "think[s]
the benefits of adopting PubSubHubbub are less clear" for this very issue
(that it is inherently a server-to-server protocol, where there wasn't really
a problem in the first place).

~~~
rakoo
Are you saying that webhooks have no purpose in 2018 ? Because while WebSub
was created as an effort to help RSS get more realtime, it's a more generic
architecture for publishers who have content and need to notify interested
subscribers.

~~~
antpls
What we are saying is : In the attempt at understanding the use cases of this
proposal, people said in the thread webhooks were created to replace RSS+GET
pull, as an "enhancement".

As we are trying to explain : this proposal doesn't cover all use cases of
RSS+GET pull, so it should not be considered a replacement of RSS nor an
enhancement of it.

If WebSub has a purpose (if any), many people fail to explain a valid one.
Does it worth wasting the resources of the W3C for such proposal?

The "world wide web" is wide, and doesn't only include servers

~~~
rakoo
> people said in the thread webhooks were created to replace RSS+GET pull

That's the root of our misunderstanding then: never was PubSubHubbub
_replacing_ RSS+GET pull, it was supposed to _help_ it for people who want
notifications, on top of the existing system.

~~~
antpls
The root of the misunderstandong is that their is no rational in the proposal.
How are people supposed to understand any proposal without a rational?

------
friend-monoid
1\. Why not a SUB HTTP request? And a PUB http request. The response URL could
be a required header.

2\. We have HEAD, can we do service discovery using HEAD?

3\. Why not let a topic be a HTTP URL? “PUB /user/john/position
HTTP/1.1\r\ndata...”.

4\. Subscription expiration as a way to force subscribers to renew and upon
renew get redirected to other servers is pretty cool. NATS has a special
message (the INFO message) to do the same, but you might be in the middle of
an important request-reply session you don’t want to abort.

5\. The authors could have made this protocol very “non-http-ish” by
implementing what amounts to Redis but in HTTP. I’m glad they didn’t. This
still feels like HTTP, which is great.

~~~
voltagex_
Middleboxes and bad HTTP servers/clients would explode, I guarantee it.

~~~
friend-monoid
But we have the PATCH method, which is... Non-standard, I think? And it works
fine.

~~~
icebraining
It's been a standard for a few years:
[https://tools.ietf.org/html/rfc5789](https://tools.ietf.org/html/rfc5789)

------
krn
Superfeedr[1] (acquired by Medium in 2016[2]) was probably the biggest company
yet, that based its entire business model on WebSub / PubSubHubbub protocol.

[1]
[https://en.wikipedia.org/wiki/Superfeedr](https://en.wikipedia.org/wiki/Superfeedr)

[2] [https://techcrunch.com/2016/06/02/super-to-
medium/](https://techcrunch.com/2016/06/02/super-to-medium/)

------
ArtWomb
Just thinking beyond a replacement for RSS, this could be used for large-scale
persistent virtual worlds. WebXR version of Second Life. GLTF as a content
transmission format. As the Hub updates with a POST request, Subscribers
callback could return local state. Hub becomes global state of truth. And
there is no need to manage peer net or continuous WebSocket connections.

------
arjun27
This has come along way from "WebSub was previously known as PubSubHubbub"

~~~
MickerNews
The former is a much better name IMHO!

------
behrtam
So it's an alternative to something like WAMP? (Web Application Messaging
Protocol on top of websockets - [https://wamp-proto.org/](https://wamp-
proto.org/))

~~~
friend-monoid
Except you don’t have an open connection where you receive messages on.
Instead, you register a callback url. It’s more like webhooks, but it’s it’s
own thing, and not a setting on a page.

------
gscott
Just bring back Fidonet

------
TheChaplain
How does this compare to ActivityPub used by Mastodon et al?

~~~
erikrothoff
See it more as an alternative to RSS. Aggregators subscribe and feed
information to their users. The polling step is just switched to pushing.

~~~
detaro
Not really an alternative, more of an addition to. The most common use case
for Pubsubhubbub/WebSub is in combination with RSS/Atom feeds, informing
subscribers about updates to those.

------
smilbandit
I wish there was a pull aspect to it, like NEWNEWS with NNTP. If the
subscriber is offline it seems that there doesn't seem to be a way to retrieve
missing items except through the publisher.

------
sajal83
> The subscriber must be directly network-accessible and is identified by its
> Subscriber Callback URL

Does this mean the subscriber needs to have a forwarded port open to the
internet for this to work? Without IPv6, users behind NAT (and specifically
behind CGNAT) wouldn't be able to use it.

------
Muhammad-saleh
I think Websub is enhancement to XMPP. Both are asynchronous. Which means it
should be faster than HTTP. Actually it's a push-based protocol where server
sends single POST to the hub, the hub then sends that notification to all
subscribers.

------
xg15
So as with webhooks, this will require you to have a permanently reachable
server sitting on the internet. Couldn't they at least have defined an
alternative transport using websockets or SSE?

~~~
icebraining
WebSub is essentially just the transport; the payload is usually Atom or RSS,
but it's not specified. Had they included a websockets version, it would
essentially be something almost completely different with the same name.

------
PaulHoule
What I don't get is what the motivation is for running the hub, other than
"get bought by Medium?"

------
EGreg
Is this compatible with new distributed crypto projects like scuttlebutt or
dat, and if not, why not?

~~~
icebraining
WebSub is just the standardization of PubSubhubbub, which has existed (and
been used by publishers like Feedburner) for longer than those projects.

~~~
nstart
So essentially there's no difference right? When I was reading the spec I
couldn't tell of any notable differences between the two texts.

