That looks quite complicated and the only benefit I can see is not having to wait 5 minutes for your RSS client to poll the source. Could anybody explain what are the true benefits over our current RSS/Atom polling systems?
EDIT: I also fail to see how this would reduce the load. It just transfers the load to the "hub" node. If a big website runs its own hub node to serve its content, then there's no gain (you just had a layer of indirection and complexity).
There could be big external hub, but what would be the point for them? Man-in-the-middle-ing the streams and inserting ads? Doesn't sound really great.
And if a bazillion micro hubs emerge, then the overall load on the providers will pretty much remain the same.
The switch from a polling architecture to a notification one greatly reduces the amount of over the wire traffic that has to take place. Consider the case of a blog where it's not updated for a month: every feed reader continues to continuously poll for the feed every hour of every day (though some are better about setting a timing decay, just checking for header last changed, etc.).
With PubSubHub the traffic sent is 0.
Some other things to consider:
1. The obvious case for this is blogs, but PubSubHub can be the foundation of other more interesting and new projects: consider a distributed Twitter like micro blogging service. I'm not saying building one is a good idea, just that it would be better built on PSH than polling.
2. People are lazy and feeds put in feedreaders are polled forever. I've worked on multiple systems where years later feed readers are still checking every hour for feeds that are missing, or just completely irrelevant.
I suppose webapps such as google reader query only once the source for all the users subscribed to a certain feed, effectively reducing the number of HTTP requests on the upstream provider. Isn't it a simpler and well tested solution to this problem? It basically makes the google server the Hub and your browser the subscriber to use the website's terminology.
You might say that Pubsubhubhub is the logical extension of the Google Reader concept. It allows you to "read" those feeds programmatically and enables the content site to notify Google Reader of updates so they get there quicker.
Also, anyone can choose to be a hub, not just Google Reader. There's your simple and well tested solution wrapped in a standard for anyone to implement.
It's converting a two-tier one-to-many pull system with 5 minute polling to a three-tier one-to-many-to-many push (err.. PuSH) system. More efficient, allows for quicker dissemination of update notifications.
Gowalla implements PuSH which lets you react to check-ins. Much better than constantly polling their API over and over again seeing if something has changed. I wish Foursquare would add this. It would open up a huge number of new use cases.
Friendfeed provided a great example of the benefits here. The shorter your feedback loops between publishing and commenting, the more conversation you see around published content.
hubs cannot insert data. the only function a hub serves is to assert that there is new data on the feed, and that clients should re-request the feed.
a malicious hub could continually signal clients to keep fetching the feed, triggering a Denail-of-Service on the feed. that's really the only possible mis-use.
hubs can also proxy. ycombinator could run their own hub that could subscribe to someone else's hub and notify by proxy all of it's clients. this negates a lot of the "big hub" mumbo jumbo you seem to fear.
to answer your original confusion, the problem PubSubHubbub solves are two fold. foremost, 5 minutes is a long time in some worlds. second, every client subscribing has to pole every 5 minutes, asserting an etag and/or if-modified-since, which the server then has to look at and validate. with pubsubhubbub, results are "instant" and the server doesnt have any load when nothing new is being posted. this is indeed a big deal.
to your last, a micro hub will have no additional load, so long as the micro hub has any subscribers.
EDIT: I also fail to see how this would reduce the load. It just transfers the load to the "hub" node. If a big website runs its own hub node to serve its content, then there's no gain (you just had a layer of indirection and complexity).
There could be big external hub, but what would be the point for them? Man-in-the-middle-ing the streams and inserting ads? Doesn't sound really great.
And if a bazillion micro hubs emerge, then the overall load on the providers will pretty much remain the same.