
Should You Build a Webhooks API? - stanleydrew
https://brandur.org/webhooks
======
marten-de-vries
If you're using a web hook to notify your users of changed _content_ , the
WebSub standard
([https://www.w3.org/TR/websub/;](https://www.w3.org/TR/websub/;) formerly
PubSubHubbub; currently a candidate recommendation) is a good option. It
solves a couple (though not all) of the problems of web hooks listed in the
article, including:

\- Uncertain security: signing notifications is in the spec

\- Development and testing: ideally, with libraries that implement the spec,
you can assume that the websub part will 'just work' and only need to test
your handler function.

\- Misbehavior is onerous: subscribers need to actively confirm their
subscription every once in a while. This might seem complex, but it allows for
periodic checks if the server/client is still functioning as expected, and
ideally can be handled automatically by libraries.

\- Retries: the spec leaves some room here, but again, a library can handle
this.

As I decided to use WebSub as an alternative for a 'plain' webhook for a
project recently, I wrote a library for the Flask web framework:
[https://github.com/marten-de-vries/Flask-Websub](https://github.com/marten-
de-vries/Flask-Websub). I personally hope libraries for other frameworks will
follow now the spec is moving less.

~~~
devmunchies
how can someone stay knowledgable of all of the many reports and standards on
w3.org? Do people visit there regularly?

I tried looking for websub on w3.org without using a search engine and it was
a challenge (finally found it)

~~~
marten-de-vries
I found out about this standard myself indirectly during a web search. As an
outsider to the w3.org process, I think the best way is to follow working
groups with themes you're interested in:
[https://www.w3.org/Consortium/activities](https://www.w3.org/Consortium/activities).
For example, WebSub is worked on by the Social Web Working Group:
[https://www.w3.org/TR/tr-groups-
all#tr_Social_Web_Working_Gr...](https://www.w3.org/TR/tr-groups-
all#tr_Social_Web_Working_Group).

------
filearts
We built Auth0 Extend [1] to address with many of these concerns. Our
hypothesis is that the logic that implements these extensibility points should
live in the host platform.

For several years now, we've had this sort of hook-via-code in the Rules
engine on our Auth0 identity product. It's been such a powerful enabler for
many different integration scenarios that we thought that other companies may
find the technology useful for their platforms.

Since committing to building Auth0 Extend, we've seen Twilio launch their
functions [2] and just today CloudFlare launched its Workers [3]. It seems
like the this is an approach that is in its infancy but that is here to stay!

1: [https://auth0.com/extend/developers](https://auth0.com/extend/developers)

2:
[https://www.twilio.com/docs/api/runtime/functions](https://www.twilio.com/docs/api/runtime/functions)

3: [https://blog.cloudflare.com/introducing-cloudflare-
workers/](https://blog.cloudflare.com/introducing-cloudflare-workers/)

~~~
tonylucas
We are just in the process of launching something similar at Converse to allow
users to extend our conversational workflow capabilities to add their own
modules, events, including extending API endpoints and the core UI.

Definitely a good way of doing things, still looking for more beta testers but
feedback is very positive so far.

[https://get.converse.ai/v2.0/docs/plugin-
development](https://get.converse.ai/v2.0/docs/plugin-development) if anyone
wants to kick the tires on it, all feedback welcome.

------
stevekemp
I've setup a few webhooks over the past few years, and one of the
complications that people rarely talk about is the data that is passed to
various webhooks.

As the article alludes to APIs change over time, and so you can find the data
that is submitted to you changes. In my case I setup an end-point that
responded to push events from various code-hosting services.

I found that the three "main" git-hosts sent widely different data in their
webhooks, and updates we often painful. (Github, bitbucket, and .. something
else. I forget now.) Trying to have one end-point to handle all services was a
recipe for frustration and failures. Even parsing out the basic details was
hard:

* This is the SSH repository URL of the thing that was committed to

In the end I moved to self-hosting git-repositories, rather than allowing
users to point at my webhook. That allowed hooks in the master-repos to fire.

(In my case I trigger DNS updates when git commits are made.)

~~~
marten-de-vries
As using webhooks for git repositories is probably the most common usage, it
would be really nice if the different APIs converged here. Now everyone is
inventing their own (and I'm in fact guilty to this myself for a side-
project).

------
ezekg
Good tips! I actually _just_ worked on detecting dead webhook endpoints, as
I've had a number of users use ngrok for local testing, create hundreds of
events, and then disable ngrok--and much like Stripe, I retry with an
exponential backoff over the course of a few days. Needless to say, my logs
were filled with failed webhook request errors and I had to figure out how to
limit that from happening in the future. Very interesting list of alternatives
to webhooks as well.

~~~
jkarneges
When it comes to axing dead endpoints, I see two ways to handle this based on
the way webhooks are configured.

* If webhooks are configured by UI, then you have to contact the user and tell them their webhook is gone, e.g. via email.

* If webhooks are configured programmatically, then the endpoints ought to be able to be swept up silently, and the receiver should know to periodically refresh the configuration for as long as it cares about it.

Ideally all webhook implementations would work the second way, but
unfortunately it is not standard practice.

------
falsedan
I guess the section on _Retries_ is closest, but we're looking at moving from
a system with straight in-process hooks to one using webhooks, and a
complication is we now have to provision a webhook endpoint with very high
reliability to accept the callback and (effectively) call the old hook.

The thing is, all the hook does is put a message on a queue…

------
adnanh
For those interested in technically easy webhook consuming endpoint server
check out
[https://github.com/adnanh/webhook](https://github.com/adnanh/webhook)

------
fovc
I would love to see some of these APIs replaced by WebSockets. It would allow
building your own client for some services much more easily while maintaining
push notifications

------
3pt14159
Webhooks should not be mentioned without also mentioning SSRF vulns. Because
most companies use their firewall as a defense permitter, these attacks can be
devastating.

~~~
ec109685
It is mentioned in the article.

------
andrewstuart2
Side note: their music is so good. Paired with their actual tech and vision, I
think the music actually makes me want to work at SpaceX more than if paired
with something less epic.

~~~
andrewstuart2
I swear I wrote this comment on the BFR story [1].

[1]
[https://news.ycombinator.com/item?id=15362999](https://news.ycombinator.com/item?id=15362999)

