I can never understand why enforcing (sane and secure) TLS and adding a (shared) secret request header isn't the usual simple answer.
As far as I know, TLS prevents all these security issues, except authentication of the webhook caller, hence the shared secret which your webhooks should check.
I've verified in the TLS specs that every issue I've seen raised by people is actually covered, yet people still think (and write) that TLS doesn't sign traffic (it does), or TLS doesn't prevent replay attacks (it does), etc.
Usually I give them pointers to the specs, and they say "it doesn't hurt to add even more security" but they're really rolling out their own encryption, which you shouldn't do.
I'd say enforce TLS and authenticate the webhook caller...
I'm the developer of Diahook (we do webhooks as a service), and I agree. It's just so annoying that we have to implement signing ourselves instead of just using client TLS certificates. Though the problem is that there are so many places in the stack that interact with your request before it hits user code (load balancer, web server, etc), that getting anything like this to work is wishful thinking...
Hey HN - I'm the author of this post. (Founder of WorkOS)
Lots of people have written about how to scale event-based systems or how to design distributed infrastructure for queue management. But what's often overlooked is the subtle-yet-important product decisions that go into designing webhooks for a great developer experience.
We don't actually offer webhooks-as-a-service at WorkOS, but we provide webhooks to developers and I've built this feature twice now, previously at Nylas and now at WorkOS. Hopefully this article helps the next person who builds them. :)
Maybe a stupid question but... why not using an MQTT server instead of webhooks? Clients could be notified in RT, TLS is already backed in the protocol with client auth... the client doesn't need to expose an endpoint, only to connect to a server...
AWS EventBridge announced API destinations recently [1]. I can't find anywhere in their docs about SSRF, but given it's a managed service running outside of your network it sounds like a good option. Anyone care to comment if this would be a good solution for webhooks?
Hehe, just did a show HN[1] for Diahook (webhooks as a service) earlier today. I wonder if that's what triggered this posting. :P
Anyhow, I wish more people had better webhooks, it's such an important part of any API, so please read this blog post, understand the challenges, and provide a great webhooks experience!
This would be a good addition: make sure that your HTTP library POSTing the webhook won't follow redirects and is blocked from accessing internal domains.
Not just logging the outbound webhooks to STDOUT and hope they get ingested into logstash or something, but actually log them in your SoR/RDBMS so that you can present that data to a debugging developer.
What people totally omit when thinking about webhooks is the parallels to web sockets and client side updates. In the last round of our system updates, we have "attached" the web sockets and webhooks to the same message bus which has simplified the overall design of the system.
Yeah, they share a lot of the same challenges as they are both unlike the rest of the web stack (which is just request -> response). There are a lot of similarities, but a lot of different challenges too.
At FastComments we send an your API token in the webhook request, and if your API doesn't return a 401 with an invalid token the integration won't activate.
I can see why you might do that, but it seems mildly hostile to me and would have broken things for me on at least one or two occasions. I know I’ve set up very simple hook arrangements before where I don’t care about authentication, at least in part because the URL is secret and it’s probably just about dumping the hook call contents for now anyway.
But at the same time, perhaps doing this would solve more problems than it creates.
This was an interesting read! If you ever decide to update this article & include tools for working with webhooks, Hookdeck(https://hookdeck.io) should be at the top of the list. Hookdeck is a complete webhook infrastructure that makes working with webhooks painless. we pride ourselves in reliability. We provide tooling for easy monitoring and troubleshooting of webhooks
As far as I know, TLS prevents all these security issues, except authentication of the webhook caller, hence the shared secret which your webhooks should check.
I've verified in the TLS specs that every issue I've seen raised by people is actually covered, yet people still think (and write) that TLS doesn't sign traffic (it does), or TLS doesn't prevent replay attacks (it does), etc.
Usually I give them pointers to the specs, and they say "it doesn't hurt to add even more security" but they're really rolling out their own encryption, which you shouldn't do.
I'd say enforce TLS and authenticate the webhook caller...