> IMO, if you're an API company, and you can't handle bursts of traffic from your customers, you should work on improving your backend and stop wasting time messing around with implementing patterns like this. It's a lose-lose situation for you and your customers.
I'm surprised this isn't more obvious, especially to a company like Stripe that is built on one of the slowest programming languages/platforms in existence. It isn't hard to come up with a list of languages that would easily provide an order of magnitude more capability per server (yes, maybe compute is not their bottleneck, but I've never seen an RoR setup that was bottlenecked by anything other than RoR). And that not only makes it cheaper for the baseline, but it also makes it cheaper to overprovision for safely handling spikes, and both cheaper and faster to scale when needed.
And like you said, not responding to demand is just preventing your customers from giving you money...but it is worse. If you are a growing company and one of your badly needed services is your bottleneck, that service is gonna be the first one to go. API limiting should be considered an attrition risk.
Why not go out and build a better stripe with a better language and show them how it is really done? Clearly you know more about building web scale payment apis than they do.
I'm surprised this isn't more obvious, especially to a company like Stripe that is built on one of the slowest programming languages/platforms in existence. It isn't hard to come up with a list of languages that would easily provide an order of magnitude more capability per server (yes, maybe compute is not their bottleneck, but I've never seen an RoR setup that was bottlenecked by anything other than RoR). And that not only makes it cheaper for the baseline, but it also makes it cheaper to overprovision for safely handling spikes, and both cheaper and faster to scale when needed.
And like you said, not responding to demand is just preventing your customers from giving you money...but it is worse. If you are a growing company and one of your badly needed services is your bottleneck, that service is gonna be the first one to go. API limiting should be considered an attrition risk.