Hacker News new | past | comments | ask | show | jobs | submit login
How we scaled the GitHub API with a sharded, replicated rate limiter in Redis (github.blog)
4 points by pimterry on April 6, 2021 | hide | past | favorite | 3 comments



The issue with this suggested implementation that it's not atomic and might cause some race conditions.

  local expires_at = tonumber(redis.call("get", expires_at_key))
  if not expires_at or expires_at < current_time then
    -- this is either a brand new window,
    -- or this window has closed, but redis hasn't cleaned up the key yet
    -- (redis will clean it up in one more second)
    -- initialize a new rate limit window
    redis.call("set", rate_limit_key, 0)
    redis.call("set", expires_at_key, next_expires_at)

Between the time the `get` is called and the `set` is called there might parallel call that might create the key and the `set` will override it.


Lua script execution is atomic in Redis.

https://redis.io/commands/eval#atomicity-of-scripts

> Redis guarantees that a script is executed in an atomic way: no other script or Redis command will be executed while a script is being executed.


You're right I completely missed that this is a Lua a script part, I was sure it's the Ruby code... (shame on me, shows how much I can't recognize Ruby code...)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: