Hacker News new | past | comments | ask | show | jobs | submit login

I'm probably a better person to answer this than Sam - I'm a former lead on this project - so I'll take a swing:

We chose Redis because it gave us the specific, incremental improvements over our existing memcached-based system that we required, without requiring us to write (yet another) component. There was enough to do, and this choice has turned out to be good enough, I think.

As the project progressed though, we treated Redis much like we did own it. We altered the wire protocol, changed the eviction strategy, and reduced protocol parsing overhead, for example. Much of that work has long since made it upstream.

[edit] grammar

Sounds like good engineering choices. I think of Twitter as having unlimited resources but, of course, that can't be true.

Two follow-on questions:

1. Did your changes make it back into open source or were they only relevant to Twitter? When you say upstream to you mean on Redis or earlier in the Twitter pipeline?

2. How much is Redis on the critical path? Is it 90% of the processing work in the large fanout cases?

1. Yes, most everything we changed is in the open source Redis code base. That's what I'm referring to as upstream, above.

2. Redis is in the critical path for a majority of API requests. I can't provide a specific percentage.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact