Well why not? They know much more than I of the problem because I don't have a website in the top whatever that generates that much traffic.
I would guess that neither do you so it's worth listening to what he has to say. Each story is different and you'd be silly to ignore someone's experience on the matter if it is relevant to you.
For example who wouldn't like to hear the stories (possibly horror stories) of the early Twitter days? Even though there was lots of downtime precisely because of that reason there is a lot to learn.
You see this with any service as it scales up, that's where the learning occurs.
As the number of requests climbs you reach limits, those limitations manifest to the end-user as suckage, slowness, errors, and in the worst cases lost data. You can't plan those limits out ahead of time, you discover them as you go.
>I don't know why people keep using reddit as a reference for how to do scaling.
I dont know if people really use them as a reference, but Reddit is one of the few sites in the sweet spot of getting a lot of traffic, and being pretty open about their architecture. Any presentation done on behalf of Reddit (pycon '09, '10, this) is great fodder for the technical Reddit users, such as myself.
Indeed - especially since I've retired from reddit. Though, I do still feel obligated to respond to complaints/criticism; in this case, though, I can't comment on technical decisions - except to say that reddit's recent scaling problems are likely more to do with their dev count (4 - yes, there are four developers working on reddit. That's 1 dev for every ~2Million uniques a month) than technical decisions.