
Twitter's  peak load is only 200 tweets per second - boundlessdreamz
http://friendfeed.com/scobleizer/50e673d8/some-stats-from-twitter-conference-compared-to
======
adbachman
BUT, that's via the firehose, which doesn't include direct messages or
twitterers who've locked their feeds. The number also doesn't reflect what has
to happen with each of those 200.

Sure it's not impressive if all they have to do is append a 140 char message
to a flat file or DB table. It's all the other manipulation that's
interesting.

~~~
Kaizyn
What kind of 'manipulation'? They're basically routing messages, with a
protocol change in the middle when the recipients' designated protocol differs
(i.e. IM to text message). This can't be all that computationally expensive
when the system is engineered right.

Knowing how little activity twitter has really makes them look incompetent in
light of the service outages they experienced over the last year. Either Ruby
on Rails is really really non-performant or the twitter code-monkies got the
system architecture wrong.

~~~
adbachman
For starters, there's a few interesting links at
[http://highscalability.com/scaling-twitter-making-
twitter-10...](http://highscalability.com/scaling-twitter-making-
twitter-10000-percent-faster)

summary: "Rails and Ruby haven’t been stumbling blocks... The performance
boosts associated with a “faster” language would give us a 10-20% improvement,
but thanks to architectural changes that Ruby and Rails happily accommodated,
Twitter is 10000% faster than it was in January"

They're dealing with duplication, consistency, searchability, etc. across
distributed storage systems and a variety of service mechanisms (more than
just protocols). The 1 000 000 follower user has every message duplicated and
cached at multiple layers up to 1000000 times. The message can show up on the
web, API, or across any of the protocols, and it has to persist.

They're not incompetent code-monkies, they just guessed way low when they
designed the architecture. It looks like the big move was from a CMS model
(not at all unreasonable for a "microblogging" service) to a messaging model.
In hindsight, targeting messaging to begin with would've saved them some down
time, but wouldn't have been practical in the short term.

From [http://gojko.net/2009/03/16/qcon-london-2009-upgrading-
twitt...](http://gojko.net/2009/03/16/qcon-london-2009-upgrading-twitter-
without-service-disruptions/) it sounds like the insertion number averages
around 9600 messages per second. That's avg follower count * avg tweets
inbound, and that's only the input.

Edit: real, solid numbers are hard to come by. I suspect it's proprietary.

~~~
pj
What you say is echoed by, _Every incorrect assumption in this post seems to
think that 1 tweet on twitter = 1 database row = "So easy!". You've left out
the user fanout! One Obama Tweet = 1M database rows, someplace_

Assuming that is true, which I have trouble believing it is, it sounds like
they need some help with normalization. I understand the tradeoffs, but it
just seems crazy.

Select tweet.* from tweets inner join tweeters on tweeters.id =
tweets.tweeterid inner join followers on followers.followerid=
_loggedinuserid_ and followers.followee=tweet.tweeterid

Yeah, okay, I know there are performance problems with joins, but there are
performance problems with 1,000,000 inserts as well and you could cache the
list of followees and do an "in" statement such as: select * from tweets where
tweeterid in ( _cached_comma_separated_list_of_followee_ids_ )

or whatever to improve select performance.

------
abstractwater
I don't see the value in Scoble's argument, if he has one. I don't see the
point of comparing the amount of tweets/sec with the rate of Google queries.
They are just very different things, in their essence, purpose, way of usage.

It would be more interesting (maybe) comparing the number of Twitter search
queries with Google's, at least we would be talking about similar operations.

~~~
jules
Yes. I think Google has to do quite a bit more per query than Twitter per
message.

~~~
pj
and google also has perhaps hundreds of thousands of servers and tens of data
centers.

His point is just to provide some measure of scale against a "known beast".

------
icey
I'm confused... Where is this figure actually coming from? (Other than Scoble
saying it.)

~~~
Maciek416
Another way to extract this figure is to monitor the ids of tweets like the
Twitpocalypse site is doing.

We've been monitoring Twitpocalypse for a while, and the data yields the
following chart:

[http://img.skitch.com/20090529-xhhnr83xsy67burr7w24kycp4u.pn...](http://img.skitch.com/20090529-xhhnr83xsy67burr7w24kycp4u.png)

From the slope, we saw about 200 tweets/second, sometimes peaking at 250 as
well. The right axis in this chart is the number of tweets per minute. You can
see it rising and falling through the day.

~~~
icey
Hold on a minute. Are we talking about peak load or average load?

Saying Twitter averages 200 tweets/second is vastly different than peaking at
200 tweets a second.

Am I just being slow?

Edit: Nevermind, Robert is saying that the average is 200 tweets/second during
periods of peak load. The headline here makes it seem like the maximum
capacity is 200 tweets/second which is misleading.

~~~
Maciek416
I think what the data shows is that the average sort of wobbles around a bit.
Check out the update to my post for a new image link to see it varying
throughout the day.

------
est
80KBps, 0.5 billion tweets per month.

~~~
jonknee
If they ran at peak for 24 hours a day during an entire month. Which isn't
what peak means.

------
TweedHeads
MySql can handle that with both hands tied in the back.

No sweat.

~~~
poutine
Uh, some of those 200 tweets per second are being sent to over a million
people. Most are sent to hundreds or thousands.

~~~
pietro
I don't think that actor guy posts every second, and the average number of
followers is way lower than thousands.

~~~
adbachman
There are 13 users with 1000000+ and the top 100 all have over 450000.
(<http://twitterholic.com/>)

But, long tail and all that, I hear average follower count is in the 120
range, you're right there. ([http://gojko.net/2009/03/16/qcon-
london-2009-upgrading-twitt...](http://gojko.net/2009/03/16/qcon-
london-2009-upgrading-twitter-without-service-disruptions/))

