
Scaling HipChat Using ElasticSearch And Redis - pscsbs
http://highscalability.com/blog/2014/1/6/how-hipchat-stores-and-indexes-billions-of-messages-using-el.html
======
taspeotis

        Stats
         * 60 messages per second
        ...
          * 26 front end proxy serves. Double that in backend app servers.
    

78 servers for 60 messages per second?

We have one MSSQL box at work here that is regularly doing 2,500 queries/sec
and that's an OLTP workload so easily more than 60 writes/sec.

 _serves_ makes me think that someone didn't proof-read the article...

~~~
georgebarnett
How did it even make sense in your head to compare an OLTP workload to a chat
service?

The proxy servers are there to terminate the large number of persistent
connections. Of course it's possible to do it using less servers, but given
the Hipchat guys are smart (disclosure - I'm an Atlassian and know the
internals) I would give them the benefit of the doubt rather than engaging in
armchair architecture.

~~~
taspeotis
The point was to ask whether the _60_ in _60 messages per second_ is wrong by
comparison.

~~~
mtrimpe
The frontend needs to keep open thousands of persistent connections; 99.9% of
which will time out with receiving a single message.

Practically speaking the number of frontend servers is related to the number
of active listening clients and not messages.

Only 60 messages per second _does_ seem very small from a business perspective
however.

~~~
taspeotis
If you take the frontend servers out of the equation it's still 52 servers for
60 messages per second.

~~~
msmithstubbs
My guess: the 60 messages per second is an average. Monday morning in the US
timezones might see several multiples of that. Weekends could be a lot lower.

~~~
powdahound
It's definitely very bursty. On weekends and holidays things are much more
quiet. During peak load we'll be in the hundreds/sec. Also keep in mind that
chat messages don't actually make up the majority of the traffic we serve;
it's presence information (away, idle, available), people
connecting/disconnecting, etc. (I'm one of the HipChat co-founders.)

------
byroot
> .5 terabytes of search data.

500 gibabytes isn't cool enough ?

