

Message Queue Evaluation Notes - wave
http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes

======
zacharyvoase
N.B.: this page wasn't updated heavily since April 2009; a slight revision was
made in January 2010.

~~~
northisup
+1 to this, a lot has changed since this was written.

------
lazy_b
I'm curious what the motivation for the potential "hundreds of millions" of
messaging queues is here. having a queue per-registered user seems, I don't
know, maybe overkill? Per group chat makes a lot of sense, but I'm having
trouble reconciling the numbers based on what was posted. What am I
overlooking?

~~~
rdw
Second Life involves a lot of per-user message sending. IM, group chats,
friend requests, marriage proposals, dialogs with in-game merchant systems,
payment notifications, persistent announcements, and a lot that I'm
forgetting. They're all roughly the same thing -- messages that need to be
delivered to either a specific user or a group of users, with flags for
whether they get discarded when the user is offline or whether the user has to
explicitly acknowledge them.

It was probably overreaching for us to try to find a system that would handle
all this for every user. But even hundreds of thousands of current users, the
bare minimum we could expect to be concurrent, were not well supported by the
existing solutions.

------
pavlov
Can someone more familiar with these products explain why they decided against
ZeroMQ even though it apparently offers by far the best performance?

The ZeroMQ "note" part sounds like there's a bug in the Python library, but
that hardly seems to be grounds for eliminating it completely from the "race".

~~~
rdw
I'm the primary author (formerly Which Linden). Surprised to see this pop up
here after all this time!

We wrote this almost three years ago, and at the time ZeroMQ was substantially
less mature than it is now. In particular, the Python bindings were broken,
weren't maintained by the ZeroMQ organization, and there was no indication
they'd be production-ready by the time we needed it (which would have been
within the next month or two).

More importantly, we were looking for something that handled our use case a
little bit more fully. Our message-rate requirements were laughably small, so
what we really wanted was software that gave us an "inbox" for each of
millions of users. We also weren't looking forward to implementing failover,
consistency, and sharding from scratch.

Edit: corrected timeline.

~~~
DEinspanjer
Do you know what they ended up settling on?

~~~
rdw
No out-of-the-box solution was used. The system remained a menagerie of
different messaging techniques (database, udp, in-memory storage), and the
company just spent time improving the reliability and performance of these
existing systems individually.

