At a guess, a few reasons. Firstly, they had everything else written against a message queue with particular behaviour - so better to upgrade that queue than switch to a completely new one and have to rewrite everything that interacts with it.
Secondly, after running a custom message queue for well over a year they know EXACTLY what they need from one, so writing their own still makes sense.
Thirdly, if they're going to start moving other core bits of Twitter infrastructure to Scala it makes sense to try it out with an important piece of the puzzle that they thoroughly understand first.
And finally, Twitter's core competency is delivering messages. As such, it's really not so extreme to use their own software to do that - they work at a high enough scale that they need to be experts in whatever solution they are using.
As Douglas Crockford once said, "The good thing about reinventing the wheel is that you can get a round one.". The fact that Twitter's reliability over the past 6-12 months has been a huge improvement, despite the enormous growth the service has seen (it's mentioned in the mainstream media all the time) would suggest that their decision to roll their own message queue paid off.
It's also worth noting that the one queue Arcieri proposed might have been interface-compatibile --- MemcacheQ --- is a single-developer side project written in C. Arcieri is "completely confused" by the fact that that Twitter didn't adopt this as the core of their service.
This is a straw-man argument. MemcacheQ is a straightforward mash-up of two very stable software stacks.. memcached and BerkeleyDB. The lines of code to accomplish it are trivial. So what.
Secondly, after running a custom message queue for well over a year they know EXACTLY what they need from one, so writing their own still makes sense.
Thirdly, if they're going to start moving other core bits of Twitter infrastructure to Scala it makes sense to try it out with an important piece of the puzzle that they thoroughly understand first.
And finally, Twitter's core competency is delivering messages. As such, it's really not so extreme to use their own software to do that - they work at a high enough scale that they need to be experts in whatever solution they are using.
As Douglas Crockford once said, "The good thing about reinventing the wheel is that you can get a round one.". The fact that Twitter's reliability over the past 6-12 months has been a huge improvement, despite the enormous growth the service has seen (it's mentioned in the mainstream media all the time) would suggest that their decision to roll their own message queue paid off.