
A list of queueing libraries - strzalek
http://queues.io/
======
wslh
He didn't mention IBM WebSphere MQ or Oracle Tuxedo that have being used in
complex scenarios before the products in the list.

I would add my implementations and resources, even if they were for
educational purposes:

i) Persisting Native Python Queues: [http://blog.databigbang.com/persisting-
native-python-queues/](http://blog.databigbang.com/persisting-native-python-
queues/)

ii) Adding Acknowledgement Semantics to a Persistent Queue:
[http://blog.databigbang.com/adding-acknowledgement-
semantics...](http://blog.databigbang.com/adding-acknowledgement-semantics-to-
a-persistent-queue/)

iii) Esoteric Queue Scheduling Disciplines (an essay about a new kind of
queue): [http://blog.databigbang.com/esoteric-queue-scheduling-
discip...](http://blog.databigbang.com/esoteric-queue-scheduling-disciplines/)

iv) Using Queues in Web Crawling and Analysis Infrastructure (just
informative): [http://blog.databigbang.com/using-queues-in-web-crawling-
and...](http://blog.databigbang.com/using-queues-in-web-crawling-and-analysis-
infrastructure/)

------
buster
No Celery, no RabbitMQ, but 0mq, i don't know if i should take this
seriously..

Just putting some google search hits on a dedicated website, aren't we?

------
iamwil
I've always found having to deal with background workers a pain in the butt.
Not only do you have to set up additional servers (queue and worker servers),
but also you run into problems as a result of your own naivety--such as not
using a connection pool for workers so the startup time is too slow, or using
a library that doesn't handle timeouts. Also, I find that I have little
visibility into the queuing throughput, or if I chain jobs for workers (result
of one worker goes to another worker), I have no idea if the data made it all
the way through.

Does anyone else find it a pain, or just me?

Also, has anyone ever tried workers as a service, like iron.io? Are there
others, and was it worth it?

~~~
joevandyk
Amazon Simple Workflow might help:
[http://aws.amazon.com/swf/](http://aws.amazon.com/swf/)

Personally, I use queue_classic + postgres triggers to keep track of each of
the job history. So I have a history of all the jobs, when they started, when
they finished, the data in each job, etc.

But yes, it's a pain.

[http://torquebox.org/](http://torquebox.org/) is supposed to help with the
pain of managing background jobs (and lots of other things) for Rails
applications.

------
X4
Here are more:
[https://en.wikipedia.org/wiki/Message_queue#See_also](https://en.wikipedia.org/wiki/Message_queue#See_also)

[http://web2.uwindsor.ca/math/hlynka/qsoft.html](http://web2.uwindsor.ca/math/hlynka/qsoft.html)

Man, wish I had time to help you complete it, I have crawled the web
thoroughly harvesting all worthy message queues, but my bookmarks are so
unorganized because I used to have all important stuff in Chrome, which really
has ZERO support in organizing things. Once you bookmark something, it's gone,
except it's in your bookmark bar.

Then I switched to Firefox, which has better support for organizing things,
starting with Tab-Groups, Tags for Bookmarks etc. unfortunately it's not
helping to organize bookmarks too. Importing all chrome bookmarks leaves me
with an unmanageable task of sorting things.

~~~
strzalek
I had the same problem and that's why I've created this page. If you'll find
something useful in your bookmarks, I'm open for any contribution, just send
PR or create issue. Thanks!

------
mindcrime
What about HornetQ[1]? Qpid[2]? OpenMQ[3]? OpenJMS[4]? JORAM[5]?

[1]: [http://www.jboss.org/hornetq](http://www.jboss.org/hornetq)

[2]: [http://qpid.apache.org/](http://qpid.apache.org/)

[3]: [http://mq.java.net/](http://mq.java.net/)

[4]: [http://openjms.sourceforge.net](http://openjms.sourceforge.net)

[5]: [http://joram.ow2.org/](http://joram.ow2.org/)

------
kailuowang
I like the idea of having a page listing all libraries regarding a certain
topic. What's the advantage of a github + Rails solution has over a wiki page?

~~~
paulgb
I'm not too familiar with Rails, but it looks like this is actually a static
site generated with Ruby and hosted on s3.

------
brandynwhite
I'd put redis up there too, with their blocking pop and pub/sub it's really
usable as a queue.

------
sylvinus
You should add RQ in there! [http://www.slideshare.net/sylvinus/why-and-how-
pricing-assis...](http://www.slideshare.net/sylvinus/why-and-how-pricing-
assistant-migrated-from-celery-to-rq-parispy-2)

------
wanderr
More useful than a comprehensive list of all queuing libraries would be a
comparison of the best ones with tl;dr summaries of how each works, pros &
cons of each approach, and some consistent benchmarks.

------
robotmay
I've always liked beanstalkd, unfortunately it still has no authentication
support so you're unlikely to see it catch on in the PaaS environments any
time soon (unless someone hooks SASL into it).

------
flexd
For this to be useful it really needs to be groupable by programming language.

And why it needs it's own domain/website instead of just a wiki page somewhere
I do not understand.

------
LeafyGreenbriar
Looks like this is getting updated, so... No ActiveMQ?

~~~
LeafyGreenbriar
Also, MSMQ while I'm at it.

~~~
grogs
IBM WebSphere MQ

------
dmgbrn
I think OP's motivation may involve
[http://xkcd.com/853](http://xkcd.com/853).

------
nmb
no [http://www.celeryproject.org/](http://www.celeryproject.org/)?

~~~
strzalek
It was never on my radar. I'll search around for some resources about it.
Thanks.

Site is open sourced, feel free to send PR with missing things -
[https://github.com/strzalek/queues.io](https://github.com/strzalek/queues.io)

------
junto
I hate to be the one that says this, but MSMQ is missing.

------
pedromorgan
Doesnt specify which language lib is coded in.

