

Show HN: SHARQ – Open Source, Rate-Limited Queuing System - ohmygeek
http://sharq.io/

======
ohmygeek
I'm the primary author SHARQ. Appreciate any feedback..

Here is the link to the slides from my PyCon India talk:
[https://speakerdeck.com/sandeepraju/sharq-an-open-source-
rat...](https://speakerdeck.com/sandeepraju/sharq-an-open-source-rate-limited-
queueing-system-pycon-india-2014)

Here is our blog announcement: [https://www.plivo.com/blog/sharq-a-flexible-
open-source-rate...](https://www.plivo.com/blog/sharq-a-flexible-open-source-
rate-limited-queuing-system/)

~~~
wc-
Looks pretty cool, looking forward to playing around with it.

How does this compare to celery w/ rate limiting of individual tasks? Celery
can be pretty daunting (heavyweight perhaps?) sometimes, having a simpler
alternative could be useful. However, I wonder if by the time you need rate
limiting of queues, you will probably need additional "heavy" features as
well.

I'd love to see a deeper comparison of sharq's usage of redis vs celery's. For
example, are there any performance gains by using sharq's lua commands or
schemas vs celery's?

There are a few typo's in the features section btw (minmal, seriazlied).

~~~
ohmygeek
We had evaluated Celery for our use case. There were a few things it couldn't
do such as create queues dynamically with per queue rate limit.

We chose Lua because the queue operations involve modifying more than one
Redis data structure and this had to be atomic.

We'll be adding more details on why we built this and how it compares with
Celery.. :)

~~~
SEJeff
Can you please not do one thing Ask (celery author) constantly does? He breaks
the celery API to a ridiculous degree between point releases. I've used celery
for 5+ years and constantly have to hard fix the version I use in a
requirements.txt or whatnot because the most seemingly minor point release
results in broken tasks and a different api.

Use semver or something sane to not make your users angry. Pretty please!

------
psobot
This is really cool. Can queues be bounded, as well as rate-limited?

(I built my own Redis-backed distributed queue system long ago because I
needed bounded queues with backpressure, which no other libraries seemed to
offer: [http://github.com/psobot/pressure](http://github.com/psobot/pressure))

------
chadrs
Seems like the exact same approach and even using the same redis
datastrucutres as this talk from RedisConf:

[http://vimeo.com/52569901](http://vimeo.com/52569901)

(Granted, the solution presented here is not open sourced.)

------
sargun
I'm curious as to the author's choice of Redis as opposed to RabbitMQ, or
another queue. Redis is problematic when it fails, which makes me question the
use of a centralized queue server altogether.

------
j_s
Missing Github link: [https://github.com/plivo/sharq-
server](https://github.com/plivo/sharq-server)

------
mperham
Looks pretty cool, congrats on the ship!

