
Rescuing Resque: Let's do this - craigkerstiens
http://blog.steveklabnik.com/posts/2012-09-22-resque--let-s-do-this
======
pimeys
Resque isn't so great after all. We used it with around 600-1000 jobs per
minute and not without problems. The failure backends could be better. We need
the retry backend and for that you have to install another gem. The retry
feature was kind of not working so well and really hard to debug (thanks to
EventMachine and Resque's a bit too complicated failure backend).

Our solution was to build yet another queueing and worker system. For queues
Beanstalkd is way faster and way more reliable than Redis. Instead of
EventMachine I built the workers with Ruby threads and making a retry system
on top of it wasn't really a problem.

Our error percentage dropped from around 10% to 0.0001% on average. Maybe some
day I might release this as a gem, if world really needs yet another queueing
system.

~~~
antirez
> "way more reliable than Redis."

I respect your opinion on different messaging solutions (Resque VS Beanstalkd)
however since we spend a lot of efforts to make sure Redis is very reliable I
wonder what's the reliability problem you experienced with Redis. Thanks!

~~~
nirvdrum
A different issue, but one that's burned me several times is when things go
wrong with resque (or sidekiq), life goes to hell pretty quickly because redis
needs to fit everything in memory. E.g., if a third party resource goes down
and jobs hitting it all of a sudden start exceptioning and get thrown into a
retry queue, all those stacktraces sitting around in memory accumulate very
quickly. redis fills up, the OS kills it, it starts back up, loading an older
DB, and then restarts processing all those jobs that just failed, without an
exponential backoff. And this process will basically repeat ad nauseum.

I'm not sure a whole lot can be done, other than resurrecting the old debate
over something like diskstore. And while EC2 is memory-constrained, dealing
with this situation isn't a matter of just adding more memory, because that
will be filled up as well. redis would either need to be able to spill over
onto disk or the processing model would need to be changed. We ended up doing
the latter, eschewing stacktraces for retried jobs, adding monitoring to
disable queues that have a very high failure rate, etc.

~~~
rahoulb
I'll second this. It's the Resque backtraces that have caused us a few
problems. And on a VPS RAM is your most expensive resource, possibly making
Resque/Redis not such a good fit.

------
robotmay
I switched from Resque to Mike Perham's Sidekiq earlier in the year, and the
improvement is just sensational. I'm curious to see how Resque can be improved
to bring it up to the same level.

~~~
lucaspiller
++ on Sidekiq. We've been using it since April and have had no issues at all.
From the readme:

"Sidekiq is compatible with Resque. It uses the exact same message format as
Resque so it can integrate into an existing Resque processing farm. You can
have Sidekiq and Resque run side-by-side at the same time and use the Resque
client to enqueue messages in Redis to be processed by Sidekiq.

At the same time, Sidekiq uses multithreading so it is much more memory
efficient than Resque (which forks a new process for every job). You'll find
that you might need 50 200MB resque processes to peg your CPU whereas one
300MB Sidekiq process will peg the same CPU and perform the same amount of
work. Please see my blog post on Resque's memory efficiency and how I was able
to shrink a Carbon Five client's resque processing farm from 9 machines to 1
machine."

------
mrzor
Hello everyone !

We recently released jobco, a simple Resque distribution. You can check it out
at <https://github.com/mrzor/jobco>

We attempted to address some of the issues we had with our resque 1.x stack
regarding productivity (ease to use both in development, and ease to deploy)
and regarding API consistency (rewrote resque-status to keep the Resque API
intact).

It doesn't address any of the high throughput issues that were mentioned
earlier.

I'm definitely up to help fixing stuff within Resque when appropriate.
However, the Jobco project is possibly more appropriate if the feature you're
looking after can be provided using plugins, or involves providing better CLI
interface to Resque.

You can find out my email address using `git log` on the jobco repo.

------
100k
I didn't realize GitHub had abandoned Resque. What are they using now? Or is
it in "works for me" status and they don't feel like changing it?

------
khangtoh
Resque should just be left alone and let it retire. SideKiq is the present and
future.

~~~
mrzor
Sometimes the forking model have desirable properties.

Numerous past articles covered this in one way or another.

Choosing between SideKiq and Resque is primarily a matter of choosing between
forked processes and threads. Retiring Resque and let SideKiq be the present
and the future would destroy that choice.

SideKiq, as a threaded impl., can claim some cool benefits : \- shorter job
launch times (only interesting if you actually run thousands of second
spanning jobs a minute) \- reduced memory footprint (you probably have little
to no amount of thread local storage)

I will venture that the reduced memory footprint is mostly the result of the
unfriendly to copy-on-write GC we have to live with using 1.9.

Resque has a clean and simple API and a decent ecosystem. It can and hopefully
will evolve into something cooler and more powerful - even if worthy
alternatives exist.

------
cypherpunks01
This post convinced me to re-think using Resque in a new webapp of mine. I'm
thinking about Celery + RabbitMQ now, does anyone have feedback about this or
other recommendations? (i'm in a python wsgi stack)

~~~
pjscott
Celery is very slick and low-effort, and should fit in nicely with the rest of
your Python code base. Go for it!

------
alrs
It may read as tangential, but this feels like the beginning of end of "forget
free software, let's get Macs."

~~~
w1ntermute
Someone needs to start a similar project with Ubuntu. Things are clearly
getting a bit desperate for them (or so it seems) . If a good number of
quality developers finds a good arrangement whereby they can donate a few
hours of their time every week to significantly improving it, some real
changes could be created.

There are definitely a lot of people who are not happy with the way that Apple
has been acting as of late. What if we were to select one (1) laptop model and
work on making Ubuntu on it a seamless experience?

~~~
alrs
The best way to improve Ubuntu is to get involved with Debian. That's where
the heavy lifting is.

~~~
emillon
This. I started to get involved in Debian a few years ago (though I'm a long-
time user) and this is really really interesting work. There are a lot of
packages with too little workforce where you can go through a list of dozens
or hundreds of bugs.

alrs is true ; in most cases your work is automatically replicated to Ubuntu
(where the package-maintainer relationship is less strong than in Debian. Most
packages don't have a dedicated maintainer there).

And everyone can do it ; no sign-up or complicated process required. You just
have to find a sponsor for your packages which is just a fancy name for "every
code needs review". Debian is often described as bureaucratic but most of the
processes are very sane.

