

Ask HN:  How do you do batch processing? - ksowocki

I'm curious to to hear what technologies others use to process large volumes of batch jobs.<p>We've found it useful to be able to offload more process-intensive jobs from an end-user execution, and pass it into a queue ( which looks something like this http://abhinavsingh.com/blog/2010/02/memq-fast-queue-implementation-using-memcached-and-php-only/ )<p>What do you use?
======
maxdemarzi
Resque. <https://github.com/blog/542-introducing-resque>

I use the Ruby version, but there is a php version too.

<https://github.com/chrisboulton/php-resque>

~~~
andrewjshults
We're using a combo of the PHP version and the Ruby version and it's been
working great for us (we migrated over from a custom queue on top of MySQL.
The Sinatra based front-end is awesome for being able to monitor everything
that's going on and with a little bit of work you can get pretty much any of
the Ruby extensions to work with the PHP version (we're using the resque-
scheduler <https://github.com/bvandenbos/resque-scheduler> to do delayed
jobs). There are a few inconsistencies between how the jobs get handled
between the two (it's not a huge difference and I've been meaning to submit it
as a patch, I need to separate out some of the changes I made to make the PHP
version compatible with resque-scheduler).

If you have questions about implementing the PHP version, feel free to ping
me.

