

Using Gearman For Distributed Alerts - omakase
http://tech.backtype.com/using-gearman-for-distributed-alerts

======
kvogt
I would be very careful with gearman.

We tested it at Justin.tv and managed to cause the main process (gearmand) to
spin at 100%, stop accepting connections, and lots of other things. Not quite
ready for production use, especially if you need persistence and use the
memcached plugin.

Also, the python client library is in pretty rough form. Our developers
submitted a patch or two, but it still doesn't reliably handle errors, network
outages, failover, or reconnects. You probably want all of these things in
your job server of choice.

~~~
shib71
What did you end up using instead?

~~~
kd5bjo
We're in the process of migrating to RabbitMQ. It looks pretty good, but we
haven't really put it through its paces yet.

~~~
omakase
To clarify, Gearman is not meant to be used as a message queue, but instead
it's a job server. The example in the post is an interesting use case (and
convenient for us because we were already using it as a job server), but not
really what it's designed for. I'll follow up with a post on how we use it to
do work.

~~~
kd5bjo
What is the difference between a job server and a message queue that requires
a return queue?

~~~
gstar
Not much, but the Gearman default is to require a response. You can almost
treat gearman like blocking unix pipes over tcp, mixed with a FIFO queue. In
fact, it comes with commandline tools to do just that. You get the option of
submitting a job without waiting for a response back too, though.

I suppose Gearman is more of an RPC broker with sync and async modes than an
MQ, but the differences are mostly semantic.

------
moe
I'm sure I must be missing obvious here but why are they not just pushing
those mails out directly?

All that gearman buys them here is... a longer delay until the mail is
actually sent.

~~~
omakase
You're right, it does increase the delay. It's more of an example of how
Gearman can be used.

Also, I _hate_ the idea of running an MTA on 30 machines that shouldn't be
sending mail.

~~~
moe
Yup and a good sized one at that, working snippets are always welcome. I was
just a bit confused due to the contrived use-case.

I don't share your concerns about the MTA (they are lightweight and it's
usually helpful to know that sending mails won't block) but to each their own.

