

I need a dedicated message queue... Or do I? - kbenson
http://yaketyhack.blogspot.com/2013/04/i-need-dedicated-message-queue-or-do-i.html

======
aartur
Have I read it correctly that you are writing a system to distribute jobs
among real people? When a person has a queue for assigned jobs? Then this is
totally not suitable for a message queue system (AMQP/JMS). MQ system is for
distributing messages among software components at runtime. Specifically it is
not designed for querying messages or doing other stuff typical for RDBMSes.

~~~
kbenson
It _starts_ as distributing jobs among real people, but eventually a good
portion will be shifted to being automated. Additionally, new job types will
be added that are automated from the beginning, and the quantity of those will
be much higher.

That more expanded use case is what's making this even worth talking about,
IMHO. If it was purely jubs for real people, I would definitely just use a
RDBMS.

I should have made that a bit clearer.

~~~
aartur
So probably you have two separate subsystems of "jobs" distribution. Assigning
to a person, querying and updating job statuses from some UI is very different
from automatic computation of a result by a software component.

~~~
kbenson
Is it? I'm trying to replace manual tasks, and I would like to do it in steps.
First by implementing a joq queue that can be used in house by employees to
drive what work needs to be done more effectively, then by trying to automate
some of those tasks, and providing what the automation _would_ have done as a
guide for the employee, and finally when there's enough confidence that the
automated routine is sufficient to the task, letting it take over without
human intervention.

~~~
dragonwriter
Assuming that the order in which tasks should be done doesn't change after
they enter the system, there is no reason you can't use a message queueing
system here. Obviously, you need software components that act as proxies for
each user, but you need that in any worflow system.

Probably the best thing to do is to see if you can draw up a flow diagram that
shows where messages need to flow to accomplish what you want. But it sounds
like using a message queueing system as the main internal mechanism for
communication.

~~~
kbenson
I've decided to implement in an RDBMS, partly because this will allow me to
become familiar with the concepts of message queues at a much lower level. I'm
going to try to provide a semi-generic solution so the result is usable for
more than my initial implementation, but I think this will provide the most
benefit to both my employer and myself. I'll be intimately familiar with the
core concepts, and will then know how to parse the feature/capability lists of
the various implementation available, and will then be able to make an
informed decision if I need to replace the system later.

I like how writing about it forces me to crytalize my thoughts, so I'm going
to write it up[1].

[1]: [http://yaketyhack.blogspot.com/2013/04/rolling-your-own-
mess...](http://yaketyhack.blogspot.com/2013/04/rolling-your-own-messagejob-
queue-in.html)

P.S. I've used a similar technique before in the absence of a reliable
recommendation on a technology, due to unknown constraints or otherwise. The
result of that was "Don't write your own ORM if you can avoid it (unless it's
just to learn)". We'll see if the same result applies here...

