

Ask HN: How do you prefer to run background jobs for a website/service? - billymeltdown

Just about any sizable web-available service generally needs to run background jobs. This almost always includes some interaction with inbound or outbound mail services which tend to be slow (even with my man Exim on the case). Kinda curious what solutions HN readers prefer for handling this. E.g. a lot of Rubyists use BackgrounDRb, but I prefer my little Looper.rb.
======
thwarted
cron. Should exist on every UNIX system, the most common syntax is
standardized (you can even call it a DSL if you want), there are multiple
implementations, it supports complex command lines, its jobs can be written in
any language, it is well documented, and its limitations and security are well
understood.

~~~
billymeltdown
I know all about cron, I was looking to see if people choose to just queue up
all their jobs in a scheduler, if they fire up stand alone scripts, daemons,
etc.

~~~
thwarted
If you want a job queue, then a scheduler won't help much. I've found that
stand alone scripts are usually better because then you write them so they can
be "restarted" and maintain state outside of their address space, so you
reduce the amount of lost work if you have to kill it. Not necessarily the
case with long running deamons.

