
Rethinking Cron - jfi
http://adam.heroku.com/past/2010/4/13/rethinking_cron/
======
aaronbrethorst
Meta point: I'd really like to see (non-www) subdomains added to the article
title. I clicked through to the article thinking that Heroku had redesigned
their Cron management only to discover something completely different.
Subdomains are included in Posterous links; why not everywhere?

------
washingtondc
I generally agree with the weak points that the OP pointed out regarding cron.
That being said, throwing out cron, or any other built-in system tool, isn't
always a great idea.

When you adopt a new scheduler, you're inevitably creating work for your
enterprise. It's yet another thing that must be learned and documented. There
may be a new syntax to learn, which will mean you're incurring training costs
(and the cost of errors) down the line.

Programmers love to improve things, but there's a real world cost to change
that can't be ignored. cron may be archaic and lacking power, but it's well
understood.

As for enterprise schedulers, they've been around for years. Off the top of my
head, I've worked with 'Autosys', and 'Appworx'. These were robust,
enterprise-ready schedulers that supported conditional execution, locks, as
well as their own scripting language.

~~~
dalore
To be fair he didn't say throw out cron. He said cron will have a place for
rotating log files etc. Which to me is infrastructure type stuff and not
application level.

So the sysadmins will still use cron to do the infrastructure type jobs. But
when a programmer needs something in his app to be scheduled he can use the
replacement, this also frees them up from asking the sysadmin to install a
cron script.

------
orblivion
_And the subtle differences between a cronjob’s shell environment and your
command prompt’s shell environment can be maddening._

This is my favorite thing to complain about with cron. I can't believe that
there's still not a way to test run cron entries. So far I just add > tmp.out
>> tmp.err, set it to run every minute, and wait.

~~~
Pistos2
[http://serverfault.com/questions/85893/running-a-cron-job-
ma...](http://serverfault.com/questions/85893/running-a-cron-job-manually-and-
immediately)

I completely agree that there should be a better, direct way to test cron
jobs.

~~~
pmccool
I used to do this by exec()ing a shell with no environment. 'Twasn't perfect,
but it caught a lot of irritating little errors.

------
nethergoat
At Bizo (<http://dev.bizo.com>), we manage scheduled tasks via Hudson.

It works great for us. It's centralized, has a web interface, and is already
familiar to everyone on the team (we already used Hudson for CI). It has a ton
of plugins, so notifications, single sign-on, and a host of other desired
features were no problem to bolt on.

Here's a quick post from a few months back on why we chose Hudson:
[http://dev.bizo.com/2009/11/using-hudson-to-manage-
crons.htm...](http://dev.bizo.com/2009/11/using-hudson-to-manage-crons.html)

------
surki
How about <http://upstart.ubuntu.com/>

~~~
ElliotH
Upstart is more for replacing init. Its less suitable for jobs that need to be
started at particular times since its designed for starting and stopping
services during boot and changes in runlevel.

~~~
surki
I think long term plan for upstart is to replace cron[1](generating events at
scheduled time ...)

And some more general information here [2]

[1] <http://upstart.ubuntu.com/faq.html#replace-cron>

[2] [http://www.netsplit.com/2006/09/01/upstart-can-now-
replace-s...](http://www.netsplit.com/2006/09/01/upstart-can-now-replace-
sysvinit/)

~~~
pyre
Hopefully they do a better job than Apple. I had a number of issues with their
init/cron replacement.

~~~
masomenos
so true! Every time i'm forced to deal with launchd for scheduling, I wish I
was using cron.

------
pinko
I'll reprise my comment from the last time a new and better cron was posted
here[1]:

While cron is clearly pushed way past its capabilities by many people (without
always realizing it), this strikes me as a reinvention of something that's
been around forever: the batch system.

If the goals are improved reliability, management, fault-tolerance, etc.,
you'd be better off using mature software that was designed to solve this
problem, and already has good community support, like Condor.

[1] <http://news.ycombinator.com/item?id=916737>

------
javan
I developed Whenever (<http://github.com/javan/whenever>) to at least make the
Cron syntax more reasonable.

~~~
tmorton
Whenever is awesome.

(incidentally, check your pull requests :)

------
nwmcsween
OP wants a distributed operating system. Linux, FreeBSD and Windows by design
are not distributed. POSIX specifically cripples this. How do you take into
account an action that needs to be distributed but have a central return
point? How will fault tolerance be handled? Expecting every application to
handle this won't work, what's needed is a distributed operating system.

------
rsl7
launchd

~~~
macrael
I'm genuinely confused as to why launchd isn't part of this conversation. It
is definitely a replacement for cron (and init) and seems to be a pretty good
one. I haven't done an extensive comparison between it and cron, but it has
definitely been easy to use and worked well for me. It is very flexible and
you use it by laying out exactly what you want it to do in XML.

~~~
pyre
I had issues in the past with scheduling processes in launchd. It's been a
while, but I remember having these issues at least two OS revisions _after_
Apple initially launched it. (The issue was related to starting a process and
having launchd restart the process if it died. There were options to toggle
this but it wasn't working for me, and I _was_ reading the Apple developer
docs on launchd.)

Also, I personally don't feel that using XML as a config file format is
necessarily an _improvement_ over what cron offers now. (I know that I'm
skipping the fact that 'XML is self-validating,' but I don't really find a
lack of a self-validating config file to be a major issue with cron)

~~~
joezydeco
One of the biggest gotchas I found was that launchd bails if the process takes
less than 60 seconds to run. Putting a "sleep 60" at the tail of the script
fixed a lot of problems for me.

At first I was kind of upset that this condition was there, but in retrospect
it might have been a very wise design choice to stop runaway triggers.

~~~
pyre
Now that you mention it, I remember that quirk. I had an issue with _not_
being able to disable that quirk when running a scheduled process. launchd
would stop scheduling a process that was scheduled to run every 5 minutes if
one of the runs took less than 60 seconds. It really irked me because this
60-second-rule only makes sense when you are talking about re-spawning a
constantly running process, and should _not_ affect the behaviour of processes
running on a schedule.

> _At first I was kind of upset that this condition was there, but in
> retrospect it might have been a very wise design choice to stop runaway
> triggers._

It doesn't _really_ stop runaway triggers if all of the scripts have a 'sleep
60' at the end of them. And if you are concerned about runaway triggers you
could always make the rule something like "bail if the process dies after <60s
more than once in a 2-minute period" or similar since there _are_ legitimate
processes out there that take less than 60 seconds to run.

~~~
joezydeco
Well, I didn't say it was _perfect_. =)

------
bobzimuta
No mention in the article about Gearman for worker queues so I'll promote it
here. I like its simplicity and language agnosticism. <http://gearman.org/>

