
Using prime numbers to avoid cron task collision at sametime - akshayB
I would like to get input from you all. We have legacy system (bit of a pain to maintain) where multiple corn jobs runs (8 to be precise) and engineers who created the system put everything to run at 5 minutes interval. Computationally task is heavy and often results into a crash or running out of memory but business owners are fine if things are delayed by like an 1 hour or so.<p>My idea was to run on prime intervals instead of every 5 minutes. Couple of them have depend on output from others tasks but that should not be an issue.<p>Job 1 - Every 5 minutes ( computational less intense )<p>Job 2 - Every 7 minutes<p>Job 3 - Every 11 minutes<p>Job 4 - Every 13 minutes<p>Job 5 - Every 17 minutes ( computational more intense )<p>.... and so on<p>This should put a significant dent in avoiding them running at same time and leaving some room for each task to finish itself. I was just wondering if there is any other alternative that I can explore here.<p>Thanks for any suggestions in advance.
======
CarolineW
If the owners don't care then just have

    
    
      */16 * * * * bin/RunJob_1
      2-59/16 * * * * bin/RunJob_2
      4-59/16 * * * * bin/RunJob_3
      6-59/16 * * * * bin/RunJob_4
      8-59/16 * * * * bin/RunJob_5
      10-59/16 * * * * bin/RunJob_6
      12-59/16 * * * * bin/RunJob_7
      14-59/16 * * * * bin/RunJob_8
    

If that doesn't work for you then tell us why. As it is your problem is
underspecified.

Also, you say some are more compute intensive - have you actually measured the
load and time-to-finish?

Prime numbers and similar techniques can help, it just depends if you actually
just want to kludge something together, or do it properly.

------
ultrablue
Is your goal to avoid all collisions?

If so your current scheme won't prevent that. For example, Job 1 and Job 2
will both run at 0035, 0405, 2025, etc. (assuming that both jobs start at
0000).

~~~
CarolineW
That depends - cron time steps always restart from 0 in the hour, so what you
say is valid, but the most obvious way of implementing what I think is being
suggested doesn't have _exactly_ that problem.

~~~
ultrablue
Ah, didn't know that, thanks!

------
pwg
> I was just wondering if there is any other alternative that I can explore
> here.

Investigate task spooler:
[http://vicerveza.homeunix.net/~viric/soft/ts/](http://vicerveza.homeunix.net/~viric/soft/ts/)

This will let you queue up the tasks, and each will run as soon as the last
has finished, but you won't get a "thundering herd" where they all are trying
to run at once.

