Cron – How to stop long-running cron process from being killed

cronpython

I have a long-running cron process, running a Python script, that seems to get randomly killed. In dmesg I see:

[230568.077358] init: cron main process (890) killed by TERM signal
[237517.974422] init: cron main process (19598) killed by TERM signal

How do I stop this from happening? I tried running the process via a lower nice value, e.g.

* * * * * nice -n 10 bash -c "/usr/local/myproject/myscript.py"

but it still gets killed. How do I diagnose and fix this?

Best Answer

Your script is being killed as a secondary effect. The log message indicates that cron itself is being killed. When a process with child processes is killed, it sends a SIGHUP to its children, causing them to exit as well. That's what's killing your python script.

If the log line you list line is immediately followed by a line that says init: cron main process ended, respawning, then you know that something outside of init/upstart killed it and it was automatically respawned. If that message does not exist, then cron was killed by init/upstart, the same as if you ran "sudo restart cron" or "sudo stop cron".

So your new research project is figuring out what is either killing or shutting down cron.

Related Solutions

Cron – How to Prevent Duplicate Cron Jobs Running

There are a couple of programs that automate this feature, take away the annoyance and potential bugs from doing this yourself, and avoid the stale lock problem by using flock behind the scenes, too (which is a risk if you're just using touch). I've used lockrun and lckdo in the past, but now there's flock(1) (in newish versions of util-linux) which is great. It's really easy to use:

* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/local/bin/frequent_cron_job

Cron Job – How to Run a Cron Job Manually and Immediately

Here's what I did, and it seems to work in this situation. At least, it shows me an error, whereas running from the command line as the user doesn't show the error.

Step 1: I put this line temporarily in the user's crontab:

* * * * *   /usr/bin/env > /home/username/tmp/cron-env

then took it out once the file was written.

Step 2: Made myself a little run-as-cron bash script containing:

#!/bin/bash
/usr/bin/env -i $(cat /home/username/tmp/cron-env) "$@"

So then, as the user in question, I was able to

run-as-cron /the/problematic/script --with arguments --and parameters

This solution could obviously be expanded to make use of sudo or such for more flexibility.

Hope this helps others.

Best Answer

Related Solutions

Cron – How to Prevent Duplicate Cron Jobs Running

Cron Job – How to Run a Cron Job Manually and Immediately

Related Topic