I'm trying to set up a cron job as a sort of watchdog for a daemon that I've created. If the daemon errors out and fails, I want the cron job to periodically restart it... I'm not sure how possible this is, but I read through a couple of cron tutorials and couldn't find anything that would do what I'm looking for...
My daemon gets started from a shell script, so I'm really just looking for a way to run a cron job ONLY if the previous run of that job isn't still running.
I found this post, which did provide a solution for what I'm trying to do using lock files, not I'm not sure if there is a better way to do it...
Use flock
. It's new. It's better.
Now you don't have to write the code yourself. Check out more reasons here: https://serverfault.com/a/82863
/usr/bin/flock -n /tmp/my.lockfile /usr/local/bin/my_script
I do this for a print spooler program that I wrote, it's just a shell script:
#!/bin/sh
if ps -ef | grep -v grep | grep doctype.php ; then
exit 0
else
/home/user/bin/doctype.php >> /home/user/bin/spooler.log &
#mailing program
/home/user/bin/simplemail.php "Print spooler was not running... Restarted."
exit 0
fi
It runs every two minutes and is quite effective. I have it email me with special information if for some reason the process is not running.
grep -v grep | grep doctype.php
you can do grep [d]octype.php
.
&
if it is cron running the script.
As others have stated, writing and checking a PID file is a good solution. Here's my bash implementation:
#!/bin/bash
mkdir -p "$HOME/tmp"
PIDFILE="$HOME/tmp/myprogram.pid"
if [ -e "${PIDFILE}" ] && (ps -u $(whoami) -opid= |
grep -P "^\s*$(cat ${PIDFILE})$" &> /dev/null); then
echo "Already running."
exit 99
fi
/path/to/myprogram > $HOME/tmp/myprogram.log &
echo $! > "${PIDFILE}"
chmod 644 "${PIDFILE}"
>
" versus ">>
". Sorry about that.
[ -e "${PIDFILE}" ]
". If it does not, then it will start the program in the background, write its PID to a file ("echo $! > "${PIDFILE}"
"), and exit. If the PID file instead does exist, then the script will check your own processes ("ps -u $(whoami) -opid=
") and see if you're running one with the same PID ("grep -P "^\s*$(cat ${PIDFILE})$"
"). If you're not, then it will start the program as before, overwrite the PID file with the new PID, and exit. I see no reason to modify the script; do you?
It's suprising that no one mentioned about run-one. I've solved my problem with this.
apt-get install run-one
then add run-one
before your crontab script
*/20 * * * * * run-one python /script/to/run/awesome.py
Check out this askubuntu SE answer. You can find link to a detailed information there as well.
Don't try to do it via cron. Have cron run a script no matter what, and then have the script decide if the program is running and start it if necessary (note you can use Ruby or Python or your favorite scripting language to do this)
You can also do it as a one-liner directly in your crontab:
* * * * * [ `ps -ef|grep -v grep|grep <command>` -eq 0 ] && <command>
ps -ef|grep [c]ommand
-eq 0 ] && <command> where wrapping the first letter of your command in brackets excludes it from the grep results.
[ "$(ps -ef|grep [c]ommand|wc -l)" -eq 0 ] && <command>
[ $(grep something | wc -l) -eq 0 ]
is a really roundabout way to write ! grep -q something
. So you want simply ps -ef | grep '[c]ommand' || command
grep -c
.)
The way I am doing it when I am running php scripts is:
The crontab:
* * * * * php /path/to/php/script.php &
The php code:
<?php
if (shell_exec('ps aux | grep ' . __FILE__ . ' | wc -l') > 1) {
exit('already running...');
}
// do stuff
This command is searching in the system process list for the current php filename if it exists the line counter (wc -l) will be greater then one because the search command itself containing the filename
so if you running php crons add the above code to the start of your php code and it will run only once.
As a follow up to Earlz answer, you need a wrapper script that creates a $PID.running file when it starts, and delete when it ends. The wrapper script calls the script you wish to run. The wrapper is necessary in case the target script fails or errors out, the pid file gets deleted..
With lockrun
you don't need to write a wrapper script for your cron job. http://www.unixwiz.net/tools/lockrun.html
# one instance only (works unless your cmd has 'grep' in it)
ALREADY_RUNNING_EXIT_STATUS=0
bn=`basename $0`
proc=`ps -ef | grep -v grep | grep "$bn" | grep -v " $$ "`
[ $? -eq 0 ] && {
pid=`echo $proc | awk '{print $2}'`
echo "$bn already running with pid $pid"
exit $ALREADY_RUNNING_EXIT_STATUS
}
UPDATE .. better way using flock:
/usr/bin/flock -n /tmp/your-app.lock /path/your-app args
I would recommend to use an existing tool such as monit, it will monitor and auto restart processes. There is more information available here. It should be easily available in most distributions.
This one never failed me:
one.sh:
LFILE=/tmp/one-`echo "$@" | md5sum | cut -d\ -f1`.pid
if [ -e ${LFILE} ] && kill -0 `cat ${LFILE}`; then
exit
fi
trap "rm -f ${LFILE}; exit" INT TERM EXIT
echo $$ > ${LFILE}
$@
rm -f ${LFILE}
cron job:
* * * * * /path/to/one.sh <command>
I'd suggest the following as an improvement to rsanden's answer (I'd post as a comment, but don't have enough reputation...):
#!/usr/bin/env bash
PIDFILE="$HOME/tmp/myprogram.pid"
if [ -e "${PIDFILE}" ] && (ps -p $(cat ${PIDFILE}) > /dev/null); then
echo "Already running."
exit 99
fi
/path/to/myprogram
This avoids possible false matches (and the overhead of grepping), and it suppresses output and relies only on exit status of ps.
ps
command would match PIDs for other users on the system, not just your own. Adding a "-u
" to the ps
command changes the way the exit status works.
Simple custom php is enough to achieve. No need to confuse with shell script.
lets assume you want to run php /home/mypath/example.php if not running
Then use following custom php script to do the same job.
create following /home/mypath/forever.php
<?php
$cmd = $argv[1];
$grep = "ps -ef | grep '".$cmd."'";
exec($grep,$out);
if(count($out)<5){
$cmd .= ' > /dev/null 2>/dev/null &';
exec($cmd,$out);
print_r($out);
}
?>
Then in your cron add following
* * * * * php /home/mypath/forever.php 'php /home/mypath/example.php'
Docs: https://www.timkay.com/solo/
solo is a very simple script (10 lines) that prevents a program from running more than one copy at a time. It is useful with cron to make sure that a job doesn't run before a previous one has finished.
Example
* * * * * solo -port=3801 ./job.pl blah blah
Consider using pgrep (if available) rather than ps piped through grep if you're going to go that route. Though, personally, I've got a lot of mileage out of scripts of the form
while(1){
call script_that_must_run
sleep 5
}
Though this can fail and cron jobs are often the best way for essential stuff. Just another alternative.
Success story sharing
setlock
,s6-setlock
,chpst
, andrunlock
in their non-blocking modes are alternatives that are available on more than just Linux. unix.stackexchange.com/a/475580/5132