Jump to: ☰ Menu

Cron jobs execution monitoring in slack

on Slack , Unix

This post is in the same vein than the previous one but applies to cron jobs instead of celery tasks.
I used to redirect stderr to a log file for my cron jobs but it took me weeks to figure the jobs were crashing in the first place.
So now I want my jobs to crash loudly, for that purpose I use slacktee util that posts stdin to slack.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#!/bin/bash

# Take a command to run in argument. Posts stderr to slack if it fails. 
runcheck() {
    ERR="$($@ 2>&1 > /dev/null)"
    if [ $? -ne 0 ]; then
         echo "\`${1##*/}\` failed" | /usr/local/bin/slacktee.sh -p -u "$HOSTNAME" --config /home/ubuntu/.slacktee-alert
        if [ ! -z $ERR ]; then
             echo "${ERR}" | sed -e 's/^/>/' | /usr/local/bin/slacktee.sh -p -u "$HOSTNAME" --config /home/ubuntu/.slacktee-alert
        fi
    fi
}

main() {
    runcheck $@
}
main $@

runcheck.sh

So my cron tasks are declared like so :

1
2
3
4

$ crontab -l
SHELL=/bin/bash

# 0 */2 * * * /home/ubuntu/bin/runcheck.sh /home/ubuntu/my_project/binaries/cron/run_critics.sh

In case of failure, a slack alert pops with all needed infos : hostname, command that fails and error message.

slack notification with developers facemojis