Jump to: Menu

Cron jobs execution monitoring in slack


This post is in the same vein than the previous one but applies to cron jobs instead of celery tasks.
I used to redirect stderr to a log file for my cron jobs but it took me weeks to figure the jobs were crashing in the first place.
So now I want my jobs to crash loudly, for that purpose I use slacktee util that posts stdin to slack.

 1#!/bin/bash
 2
 3# Take a command to run in argument. Posts stderr to slack if it fails. 
 4runcheck() {
 5    ERR="$($@ 2>&1 > /dev/null)"
 6    if [ $? -ne 0 ]; then
 7         echo "\`${1##*/}\` failed" | /usr/local/bin/slacktee.sh -p -u "$HOSTNAME" --config /home/ubuntu/.slacktee-alert
 8        if [ ! -z $ERR ]; then
 9             echo "${ERR}" | sed -e 's/^/>/' | /usr/local/bin/slacktee.sh -p -u "$HOSTNAME" --config /home/ubuntu/.slacktee-alert
10        fi
11    fi
12}
13
14main() {
15    runcheck $@
16}
17main $@

runcheck.sh

So my cron tasks are declared like so :

1$ crontab -l
2SHELL=/bin/bash
3
4# 0 */2 * * * /home/ubuntu/bin/runcheck.sh /home/ubuntu/my_project/binaries/cron/run_critics.sh

In case of failure, a slack alert pops with all needed infos : hostname, command that fails and error message.

slack notification with developers facemojis