Web based file watcher

We needed to check a file was being written to a remote server at least every 15 minutes or so. This bash script, run as a cron job, uses curl to get the HTTP headers and then sed to extract “Last Modified” time. The actual log file is not downloaded.

#!/bin/bash

START=$(/bin/date +%s)

LASTMOD=` /usr/bin/curl -Is http://www.example.com/logs/log.txt | /bin/sed -n 's/^Last-Modified: *\([^ ]*\) */\1/p' `

LASTTIME=$(/bin/date +%s -d "$LASTMOD")

DIFF=$(($START - $LASTTIME ))

if [ "$DIFF" -gt "900" ]

then

MAIL_TXT="Subject: Log file not updating\nFrom: info@example.com\nTo: info@example.com\n\nAge is $DIFF"

echo -e $MAIL_TXT | /usr/sbin/sendmail -t

fi

Cron entry is

*/15 * * * * /home/robin/checkweblastmod.sh