Papertrail has had the ability to alert on searches that match events for years, but what about when they don’t? When a cron job, backup, or other recurring job doesn’t run, it’s not easy to notice the absence of an expected message. But now, Papertrail can do the noticing for you with inactivity alerts. Papertrail inactivity alerts allow you to setup notifications when searches don’t match events.
Set up an inactivity alert
From the create/edit alert form, choose “Trigger when no new events match”
Once saved, the alert will send notifications when there are no matching events within the chosen time period. Use this for:
- cron jobs
- background jobs which should run nearly all the time, like system monitors/pollers and database or offsite backups
- lower-frequency scheduled jobs, like nightly billing
If you have cron jobs, backup jobs, or other recurring or scheduled jobs, they almost certainly already generate logs. Here’s how to have Papertrail tell you when they don’t run or run but don’t complete successfully:
- Search for the message emitted when a cron job finishes successfully (example: cron)
- Click “Save Search”
- Attach an alert, like to notify a Slack channel or send an email.
No logs? No problem.
Very rarely, a recurring job doesn’t generate log messages on its own. For those, use the shell
&& operator and
logger to generate a post-success log message. For example,
./run-billing-job && logger 'billing succeeded' will send the message billing succeeded to syslog if and only if
run-billing-job finishes with a successful exit code. Use
'billing succeeded' as the Papertrail search.
What do you think?
Give inactivity alerts a try and if you have questions or feedback, let us know.