I'm dealing with a recent issue where a scripted Oracle process failed to send offsite backups, even though the daily email was sent out without any visible errors. I want to set up an alert in my monitoring tool (WhatsUpGold) for both detecting failure messages within the email body and confirming if the email even arrives at all. Ideally, I'd prefer a direct method through WhatsUpGold rather than relying solely on emails. If there are other processes or scripts that could help streamline this monitoring, I'm open to suggestions. Just looking for some solid advice on best approaches!
5 Answers
Power Automate is a great way to handle this! You can create a flow that activates when an email is received. Just have it check off a bit of data to confirm the email came in, then run a follow-up process at the end of the day to see if the email was logged. If not, it reports back. To cover your bases, consider integrating Teams messaging as a fallback if email goes down for any reason—this adds another layer of reliability!
Email alerts can be a bit of a patchwork solution and usually aren't the best way to handle critical monitoring. Ideally, your Oracle script should send success and failure messages directly to WhatsUpGold. This way, your monitoring tool becomes the main source of truth, allowing it to manage notifications better—be it through email, SMS, or just updating on a dashboard. If you have to stick with emails and if WhatsUpGold can't process them, consider using Power Automate to read the emails for success/failure content and update your monitoring tool appropriately. It’s not the most scalable, but it works!
To tackle the issue of 'email never arrives', consider using a dead man's switch pattern. You can have your Oracle script ping a URL (like Healthchecks.io, which has a free tier) upon a successful run. If no ping is received within a specified timeframe, it triggers an alert—bypassing the need to monitor email absence entirely. For checking failure messages in the email body, tools like Power Automate can help. However, if you can modify the Oracle script, aim for it to exit with a non-zero code on failure and report status directly to the WhatsUpGold API instead of relying on email alerts.
If you can script outputs, you might be able to store the failure message as a variable in WhatsUpGold using %ActiveMonitor.Payload if that's relevant to your monitoring setup. It’s been a while since I used it, but this community has a lot of knowledgeable folks—check the forums for more specific advice!
You might want to give SIGNL4 a shot—it integrates with WhatsUpGold and allows for email alerts as well. It also has heartbeat checks, so if a webhook request isn't received in time, an alert will be triggered. A cron job could be set up to ensure your server is online too.
Just a heads-up, Healthchecks.io can also ping via emails and filter specific keywords in the subject or body. But modifying your script to return a non-zero exit code on failure is ideal. That way, your monitoring system stays simple and doesn’t need to search for keywords.