I manage a number of HP ProLiant servers with licensed iLO, all set up with email alerts configured identically. Each server has staggered monthly reboots scheduled during maintenance windows, during which they should send emails for various alerts like NICs going offline. However, I've noticed that four of the servers only send emails when I test the email alerting, but not during the reboots. I've double-checked the configurations, and everything seems correct. Without having to disconnect network cables or unplug storage drives, what are some ways I can simulate iLO alerts to troubleshoot this issue during work hours?
7 Answers
If you have a dual power supply, one option is to pull one of the power cords. This is usually the easiest thing to do if you have physical access to the server. If not, and it has a single power supply, you could try removing the server cover to trigger an intrusion alert.
Great suggestion! Just make sure to keep an eye on what happens next.
Honestly, to test alerting, I sometimes unplug one of the PSU cables and cross my fingers. I've also used a remote PDU to cut power to specific slots if I want to do it safely from afar. Just a caution, though, make sure that the other PSU is reliable!
Another option is to shut down or 'no shut' the switch port that's connected to the server. This can simulate a network issue without needing to physically unplug anything.
Have you considered using their REST API? It might provide some options to trigger alerts without any physical intervention. Just a thought!
Instead of pulling cables, you could simulate other alerts too, like adjusting the temperature thresholds for low/high alerts that match your server’s current conditions. I do that on my Dell DRACs, and it has worked pretty well for me!
Are you trying to test the email from the iLO device itself, or are you checking if alerts are triggered inside the iLO portal? It might be a good idea to validate your email flow first, checking logs on your mail server to see if the alerts are being sent at all. If the email logs look good, you could safely disconnect less critical components like a system fan or put the server in maintenance mode to conduct more significant alert testing without downtime.
Make sure to check the IML or iLO Log for events since the last reboot. There might be alerts that get triggered by events logged there. Some alerts, like a NIC going down, depend on the AMS Agent for Gen8 and later. If those four servers don't have that agent installed, that could be why you're not seeing alerts.
Yeah, pulling a cord can definitely cause an alert if you've got dual supplies! Glad you mentioned covering options for single supplies too.