PowerEdge Server Hard-Resets Every 728 Minutes With No Crash or Hardware Log

0
0
Asked By MellowCedar47 On

I'm troubleshooting a Dell PowerEdge T340 running Windows Server 2016 that has been hard-resetting at remarkably consistent intervals. Every shutdown produces Kernel-Power 41 with BugcheckCode 0, Event 6008, no blue screen, and no dump file. The actual shutdown times come from the Event 6008 message text rather than the Event 41 timestamps written after reboot.

Across twelve consecutive events, the interval averages 728.47 minutes—12 hours, 8 minutes, and 28 seconds—with only about 1.2 minutes of total variation. One uninterrupted gap was 3,641.7 minutes, exactly five periods, meaning the apparent timer continued running through five missed events and then reset the server on the next cycle. The server does not appear to reboot or hang during the skipped intervals.

A heartbeat logger flushed data every five seconds. Immediately before each failure, free memory remained around 56 GB, nonpaged pool was stable near 350 MB, disk queues were empty, and handles and thread counts were steady. The machine stops instantly, then takes roughly three minutes for POST and startup.

I have already investigated the power supplies, thermals, iDRAC watchdog and SEL, Dell management software, Windows Update, endpoint security, scheduled tasks with 12-hour repetition, VSS tasks, MySQL memory usage, the disconnected secondary NIC, the CMOS battery, and ordinary BSOD causes. The iDRAC SEL is completely silent, although it has logged genuine power-input failures in the past. Both PSUs report healthy, and the server is connected through an APC Back-UPS XS 1500M. I do not have historical UPS transfer data because monitoring software was not installed when this began.

The remaining possibilities seem to be an APC self-test or transfer, a power or electrical event outside the server, outdated PSU/BMC firmware, or some other device below the operating-system level. My next test is to bypass the UPS and run the server directly from mains power.

What could create a free-running period of about 728.4 minutes that survives reboots and occasionally skips several cycles? Could an APC Back-UPS self-test or a power event cause an instantaneous reset without an iDRAC SEL entry?

4 Answers

Answered By HarborViolet62 On

Do a detailed review of both the Application and System logs around the predicted failures, but also check anything external that might control power: UPS self-test history, outlet groups, building automation, managed PDUs, and equipment sharing the circuit. A scheduled procedure could be switching the wrong outlet, especially if the server is connected to an outlet group intended for another appliance.

CobaltLark31 -

The weekend gap is worth examining too. Compare it with business hours, holidays, maintenance schedules, and who or what is active in the building. A process that runs only on working days could explain why the free-running timer sometimes appears to skip several cycles.

Answered By QuartzOtter8 On

The cleanest next test is to remove the UPS completely: connect the server directly to mains power, with no USB connection to the UPS. If the resets stop, add the USB connection back, and then reconnect the UPS afterward. With redundant PSUs, you could also put one PSU on mains and one on the UPS, but change only one thing at a time and replace the power cords if possible.

MellowCedar47 -

That is the next isolation test I’m planning. The server currently has both redundant PSUs connected through the same UPS, so separating those power paths should help distinguish a UPS issue from a server-side fault.

Answered By RivenPine90 On

Update the BIOS, iDRAC, and related Dell firmware before spending too much more time on software theories. The current versions are several revisions behind, and firmware bugs involving power management or watchdog handling may not generate a Windows bugcheck or an SEL entry. I’d also verify that no expired server or management license is triggering a power-management action, although that is a less likely explanation than the UPS or electrical path.

Answered By BlueMaple19 On

I would not treat a quiet iDRAC SEL as proof that power is innocent. A very brief voltage sag or an interruption handled below the BMC’s logging threshold could still reset the machine without leaving a useful record. The exact interval makes the UPS, a managed outlet, or another timed electrical load more suspicious than ordinary hardware failure.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.