I'm in a bit of a bind here. I recently started a new job and inherited a system admin position where the previous person left no documentation—no passwords, no network diagrams, nothing. I've just found out that we're currently locked out of our ESXi environment on a Hitachi advanced server DS120, which is critical to our infrastructure. Rebuilding everything would be really costly, especially with the medical vendors involved.
The server hasn't been rebooted in years and I'm uncertain about backups. I've opened a ticket with Hitachi support but I'm still waiting for a response. I looked through their documentation, but it's quite vague, and I couldn't find any helpful videos.
I'm considering resetting the BMC to regain access to the ESXi password, but I'm worried about the potential impact on the virtual machines. Someone mentioned holding down the (I) button on the front panel for 30-60 seconds to reset the IP and credentials. I'm just unsure if that would affect the running VMs. There's also a reset button on the front panel, but I'm not clear on what it does.
I'm even thinking about using N-able to perform a system backup and restore on another ESXi host if needed. Has anyone reset the BMC on a Hitachi DS120 before? Did it impact your production environment?
3 Answers
I've reset the BMCs, iDRACs, and ILOs multiple times at my last job, and from my experience, these procedures are generally safe and shouldn't disrupt your operations.
Resetting the BMC is usually safe and shouldn't have any impact on your running VMs. But if you're looking to reset your VMware host root credentials, the process can vary depending on your ESXi version, and there's really no officially supported way to do it. Typically, the supported method involves reinstalling ESXi, which will temporarily take your VMs down. The installation process should detect your existing datastore—just make sure to preserve it during the install so you can re-register your VMs afterward.
I can't give you tech advice directly, but this sounds like a significant issue for the business as a whole, not just for you as the IT person. It might be worth your time to run the financials on getting the credentials legally from the previous admin versus the cost of a rebuild. Handover of credentials is typically something that companies can demand from former employees.
If I were in your shoes, I'd be cautious about touching production systems. I'd definitely want something in writing from management acknowledging the current risks and the potential downtime, so you're not left holding the bag for past oversights.
Totally hear you on that. I've been thinking along those lines, but getting that reassurance in writing does seem like the smart move. Appreciate it!
Thanks for clarifying that! I was leaning toward thinking it wouldn't affect anything, but I needed confirmation. So, it's good to know.