I've just discovered a suspicious binary running on my server that's been active for at least 22 hours. It's located at /root/GZ5pBwko/cCxf and is running various commands. While I normally handle server and container maintenance, I'm really unsure how this happened. My initial response was to block all ports except SSH and to reset all SSH keys, but I'm worried it might not be enough. Should I back up everything, wipe the server clean, and then reinstall the OS? What are the best steps to ensure the server is secure moving forward?
5 Answers
While wiping the server might seem like the easiest fix, you should consider consulting with a forensic team first. They can help figure out how the breach occurred without losing valuable evidence. Also, review your firewall logs for any suspicious activity.
I'd suggest checking your backups to see when the malware was first installed. It's safer to lose some data than to leave a threat actor with access. Change all critical passwords, especially for admin accounts, and consider getting professional help for a thorough investigation.
It's likely the server was compromised through SSH, possibly due to weak passwords or a compromised Docker image. To really secure yourself, you should definitely consider doing a full wipe and fresh install. Check your logs, though; they might contain clues about how this happened.
First, check the timestamps of that suspicious binary and correlate them with your login logs to figure out how it might've got there. You might be able to track down the IP that accessed your server and find any other systems it might've infected. If you're not running any sort of monitoring like Tripwire, you should definitely consider adding that in the future.
Make sure to back up your configurations, Docker Compose files, and databases before you wipe the server. It's also crucial to check if any of your containers have the Docker socket mounted, as that could give an attacker root access to your host system.

Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures