Should I Replace SSH Keys When Upgrading My Server’s OS?

When moving to a new OS, I don't typically bring over the known hosts; I just take the authorized keys. The first time you connect, it prompts for confirmation of the new key, and you can accept it then. Some folks might find this process challenging, but it provides a clean slate. However, don't use the option to ignore the host key; it's risky.

It might be worth considering certificate authentication instead. If your server can use a certificate, it could take the place of the host key, and that might simplify some of your trust issues.

Generally, you wouldn't need to regenerate your SSH keys during an in-place upgrade unless there's a known security compromise or you're moving to a new encryption standard. Since you're keeping the same server functionality, it makes sense to keep the existing keys to avoid those trust issues with automation tools. If you're using something like Ansible, it can help manage host key changes, but that's often more hassle than it's worth if you're not dealing with a compromised system.

If you're not planning to replace the entire system, I see no reason to change your host keys. The server is still the same entity in terms of user connection, and changing keys could confuse users unnecessarily. As long as the old system wasn't compromised, keeping the old keys is the way to go.

I prefer to rotate host keys during a rebuild for a clear trust boundary. Sure, it breaks things temporarily, but if you manage it well via config management, it's manageable. New keys are more secure, and over time, this approach keeps your system cleaner.