What's a good break-glass path when SSH is restricted to a bastion?

November 21, 2026 5 answers
Asked by _a57!mavquelo.8

We're planning to allow SSH to instances only from a bastion security group. What should the emergency access path look like if the bastion fails, without leaving a management network open as a permanent bypass?

I'm also curious what to do if Session Manager isn't available during the same incident.

Answers

_g57+vornexa.4

Session Manager is a reasonable fallback if the instances are already managed by Systems Manager. Put StartSession access in a separate emergency role, use short-lived role sessions or an approval process, and alert on its use through CloudTrail. Enable session logging where it applies.

Test the dependencies before relying on it. The agent needs to be running, the instance needs the right permissions, and it needs connectivity to the Systems Manager endpoints. A broken agent or endpoint path can take this option out during an incident.

Also check the logging mode you actually use. Session Manager cannot log the contents of SSH or port-forwarding sessions because that traffic is encrypted inside the tunnel. A plain interactive session and an SSH tunnel don't give you the same visibility.

_a57!mavquelo.8:

Good distinction on logging. We'll test both the session path and what our alerts actually show.

Reply to this answer
Vexmelo-3r

Make the bastion replaceable instead of treating it as a precious machine. A second bastion in another availability zone, health checks, and an auto scaling group can reduce the chance that one failed host blocks everyone.

Have a way to rebuild it from a known image and restore its configuration. Test the gateway service too, not just whether the instance is running. A healthy VM with a broken authentication service is still a failed bastion.

Reply to this answer
q.582+pluvnelo

For an agent or network problem on the target, look at EC2 Serial Console as a separate recovery path. It can help troubleshoot a boot or operating system issue without depending on the instance's network connection, but it needs to be enabled and authorized in advance and has instance and operating system prerequisites.

If the machine still won't come up, another option is to detach its EBS volume and attach it to a recovery instance for inspection. That is disk recovery, not a live SSH session, so document the steps and risks before you need them.

Reply to this answer
4v!bracketwicket_529

Do you need the bastion for anything besides SSH? If not, consider using Session Manager as the normal access path rather than maintaining a gateway plus an emergency gateway. That can remove inbound SSH altogether, though it also makes the agent, IAM, and endpoint setup central to access.

There are tradeoffs. File copying can be less convenient, and access to other private services may still need a suitable host or port-forwarding setup. Don't swap architectures just for the sake of it.

Reply to this answer
r.529~torvani

EC2 Instance Connect Endpoint is another option worth evaluating if you need SSH without a permanent bastion. It still needs the right IAM permissions, network reachability, security group rules, and SSH authentication on the instance. It doesn't magically bypass those controls.

If you use it for emergencies, make the access narrowly scoped and time limited, alert on the relevant access, and test how the temporary permissions and security group changes get removed afterward. Avoid leaving a broad management CIDR open just to make the runbook easier.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions