I'm learning AWS and created an EC2 instance in a private subnet. The subnet uses a NAT gateway for outbound access, while another instance is in a public subnet with a route to an internet gateway. I can connect to the private instance through AWS Systems Manager Session Manager, but I can't connect with SSH even though the security groups allow port 22.
I'm trying to understand what makes an EC2 instance private and what would be required to SSH into it without using a public IP, a private connectivity service, or an interactive SSM session. Is there a way to do this without an application or network load balancer?
4 Answers
The key networking distinction is that a private subnet has no default route to an internet gateway. Its NAT route is for outbound connections only. Security-group rules cannot create reachability where the routing path does not exist, so opening port 22 alone will not solve this. SSM works because the instance initiates outbound connections to the Systems Manager endpoints.
A bastion host is a traditional design: connect to the bastion’s public address, then SSH from the bastion to the private instance. The private instance’s security group should allow port 22 only from the bastion’s security group, not from the whole internet. A VPN or an AWS Client VPN connection is another way to obtain private network access. These approaches can work, but they add infrastructure and operational responsibility compared with SSM.
A more accurate description is an EC2 instance in a private subnet. Its route table does not have a path through an internet gateway, so unsolicited connections from the internet cannot reach it. A NAT gateway only allows connections initiated from the private subnet to go out; it does not allow inbound SSH connections back in.
The usual choices are a bastion host in a public subnet, a VPN, or AWS Systems Manager. You can also use SSH over an SSM tunnel, which gives you the normal SSH client experience without exposing port 22 publicly. For example, an SSH configuration can use an SSM ProxyCommand that runs `aws ssm start-session` with the `AWS-StartSSHSession` document. This requires suitable IAM permissions, AWS CLI v2, and the Session Manager plugin.
You generally do not need an ALB or NLB for administrative SSH access. SSM is usually the safest and simplest option.
A public or elastic IP by itself does not make an instance reachable. The subnet also needs a route to an internet gateway, and the instance needs security-group and network-ACL rules allowing SSH. Putting an instance in a genuinely private subnet while assigning a public IP will not provide the required inbound path.
For production systems, direct interactive access should usually be rare. A common design is to deploy from tested images or an automation pipeline and replace instances rather than manually changing them over SSH. When emergency access is needed, Session Manager provides IAM-controlled, auditable access without making SSH publicly reachable.
If troubleshooting a bastion connection, check the route tables, internet gateway path, security groups on both instances, network ACLs, the bastion’s source and destination settings, and the SSH daemon itself. VPC Flow Logs can show whether traffic is accepted or rejected at the network interface, although they will not show every hop or diagnose an SSH authentication problem.