I'm evaluating a custom, provably diverse network between two AWS Regions. The proposed path is: Region 1 → Dedicated Direct Connect → cross-connect → third-party long-haul fiber → cross-connect → Dedicated Direct Connect → Region 2.
The main question is whether both Direct Connect endpoints can be connected through a transparent Layer 2 service, with BGP and routing configured through AWS, or whether I need customer-owned routers at the two ends to terminate the BGP sessions and route traffic between the regions.
The goal is to avoid depending on the AWS backbone for this inter-region path and to provide better control over latency, carrier diversity, routing policy, and integration with non-AWS infrastructure. The AWS-hosted endpoints are required; the transport between them is what we want to customize. Has anyone implemented or validated a design like this?
3 Answers
For a robust implementation, terminate each Direct Connect circuit on redundant routers in the respective colocation facilities, then connect those routers over one or more diverse private circuits. Confirm that the carriers preserve the required VLAN tags and MTU end to end, and avoid assuming that a service advertised as 9000-byte capable supports every frame size you need. MACsec on Direct Connect protects the customer-to-AWS hop; it does not automatically protect the intervening long-haul service, so encrypt that segment separately if required.
Pay close attention to route policy and prefix limits. Explicitly filter and summarize advertisements in both directions rather than blindly re-advertising every network from one Region into the other. Also document the physical route and carrier paths, use separate facilities or waves where possible, and validate that the AWS-side terminations are genuinely diverse. AWS features that provide inter-Region connectivity may still use the AWS backbone, so they do not meet a requirement to avoid that dependency.
The physical fiber path may be possible, but the practical design needs a customer router or router pair at each end. Direct Connect is not a transparent Layer 2 extension between Regions; each private virtual interface uses BGP and terminates on a customer-side device. The routers can establish eBGP sessions with AWS locally and use iBGP or an IGP across the private long-haul connection. That gives you control over route filtering, encryption, monitoring, failover, and traffic engineering.
A transparent Layer 2 design might look plausible on paper, but it is not a supported or operationally useful model. There would be no customer control plane between the AWS endpoints, and troubleshooting would be difficult if the service failed or VLAN handling changed.