I'm rebuilding the networking on my Docker host, LeopARRd, and would like advice on a maintainable design. The host has addresses on VLAN-10 and VLAN-30. Every service should be reachable from the local network through one of the host's IP addresses. I also run two independent Compose projects, using separate compose.yaml files, and containers in either project should be able to reach services in the other by name, such as http://navidrome, instead of relying on fixed container IPs.
Some containers must use 192.168.30.1, a VPN gateway on 192.168.30.0/24, for Internet access, while others should use the regular gateway at 192.168.10.1. My current setup creates Docker bridge networks for the different VLAN or egress groups and uses host policy routing plus separate masquerading rules. It mostly works, but boot ordering is unreliable: Docker networks and bridges may not exist when the routing rules are restored, so routes referring to bridges such as br-vlan32 fail. The masquerading rules themselves persist correctly.
I'm wondering whether this is unnecessarily complicated and would prefer to redesign it. What architecture would provide shared service-name DNS between both Compose projects, separate egress paths, restart-safe routing, and minimal custom routing or firewall maintenance? Ideally, most of the network configuration would remain in the Compose files.
2 Answers
If the networks must survive independently of either Compose project, create them before deploying the stacks and mark them as external in both files. For example, maintain a shared network for cross-stack traffic plus separate networks such as normal-egress and vpn-egress. A small provisioning script, systemd unit ordered after Docker, or configuration management tool can create or verify those networks before Compose starts.
This is less self-contained than defining everything inline, but it avoids having Compose recreate networks unpredictably. Health checks and service dependencies can control application startup, though they do not replace network initialization or host routing. The policy-routing tables and rules should likewise be managed by a persistent host service that waits for Docker’s bridge interfaces, or replaced with a dedicated routing component so the host does not depend on bridges that may not yet exist.
Create one shared user-defined Docker network for service discovery, and attach containers to additional networks according to their required egress path. The shared network can be external and referenced by both Compose projects, so Docker’s embedded DNS lets containers resolve each other by service name. Keep the normal and VPN egress networks separate, and attach each service only to the networks it needs.
For incoming access, prefer publishing ports on the host and binding them to the appropriate host IP rather than giving every container direct VLAN access. That keeps the container networks simpler and makes the host firewall the single place controlling exposure. The remaining limitation is that selecting different upstream gateways is not something Compose’s network definitions solve by themselves; host policy routing, a dedicated router container, or a VPN gateway appliance is still required.
That makes sense for the shared DNS network, but I’m still unclear about the egress networks. How would you connect the ordinary and VPN paths without making the routing fragile at boot?