I'm building a small Kubernetes homelab with three nodes, each acting as both a control-plane and worker node. Each node has only one NIC, and I'm considering configuring the switch ports as VLAN trunks so the cluster can use separate DMZ and LAN VLANs.
The goal is to run two service load-balancer address pools through Cilium or MetalLB: one for the DMZ and another for the LAN. That way, workloads can be exposed through the appropriate gateway based on their intended level of exposure.
I'm mainly interested in improving isolation and security, but I'm unsure whether this is unnecessarily complex. Could Cilium network policies provide the same protection inside the cluster, or do separate VLANs make sense because they also control where the load-balancer IPs exist on the network? I'd also like to bootstrap the nodes with iPXE, so I'm wondering how that would work when the switch link carries tagged VLAN traffic.
3 Answers
If all the nodes are on one VLAN, Cilium network policies can give you fairly granular control over traffic between workloads. That may be enough for a simpler homelab, especially if the main concern is pod communication.
However, policies inside Kubernetes don’t replace network separation for externally exposed services. If the DMZ and LAN addresses need different routing, firewall treatment, or trust levels, putting them on separate VLANs gives you a clearer boundary. The choice mostly depends on whether you want isolation only inside Kubernetes or also at the physical and routing layers.
Another option is to use Cilium’s BGP integration if your router supports BGP. Cilium can advertise service IP ranges from the nodes, and DNS can point service names directly at those advertised addresses. Depending on your ingress and Gateway API design, this may let you avoid some dedicated external load-balancer components and simplify the traffic path.
It’s a different architecture rather than a replacement for VLAN isolation, though. BGP can advertise the routes, while VLANs and firewall policies still provide the network boundaries you want.
This isn’t necessarily overengineering. Network policies are useful for controlling pod-to-pod traffic inside the cluster, but they don’t determine which external network a load-balancer IP belongs to. If you want LAN and DMZ services to remain genuinely separate, using separate VLANs is a reasonable design.
For example, Cilium LB-IPAM can use separate address pools, with labels selecting which pool a service receives an address from. Your firewall or router can then control traffic between the VLANs. Hubble could help you observe traffic inside the cluster, while the firewall remains the place to enforce and monitor inter-VLAN routing.
For iPXE, a common setup is to leave the provisioning or boot network untagged on the switch port and carry the additional VLAN as tagged traffic. The node’s VLAN configuration needs to be available early in the boot process, so make sure it is applied consistently during installation and every subsequent boot.
That makes sense. I was thinking of using Cilium Hubble for visibility inside the cluster and relying on my OPNsense firewall for the VLAN gateways and inter-VLAN rules.