Hey everyone, I'm working on a small office/home-office network project and could use a sanity check on my design, which might be a bit over the top. Here's what I want to achieve: I need to use multiple ISPs with strict policy-based routing so that two work PCs consistently connect via different ISPs. I'm looking to separate office Wi-Fi, servers, CCTV, and IoT devices, plus ensure that my CCTV cameras have no internet access. I also want remote access through a VPN without exposing any services. This setup is all about reliability, predictability, and clean separation, rather than anonymity or getting around any rules.
For hardware, I've planned to use OPNsense as my firewall/router, alongside a TP-Link JetStream managed switch and Omada APs. My servers include a Proxmox host for VMs, a mini PC for WordPress sites, and a two-NIC NVR for CCTV. I have three fiber ISPs that connect through OPNsense, with a structured VLAN design aiming for clear separation of traffic. I'm worried, though—am I overengineering this whole thing? Are there any common pitfalls with multi-WAN and strict policy routing in OPNsense? Is the two-NIC design for the NVR feasible long-term? Would you suggest simplifying anything while keeping that isolation intact? I'm comfortable with OPNsense administration but want to avoid a fragile setup that may fail unexpectedly. I'd appreciate any feedback, particularly from people who've run similar setups with multi-WAN and OPNsense or comparable environments.
5 Answers
I've worked with setups like yours using pfSense (which OPNsense is based on), and if you know your way around it, you're in a good spot. Just be cautious with your switch settings since CARP can cause some issues if the switch isn't set up to handle virtual MAC addresses properly. I'd recommend running CARP on a separate interface to prevent any hiccups. On the other hand, if you're noticing the complexity outweighing the benefits, a simpler SMB router might actually be more reliable.
Honestly, it seems like you might be going a bit overboard with three ISPs and such a complex setup. Unless you have a very unique situation that warrants this kind of design, you could likely simplify things without sacrificing too much. For example, using just two ISPs and setting up failover between them could offer better reliability without all the complications that come with policy routing and managing multiple connections for each PC.
Totally agree! It might make things more manageable in the long run. You can still achieve high availability without the headaches.
I’m curious why you need to keep those two work PCs on different ISPs all the time. It sounds like it might be more beneficial to set them up for failover instead. That way, if one ISP fails, you won’t lose connectivity, plus it sounds easier to manage!
Yeah, I thought about that too! Keeping them off the same ISP could mean more stability, but I wanted that separation for specific reasons.
Make sure your VLAN setup is streamlined—keeping them isolated is good, but too many intricacies can lead to management issues later! Keep it clean and straightforward, especially with access controls.
Having that many fibers could be risky since if one goes down, you might lose redundancy if they’re from the same infrastructure. Consider mixing in a different type, like a cable or cellular backup. That way, you won’t be totally out of luck if there's an issue.
That's a solid point! Keeping things straightforward can often lead to a more stable environment.