Do I need to turn off DHCP and DNS on my router when using Windows Server for DHCP and DNS on a Windows 11 VM?

Sure, you can have both running, but it's tricky! Configure the Windows Server's DHCP properly to avoid overlapping ranges with your router. Ideally, sticking to one DHCP server is safer. But with careful configuration, you might manage both. Just be aware that devices will pick up whichever DHCP server responds first.

You can't have two DHCP servers on the same subnet as they'll compete for requests, which can lead to issues like IP address conflicts. If you're going to use the Windows Server for DHCP, it's best to turn off DHCP on your router to avoid problems. However, you can definitely have multiple DNS servers running without issues since devices will resolve to the DNS server specified by DHCP, or whatever is set manually on clients.

If you're running ESXi, consider creating a separate virtual network for your Windows Server. This way, you can set up DHCP and DNS on that isolated network without interfering with your router. Just make sure to route traffic properly between your VMs and the outside world with your Pfsense configuration.

You can technically run both DHCP servers, but I really wouldn't recommend it. You might end up with duplicate IPs if both look to assign addresses at the same time. On the other hand, having multiple DNS servers is totally fine and even beneficial for redundancy.