I'm planning to set up two servers to host a variety of virtual machines for engineering workstations, VDI, and running CPU and memory-intensive simulations. I've narrowed it down to Proxmox and XCP-ng as potential hypervisors. After quite a bit of research and testing both on a trial setup, I prefer the XCP-ng interface, but I faced challenges while managing Windows VMs, which raises some concerns for me.
Here are the specifics:
- **Servers**: 2 dual AMD EPYC processors (128 cores total), 1.5 TB of RAM, and dual RTX GPUs.
- **VMs**: Between 10 to 50 VMs. The smallest will have 4 cores and 16 GB of RAM, while the largest will be heavy hitters with 128 cores and 700 GB of RAM. The smaller VMs will be for general desktop tasks, while the larger ones will handle complex engineering simulations. I'm using NFS/iSCSI from a dedicated NVMe SSD storage server running TrueNAS for storage.
**What's important to me**:
1. Ease of use and a solid user interface, as my team isn't comfortable with extensive Linux command line work.
2. Performance and stability of Windows VMs, especially for long-running simulations.
3. Reliable storage I/O performance with TrueNAS integration for backups.
4. Support options (both community and enterprise).
I have a few specific questions:
1. Can Proxmox or XCP-ng handle 128 vCPU Windows VMs? I read that Proxmox has no such limitations, whereas XCP-ng might.
2. For CPU-intensive workloads, can I expect performance close to bare-metal on either platform?
3. Which hypervisor generally provides a better Windows guest experience regarding stability, driver support, and performance?
4. How does NFS compare to iSCSI on Proxmox vs. XCP-ng with TrueNAS? What's the stability and performance like under heavy I/O?
5. I've heard XCP-ng is based on an older Xen codebase. For my situations with large VMs and extensive computations, should I worry about potential limitations as I scale?
1 Answer
Hi there! Regarding your questions:
1. Proxmox has no caps on vCPUs, whereas XCP-ng does have some limits for larger VMs.
2. For CPU and RAM loads, XCP-ng does offer excellent performance, we're constantly enhancing the storage solutions too.
3. As for Windows, the guest tools are developed in-house at XCP-ng, which should provide a reliable experience.
4. iSCSI tends to be more popular for heavy workloads among our users; however, we are still finalizing performance improvements.
5. The previous drive limit was challenging, but we're adding support for qcow2 now, so that's no longer a concern.
Yes, with the right hardware, reaching those transfer rates is definitely possible!
Thanks for clarifying! For my needs, I can manage with slower backups, but fast shared storage is crucial. With a good setup (think 50Gbe NICs, Gen 5 SSDs), can I really maximize transfer rates hitting 2GB/s between TrueNAS and the VMs?