I'm working on optimizing my Proxmox server setup which has 8 20TB HDDs, 2 2TB enterprise NVMe drives, and 2 SATA SSDs. For optimal performance and to minimize I/O stalls, I'm considering a ZFS configuration with RAID 2 for the HDDs, a mirror for the SLOG on the SSDs, and using the NVMe drives for caching. However, I'm wondering if using them as a cache is the best use, or if I should consider them as a special metadata mirror instead.
1 Answer
I’d suggest you focus less on cache drives and invest in more RAM instead. ZFS really thrives on memory because of its ARC (Adaptive Replacement Cache). Also, just a note: a slog isn't the same thing as a write cache, although it can assist with write operations. If you're still inclined to use SSDs, consider creating separate pools for your NVMe and SATA SSD drives. You might even want to OS install on those SATA drives.
That was my initial setup too! Only the VMs on the NVMe drives are performing well. I'm trying to get more performance by layering everything.