I have three physical Proxmox servers running a K3s cluster. Each server has a control-plane VM, and servers two and three also run worker VMs. I want to deploy Longhorn with three volume replicas so that each replica is stored on a different physical server. That way, losing one complete Proxmox host will not take down the data. I am confused about how Longhorn schedules its components and replicas, especially because one physical server currently has only a control-plane VM. I have tried node labels and tolerations, but some Longhorn UI and driver pods remain stuck or enter an error state. What is the correct topology and configuration for this setup?
4 Answers
Longhorn system components and volume replicas are separate concerns. You can use node selectors, taints, and tolerations to control where managers, UI, CSI drivers, and other Longhorn pods run. For replica placement, label nodes by physical host and disable soft replica anti-affinity if you want a hard requirement that replicas cannot share a node: `defaultSettings.replicaSoftAntiAffinity=false`. Also configure the StorageClass with three replicas. Check the pod events and Longhorn logs rather than relying only on the pod status when the UI or driver is failing.
For production, I would avoid depending on a control-plane-only VM as a storage node unless you deliberately tolerate its taint and have verified the disk setup. The cleaner design is to run at least one worker/storage node on every physical server, then schedule Longhorn storage across those three workers. If one host has no worker or Longhorn disk, Longhorn cannot guarantee one replica per physical host, regardless of topology labels.
Treat the physical servers as failure domains, not the individual VMs. Label the Kubernetes nodes according to the host they run on, for example `topology.kubernetes.io/zone=server-1`, `server-2`, and `server-3`. Then configure the Longhorn StorageClass with `numberOfReplicas: 3` and enable replica zone awareness or replica auto-balancing. Longhorn can then place one replica in each zone when suitable storage is available. Make sure each physical host actually has a Longhorn disk exposed to a Kubernetes node; labels alone do not provide storage.
You can reduce the default replica count if your workload accepts the risk, but three replicas are appropriate when the goal is surviving a complete host failure. Keep in mind that Longhorn replication is not a backup: accidental deletion, corruption, or an application-level problem can be copied to every replica. Configure recurring backups to storage outside the three Proxmox servers as well.

Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures