I'm setting up a new Kubernetes cluster and I want to plan ahead for backups and disaster recovery. I'm considering using a CSI just for VolumeSnapshots, which could be stored on something simple like NFS on a separate backup server. However, I'm wondering if, in the event that the entire cluster is deleted and rebuilt, I can still access these VolumeSnapshots. My knowledge is pretty limited—I just know they're linked to specific CSI drivers. But what happens if the CSI driver, the cluster, and etcd are all wiped out and then restored using Terraform and ArgoCD?
1 Answer
To make DR work, you need to have your backup outside of the cluster's infrastructure. If everything is deleted, including the cluster, you won't be able to access those VolumeSnapshots unless they're stored by something that isn't affected by the cluster lifecycle. Depending on the CSI implementation, some might store backups externally with a deletion protection feature, but that really depends on the CSI driver you're using.
Yeah, I've changed my mind about that too. I realized that you need to back up the etcd objects along with the VolumeSnapshot handles. I’m now looking into Velero as a solid option for disaster recovery.