I'm trying to cut down on hardware overhead by migrating all my Kubernetes storage to an external Ceph cluster in Proxmox. I would love to get some guidance on this setup.
Currently, my Kubernetes nodes are virtualized on Proxmox, utilizing physical disk passthrough to provide persistent storage through Longhorn.
My aim is to leverage the Proxmox Ceph (Squid) cluster to supply storage for all my Kubernetes clusters while still retaining a Longhorn-like experience, including features like GUI management, snapshots, and backups/restores.
From what I gather, Ceph Rook should be able to deliver features such as RWO, RWX, S3 support, snapshotting, backup/restore capabilities, performance statistics, and a GUI while working with an external Ceph cluster like mine. This would involve setting up a storage pool for each storage type for every Kubernetes cluster.
Is this a sensible approach, or am I missing something important? I really appreciate any insights you might have!
1 Answer
You might want to consider running a hyper-converged Ceph setup alongside Proxmox, and opt for Ceph-CSI in your Kubernetes cluster for managing storage. Rook might not be necessary in your case.
I've also thought about just using Ceph-CSI, but I heard it doesn’t support S3 on its own unless integrated with something else, plus it lacks a dashboard. A dashboard would be nice for a quick overview across clusters!