I'm looking for backups that protect against accidental deletion or corruption after an update, not just storage redundancy. Database systems often provide export or backup tools, but I'm unsure how to handle a custom application that uses SQLite along with static files.
For this kind of app, a consistent backup would need an online SQLite backup plus a copy of the related files. In Kubernetes, should this run as a sidecar container sharing the application's PVC, a scheduled Job or CronJob, or through a backup operator? The result would need to be uploaded to remote storage, such as an S3-compatible service. I'd also like to avoid depending on a particular host path as the cluster grows and storage moves to something like Longhorn.
4 Answers
A Kubernetes backup operator can handle this without writing all the scheduling and upload logic yourself. For example, K8up can run a Job that mounts a PVC and backs it up with restic, or execute an application-provided command and capture its output. That can be useful for custom applications, although SQLite still needs an application-aware export if you want a coordinated database backup.
There are two different cases: general volume backups and application-aware backups. Tools such as Velero can handle broader Kubernetes resources and volume snapshots, while databases often need their own backup tools. For SQLite, you’ll likely need an application-specific script that uses SQLite’s online backup mechanism and then copies the associated files.
If possible, simplify the application architecture by putting persistent state in managed PostgreSQL or MySQL and storing files in object storage. Then you can use the database’s native backup process and the object store’s replication or versioning. That isn’t practical for every self-hosted application, though, so a PVC backup or sidecar export is still useful for apps that only support local SQLite and filesystem storage.
For a small SQLite application, a sidecar sharing the application’s PVC is a reasonable approach. Have it run a scheduler or backup process, create the SQLite backup, archive the required static files, and upload the completed result to remote storage. This avoids relying on whichever node currently hosts the application.
A separate scheduled Job can also work, but PVC access modes matter. ReadWriteOnce generally allows access from one node, while ReadWriteOncePod restricts the volume to one pod, which can prevent a separate backup pod from mounting it. A sidecar avoids that issue.
Be careful about consistency: an SQLite backup and a file archive may represent different points in time if the application changes both at once. Temporarily pause writes or use an application export that coordinates the database and files. The local copy can be made consistent before the upload finishes. Prevent overlapping runs, only report success after the remote upload completes, and alert when the last successful backup becomes too old.
Most importantly, regularly restore a backup into a fresh PVC and test instance. Confirm that the database and referenced files work together, rather than merely checking that the archive can be extracted.
That makes sense. My main uncertainty is where the application-aware script should run in Kubernetes. Previously I ran it from the host with access to bind-mounted data, but I’d rather let the PVC and its storage system manage placement as the cluster grows.