I'm migrating my homelab from Docker Compose and Ansible to k3s with Flux, and I'm struggling with applications that expect a whole configuration directory rather than a few environment variables.
With Compose, I can mount a directory directly:
volumes:
- ./grafana/provisioning:/etc/grafana/provisioning
That directory contains several subdirectories and dozens of files, including dashboard JSON files, datasource YAML, alerting configuration, and provisioning settings. With Ansible, the files lived in Git as templates, were rendered onto the host, and then mounted into the container. It was straightforward and fully managed as code.
In Kubernetes, the obvious choices seem imperfect. ConfigMaps work for a few files, but putting 20–30 dashboard files into YAML feels unwieldy, and I'm not sure how to preserve the directory structure cleanly. A PVC would provide a filesystem, but it seems more appropriate for persistent data than version-controlled configuration, and I don't want the configuration to become an opaque volume on a node.
What is the usual Kubernetes approach for this kind of nested, multi-file configuration? Is the practical answer to use several ConfigMaps, or is there a better pattern for keeping the directory tree readable, declarative, and managed through GitOps?
4 Answers
For a setup like this, multiple ConfigMaps are normal. Generate one per subdirectory or leaf directory and mount them at the paths the application expects. It is more verbose than a single bind mount, but the files remain in Git and the deployment stays declarative.
Be aware of the roughly 1 MiB object-size limit. If the dashboards become too large, split them across more ConfigMaps, bake relatively static files into a custom image, or use an init container to assemble files into an emptyDir volume.
Another workable pattern is an init container that copies or renders the configuration into a shared emptyDir volume before the main container starts. This is useful when the application requires one complete directory tree or when combining several sources is easier than mounting many ConfigMaps.
The init container can consume generated ConfigMaps, Secrets, or a packaged configuration image. It keeps the final filesystem layout familiar without treating a PVC as configuration storage, although it adds more moving parts than direct ConfigMap mounts.
If the question is specifically about Grafana dashboards, Grafana’s Kubernetes integration can watch labeled ConfigMaps and load dashboards automatically, so you may not need to mount every dashboard into the main provisioning tree. Kustomize can still generate those ConfigMaps from files in Git.
For other applications, the general rule is still ConfigMaps and Secrets for configuration, PVCs for persistent application data, and custom images when a large set of mostly immutable files is really part of the application package.
There are also application-specific options such as dashboard CRDs or Git synchronization, but those solve the Grafana case rather than the broader problem of arbitrary nested configuration directories.
The common solution is Kustomize’s configMapGenerator. Keep the real files in Git in whatever directory structure makes sense, then have Kustomize build ConfigMaps from those files during deployment. That avoids embedding large JSON and YAML documents manually inside another YAML file and preserves the original file contents without escaping or indentation problems.
A typical layout is one generated ConfigMap per logical directory, such as dashboards, datasources, and alerting. Mount each one at its corresponding directory in the container. Flux can then apply the generated resources and add a content hash so workloads roll when the files change.
That also keeps the source files easy to edit and review. The generated ConfigMap may look noisy, but it doesn’t need to be the format humans maintain.
My main concern was having to paste every file into YAML and deal with escaping. Generating the ConfigMaps from the source files sounds much more manageable than hand-writing them.