I've noticed that it's really easy to accidentally commit a faulty YAML file when working with Custom Resource Definitions (CRDs). These files can pass YAML validation but still cause issues when deployed, like adding a field that's not recognized (for example, "oldname" in a certificate resource). I'm curious about the methods you all use to validate your CRDs before pushing them to your GitOps tooling. Are there tools or processes that you find particularly effective?
6 Answers
I'm actually building a Dagger module that runs in a GitHub workflow. It creates a kind cluster with the specific Kubernetes version to validate the CRDs against it. It's super powerful and ensures our custom resources can be built correctly from them.
On my end, I rely on Terraform. It validates the configs and if there’s anything missed, it’ll usually crash during the plan phase!
We keep all our CRDs as OpenAPI Schemas in a Git repo. When a pull request comes in, we scan all the YAMLs using kubeconform for validation. We also run other checks like kustomize-fix and prettier to keep everything tidy.
Before I deploy, I use kubeconform, kubelint, and kubectl dry-run. I find that's essential, but honestly, the only way to catch all errors is with a dry-run in a cluster.
One thing I do is test everything in non-production clusters. It's a repetitive process, but I find it helps catch issues before they go live. Just a back-and-forth between clusters to ensure everything runs smoothly!
Yeah, I totally get that! It does seem like a lot of work just to validate things manually.
We manage this by having multiple test environments set up. It can be a bit slow with the back-and-forth between clusters, but it ensures that everything is validated correctly before deployment.
But wouldn’t that approach take longer than just validating the individual resources?
That's a neat approach! So you’re using GitHub Actions to run these validations, right? But does that only work for PRs and not direct commits?