We've been upgrading one of our older Kubernetes clusters from version 1.19 to 1.31, taking it one step at a time. Everything was smooth sailing until we hit version 1.25, where Helm suddenly refused to upgrade one of our internal charts despite the manifests looking fine. I discovered that the issue was an outdated `policy/v1beta1` PodDisruptionBudget reference that was causing a break in the release metadata since it was removed in v1.25. The solution? I used a Helm plugin called `helm-mapkubeapis` that rewrites old API references in Helm metadata to prevent upgrade issues, even if the chart has been updated. I'm curious to know if anyone else has faced similar problems while managing upgrades across deprecated or removed APIs—what strategies do you use?
5 Answers
If you missed that an API was removed until after the upgrade, you've got to rethink your upgrade strategy. It won’t always be Helm—things change constantly. Having thorough pre-upgrade checks is essential!
Haha, the current version is 1.33! This guy is discussing 1.25, which is already EOL. What’s up with that?
Upgrades can be tricky! We were stuck on Ubuntu 14.04 for ages until our cloud migration.
And just wait until he upgrades—he’ll be wondering why his PodSecurityPolicies stopped working!
We don’t upgrade our clusters; instead, we create fresh ones from scratch in a staging environment and troubleshoot there. Once everything’s ready, we take down the production cluster and promote the staging to production. This approach helps us ensure that all parts of our cluster are in Git and deployed automatically. It took a bit to perfect, but it really simplifies upgrades! Plus, we keep our apps updated regularly, which means fewer breaking changes down the line. Managing multiple apps and stateful data has become a breeze!
Isn't that more complicated? My team maintains a cluster with 55 different independent apps, and we manage to keep it updated without any major hiccups.
How do you manage zero-downtime upgrades with that process?
We always make sure to go through all changes before a Kubernetes version upgrade. Thanks for mentioning the mapkubeapis plugin—it’s super helpful!
You should definitely check out Pluto! It helps identify Kubernetes resources that are deprecated. You can find it at https://github.com/FairwindsOps/pluto. This tool is great for finding deprecated APIs before they become an issue during upgrades.
How does Pluto stack up against Popeye? It feels like there are just too many tools like these out there!
Absolutely! While creating new clusters is fine, you still need to manage your manifests. Pluto helps catch issues proactively, and you can integrate it into your CI/CD pipeline for better visibility on upcoming changes.
It was a genuine mistake, though! Despite our pre-checks, it slipped through. We maintain the Kubernetes JSON schema to validate against it, but an option wasn’t enabled in our CI, which passed all checks!