I'm planning to upgrade a production Kubernetes cluster from v1.34.8 to v1.35, and Elasticsearch is currently running in the cluster. I'd like to understand how to verify compatibility and reduce the risk of downtime or data loss.
What should I check regarding Elasticsearch, the ECK/operator version, StatefulSets, storage classes and CSI drivers, ingress, CRDs, deprecated APIs, autoscaling, and other dependencies? Should I upgrade ECK or Elasticsearch before Kubernetes, or the other way around? Is it safer to upgrade the control plane and worker nodes separately? I'm also looking for recommendations on testing, snapshots, rollback planning, deprecated-API detection, and a production upgrade sequence. Any experience with a similar upgrade would be helpful.
4 Answers
Take verified snapshots before starting, confirm that they can be listed and restored, and record the current versions and configuration. Review pod disruption budgets, anti-affinity or topology spread rules, resource limits, persistent-volume behavior, CSI driver support, ingress, admission controllers, monitoring, and autoscaling. Useful preflight checks include API deprecation scanners, server-side dry runs, cluster health checks, operator logs, and a workload inventory. Make sure you have a maintenance window, clear stop conditions, and a tested way to recreate the cluster if a rollback is needed.
Start by checking the ECK/operator compatibility matrix and upgrade to a release that supports both your current Kubernetes version and 1.35. Also review Kubernetes API changes for removed or deprecated APIs and confirm that your CRDs are current. A sensible sequence is usually to update the operator first, validate Elasticsearch health, and then upgrade Kubernetes. Check autoscaler compatibility afterward if it recommends matching the Kubernetes version.
During the node upgrade, Elasticsearch can usually remain available if the cluster has enough master, hot, and warm nodes and its disruption budgets are working correctly. Drain old worker nodes one at a time or in carefully controlled batches so the scheduler can move pods while Elasticsearch remains healthy. Wait for shard recovery and cluster health before proceeding to the next node. Upgrade the control plane and worker pools as separate stages, and use a canary worker pool when possible.
The safest approach is to test the upgrade on a separate cluster that closely matches production. Restore an Elasticsearch snapshot there, run the same workloads, and verify indexing, search, shard allocation, storage, ingress, and node recovery. Kubernetes APIs such as StatefulSets and most CSI integrations are generally stable, but the test environment should also include your actual operator, autoscaler, ingress controller, and storage drivers.
For rollback, don’t rely on simply downgrading Kubernetes. Keep tested Elasticsearch snapshots in durable object storage and verify that you can restore them into a clean cluster. That gives you a practical recovery path if the upgrade damages the environment.