I'm learning Kubernetes locally with kind while building a microservices project. The application has five Node.js/Express services, and each service has its own PostgreSQL database managed by a StatefulSet. I'm using Drizzle ORM and already have manifests, ConfigMaps, and Secrets for the services and databases.
I'm unsure how migrations should be applied inside the cluster. The approaches I've found are running them in an init container, using a standalone Kubernetes Job, or executing them from a CI/CD pipeline. For local development, should I use one of these approaches differently than I would in production? If each service has its own database, should every service have a separate migration Job? I'd also appreciate advice on how to make the setup reliable, avoid migration races when scaling replicas, and ensure applications don't start before their schemas are ready.
5 Answers
Each service should normally own and run migrations for its own database. You could create one Job per service, with the correct database credentials and migration image, rather than having one process manage all five databases. That keeps ownership clear and lets you deploy services independently. If the databases are tightly coupled, a coordinating workflow is possible, but separate migration Jobs are easier to reason about and retry.
A standalone Kubernetes Job is generally safer for migrations than putting the migration command directly in an init container. An init container runs once for every Pod start, so multiple replicas can try to migrate concurrently, and the command runs again after restarts or rescheduling. A Job has a single completion state and can be retried or inspected independently. Make the migrations idempotent and ensure the migration tool tracks which migrations have already been applied.
Running migrations automatically on every application startup can work for a single replica and a well-designed migration tool, but it becomes risky with multiple replicas or long-running changes. Startup races, database locks, failed rollouts, and incompatible schema changes can make failures harder to diagnose. For learning locally it’s fine to experiment with an init container, but a separate Job or CI/CD migration step is a better production habit. Keep rollback or recovery procedures documented, since database migrations are not always safely reversible.
For production, the cleanest pattern is often to run migrations as a controlled deployment step in CI/CD before publishing or rolling out the new application version. The migration should be backward-compatible with both the old and new application versions, especially during a rolling update. This avoids having every application replica responsible for schema changes. A Kubernetes Job can still be the mechanism used by the pipeline, while Helm hooks can help attach it to a release if that fits your deployment process.
The important part is keeping migrations non-breaking. Add new columns or tables first, deploy code that can use them, and remove old schema elements only in a later release.
An init container can still be useful as a guard, but it should not usually perform the migration itself. It can wait until the relevant migration Job has completed before allowing the application container to start. In practice, many teams instead let the deployment pipeline enforce the ordering, because a Pod-level wait needs careful handling of Job status, permissions, retries, and failed migrations. Don’t let the application silently start against an unknown or partially migrated schema.
That makes sense. I’ll use a separate Job and have the application wait for it to complete before starting.