Hey everyone! I'm looking for insights on how you manage short-lived Docker containers for integration testing in Java applications. I frequently see folks using tools like Jib or Buildx for building Docker images and Helm/Terraform for deployment. But what about actually running these containers during integration tests, like Postgres, Redis, or Elasticsearch, either locally or in CI? Are you utilizing `docker run` in your CI scripts, creating custom bash scripts, or leveraging tools like Testcontainers? I'm really curious about what works best and what might not—especially when it comes to ensuring reliable operation and proper cleanup. Have any of you faced issues with reliability or leftover containers in your Java test flows or CI pipelines? Thanks for sharing!
4 Answers
In our case, we build apps with Maven and integrate Testcontainers during the test phase, specifically the verify phase. We've found a useful workflow using the Docker-in-Docker (DinD) image in our CI runners to run Testcontainers effectively. It’s been a real game-changer for us!
In my setup, I have separate AWS ECR registries for different environments like development and production. I promote software versions by copying artifacts between those repositories. For testing, I package both the app and dependencies using Helm, which makes deployment a breeze. I make it a point to keep my development environments ephemeral, and I use lifecycle policies for cleanup in pre-prod and prod environments. Oh, and although I found Testcontainers useful, I had some trouble since it was primarily for Docker, not Kubernetes—especially after the recent updates that affect using the `/run/docker.sock` hack. I'm keeping an eye on alternatives like TestKube for better solutions.
How do you handle ensuring that a lifecycle policy doesn't accidentally remove an image that is still in use?
I'm also a fan of Docker Compose for CI/CD builds! My script looks something like this:
```
docker compose pull
docker compose start database
docker compose start backend
docker compose start cache
docker compose start nginx
docker compose exec backend run_tests
docker compose down
```
This method is straightforward and works well for managing my testing containers.
I rely on Docker Compose to manage all my dependencies for tests. I've got a GitHub Actions workflow set up that starts everything up neatly and runs tests as part of the process. Health checks in my setup are key for making sure containers come up in the right order, which saves headaches later on. Testcontainers could offer more integration, but I like the flexibility of Docker Compose.
How did you set up your CI runners with DinD specifically?