How can I effectively build and push Docker containers using AWS CodeBuild?

You could set up a dedicated ‘builder’ image that incorporates all your required tools and dependencies. Just ensure that this image is defined in your PipelineProject’s build environment so you can push it to Amazon ECR and reference it. It might be a game changer for you!

Check out the AWS documentation; they have several options listed, including Docker-in-Docker (DinD). The common concern about DinD usually applies when you control the host, but in CodeBuild's case, you don’t really have that control, so it can work in this environment.

Consider using a Custom Image, preferably from Amazon ECR as opposed to an external Docker registry. If you go the route of building a Docker image inside a Docker container in CodeBuild, make sure to enable the Privileged flag for the step to allow elevated permissions.

There's absolutely nothing wrong with using Docker-in-Docker in CodeBuild. Just keep an eye on performance, as using a custom image might be slower to load compared to the standard CodeBuild images which are cached. By the way, I moved to GitHub Actions which I find to be much better for caching and overall features.

We like to create ephemeral image builder container recipes that we then use to generate Docker images based on the output from the build process.