I'm dealing with a pretty massive Docker image situation here. We have a base image that's about 6 GB and then specialized images that range from 7 GB to 9 GB. Most users work with the development image (9 GB) since they need to develop their own applications that run with the main application. The big issue is that every time there's a minor change, like an update to system runtime tooling, users have to download the entire 9 GB image again. For instance, we recently had to republish an apt package for a tool due to a path change, which means everyone had to download the image again just to access that small update. With changes happening regularly, it's extremely inefficient to keep downloading the whole image, especially for small updates. I'm wondering if there's a way to mitigate this or if users can perform an 'apt upgrade' to just get the necessary updates without having to download the entire image anew?
4 Answers
You might want to consider using Nix for package management, which can simplify this process a lot. It allows for declarative approach to package handling, making it easier to manage dependencies without the bloat.
Docker was initially designed to not rebuild the image at runtime. The principle is to build images in layers and keep the frequently changing layers separate from the static ones. This way, when users do a `docker pull`, only the modified layers are downloaded. Alternatively, you could set up your image to run an `apt update` at the start. This method is used by some developers, but it feels a bit hacky and isn't officially endorsed yet, mainly because it doesn't fully address the image size issue.
One effective approach is to treat the large image as a base image. You could create a smaller overlay image that strictly runs `apt upgrade`. This way, you control when the base image changes, and you keep the updates lightweight and fast. Your developers could pull the new overlay while still using the old one, minimizing disruption. Although this doesn't speed up base image updates, it keeps the overall download size down effectively.
Honestly, unless you can split your image into smaller components, the options are limited. Docker suggests looking into multi-stage builds for size, but with the complexity you're describing, that might not help much. Also, 9 GB is hefty; have you really exhausted all the possibilities for size reduction? There are many guides on this kind of optimization available online.
We follow best practices and multi-stage builds rigorously. Each part of our toolchain is built in its own pipeline, and we publish an apt versioning to our repository. The base image is quite simple, just creating a user and installing necessary packages—so sadly not much room for optimization!
Even if you layer installs from least to most volatile, all layers after a changed layer become invalidated, which doesn't really solve the size problem. For example, if one library updates, it still results in several GB of data that need to be downloaded again. Some developers have begun using scripts that update specific packages, letting apt handle dependencies, but it's not a clean solution.