I'm considering mounting model weights as an image volume instead of downloading and unpacking them with an init container. How does artifact compatibility work, and what should I watch for with rollouts and cold starts?
Has anyone looked at using image volumes for model weights?
Answers
Check the runtime and artifact format before changing the pipeline. Image volume support doesn't necessarily mean the runtime can mount every kind of OCI artifact. Containerd has been reported to support container images for this use, but not arbitrary artifact formats.
If your model package isn't accepted directly, a data-only container image built from scratch is one possible route. Test the actual runtime and registry combination you plan to deploy, since a format conversion step could still mean downloading the full model first.
Reply to this answer
Pin the model reference by digest and update the Pod template when you release a new version. Moving a mutable tag alone won't trigger a Deployment rollout or make existing Pods remount the new contents.
That also keeps the serving image and model version independently selectable without making the deployed model depend on where a tag happens to point.
Reply to this answer
Benchmark the cold start per node, not just per replica. Twenty Pods on a warm node and twenty Pods spread across fresh nodes are very different tests for multi-gigabyte weights.
Try an empty image cache, simultaneous starts and the node cache setup you actually intend to run. I wouldn't assume that a good warm-cache result tells you much about adding new nodes.
_p39~keldruno.6:
Peer-to-peer image caching may be worth testing too, but I'd still include an empty-cache case so the first pull doesn't get hidden.
Reply to this answer
The read-only mount is a nice fit if the serving process can load weights directly from the mounted paths. It can remove the unpack step, while letting you version the serving image separately from the model.
Check the expected directory layout and any startup process that writes alongside the weights before switching. A read-only volume won't be a drop-in replacement for every unpacked directory.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Mavlixo-5r:
That distinction helps. The main thing I want to avoid is removing the init container only to introduce another full download during conversion.