Anyone using OCI registries to distribute large ML models?

November 17, 2026 5 answers
Asked by 9t!ferrulepuzzle_628

I'm considering packaging model weights as OCI artifacts and pulling them by digest instead of fetching them from object storage at startup. How do people handle large layers, registry limits, and caching on GPU nodes? I'm interested in the trade-offs between init containers, CSI drivers, and image volumes.

Answers

_x81~pivrel.6

A node-local cache exposed through a CSI driver is worth considering for large weights. The driver can pre-pull the artifact onto a shared node volume, and pods can mount it rather than downloading the same model on every restart.

An init container is simpler to set up, but whether it pulls again depends on how you arrange its storage and cache. For a model in the tens of gigabytes, that difference can dominate startup time.

_w45!ferrymosaic.8:

Cache cleanup matters too. A node holding several model versions can fill its disk quickly, so plan how old digests get removed.

Reply to this answer
_v45!cobaltspoke.8

Split the weights into multiple layers rather than making one enormous blob. Shard-sized layers can be uploaded and retried independently, and pulls may run in parallel. If only some files change between model versions, unchanged layers may be reused.

Check download concurrency on the GPU nodes, but tune it against registry and network capacity rather than just cranking it up. Compression is also worth testing. Weight files may not shrink much, while decompression adds CPU work to the pull.

Reply to this answer
Jelvoro.6p

Test your exact registry and client with the actual model sizes before committing to the design. Large pushes and pulls can hit timeout limits, and raising client timeouts may be necessary. A successful small artifact test doesn't tell you much about a tens-of-gigabytes transfer.

Also check how the registry handles retention and garbage collection so the digests you deploy remain available.

Reply to this answer
ZORVANI-8c

Image volumes sound convenient, but check support for non-image OCI artifacts in your particular runtime and Kubernetes setup. Support isn't consistent. Pulling and unpacking into a volume with an init container or CSI driver may be the more portable option for now.

Reply to this answer
7x+drumgauge_491

The attraction is using the registry workflow for models as well as images. Digest references give you a specific artifact to deploy, and you may be able to reuse signing, replication, and admission policy tooling. Verify that those tools handle your artifact type rather than assuming image support carries over.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions