I'm building image search for an e-commerce startup. My product embeddings are already stored in Qdrant and were generated locally. I want an uploaded image to pass through an embedding pipeline, then use its vector to search the product collection.
I tried Cloudflare Workers, but the task is too large. Cloud Run timed out before loading the model, and Railway's free tier only provides 0.5 GB of memory, which isn't enough for the model I'm using: qdrant/clip-vit-b-32-vision. I'm looking for a free or very inexpensive way to validate the complete workflow before committing to a paid hosting tier. Ideally, I'd like a serverless option that can handle the model without requiring me to manage a constantly running server.
4 Answers
Before changing providers, make sure the query images are embedded with exactly the same model, weights, preprocessing, and vector dimensions as the products already indexed in Qdrant. Otherwise the search will still return ranked results, but they may be meaningless. Pin the model version, and if you switch models later, re-embed the entire catalog. A useful test is to submit an already-indexed product image as a query; its own product should appear at the top with a very high similarity score.
The model may be smaller than your current runtime suggests. PyTorch and Transformers can consume over 2 GB because of their dependencies, even when the CLIP weights are much smaller. Converting the model to ONNX and quantizing it can bring CLIP ViT-B/32 down to roughly 350 MB of memory, depending on the runtime. Tools such as FastEmbed or ONNX conversion utilities can help. That may fit a small container, though you should still test actual peak memory during startup and inference.
If you use Cloud Run, package the model inside the container image rather than downloading it during startup. Give the service around 2 GB of RAM and consider keeping one minimum instance running to avoid repeated cold starts. The free allowance may cover a single low-traffic instance, but it won’t be completely serverless in the scale-to-zero sense. For a proof of concept, a hosted inference API is probably less work; for predictable production costs, a small warm container or optimized ONNX model may be better.
The simplest option may be to use a hosted embedding service instead of loading CLIP inside your function. Qdrant Cloud Inference can generate an image vector and use it for a search request, so your Worker only handles the upload and API call. Hosted CLIP endpoints from other providers are another option, and at low startup volume the per-image cost may be negligible. Just check whether the specific model is available on the free tier—some models require a paid plan.
That sounds promising. I’ll verify the free-tier model availability first, since the image model may not be included even if the inference feature itself is available.