I'm building a highly available homelab to learn more about on-prem infrastructure. The setup is a three-node Proxmox cluster, with each node running VMs for DNS, HAProxy, a Patroni database node, and Apache web servers. HAProxy handles load balancing, reverse proxying, TLS termination, and configuration deployment through infrastructure-as-code as web servers are added or removed.
The web servers need to serve a collection of images. I originally stored them on a NAS, but that created a single point of failure. I later moved the storage to Ceph for high availability, although performance is noticeably limited because the nodes are connected over 2.5 Gbit/s networking.
I'd like to experiment with image caching, both to improve read performance and to learn which approaches are practical. The options I'm considering are:
- Using HAProxy's built-in cache directly
- Adding a caching proxy such as Varnish or Nginx between HAProxy and Apache
- Using Apache's mod_cache or a filesystem cache on the shared storage
For anyone with experience running these in production or homelabs, which approach would you recommend? Are there important limitations or design considerations, especially around cache size, disk storage, cache invalidation, and keeping multiple cache nodes consistent?
1 Answer
HAProxy does include a built-in cache using its cache configuration and HTTP request/response rules. It stores data in RAM, and each HAProxy instance maintains its own cache, so it’s best suited to small, frequently accessed objects such as thumbnails, icons, or CSS files. It probably isn’t the right choice for a large image library.
For larger objects, Nginx with proxy_cache or Varnish with a disk-backed cache are more typical choices. Keep in mind that caching only improves speed; if the cache misses, the request still has to reach the origin storage. If availability is the main objective, consider replicating the images onto local disk on each web VM or using distributed object storage. Then caching becomes a performance optimization instead of something required for the service to remain available.
That makes sense. I’ve already moved the images from the NAS to Ceph, so the storage layer is redundant even though it’s slower than I’d like. I’ve started experimenting with Varnish and it’s been a useful learning exercise.