I'm looking to learn how others tackle scalability and performance challenges when they can't use a CDN, whether it's due to frequent updates, compliance issues, geographic restrictions like in China, or dynamic applications. Previously, we've simply added more replicas of our CMS or search services, boosted CPU and memory on the cluster, and deployed system copies across different data centers while trying to manage replication. However, this approach always felt a bit fragile, costly, and like a temporary fix—just a way to patch bottlenecks instead of actually addressing the root problems. So, what methods or architecture designs have you found helpful in achieving scalability and performance without leveraging CDN caching or global distribution? I'd love to hear your experiences, best practices, or even some lessons learned from tough situations.
2 Answers
Leveraging cache systems like Memcache or Redis can be a game-changer. They really help in reducing load times. Also, consider using a reverse proxy, like Varnish, though be mindful that it shares some aspects with CDNs regarding data ownership.
In my experience, built-in caching frameworks work wonders—like basing cache invalidation on the last updated time of records. Placing a load balancer in front of a horizontally scaled setup, with each part having its own caching, can lead to solid performance before even considering a CDN.
I've been doing something similar with AEM for about 14 years. It's effective as long as you don't bypass the cache too often and are okay with limiting to one or two geographic locations. However, it mostly benefits static content.
Right, I have to agree that Varnish feels similar to CDN solutions. Could you elaborate on how you effectively use Memcache in your setup?