Suppose a codebase currently handles about 1 million daily users and 300,000 concurrent users comfortably. If the expected load increases to 10 million daily users and 3 million concurrent users, would teams normally rely on cloud autoscaling and horizontal scaling, or would they usually need to modify the application code and architecture as well?
3 Answers
Scaling can mean either vertical scaling—using larger machines—or horizontal scaling by adding more machines. Horizontal scaling is often more resilient, but it may require redesigning parts of the system so work can be split across instances. Increasing traffic by ten times commonly exposes limits in connection pools, slow queries, caching, storage, and partitioning, so code and architecture changes are often necessary.
There isn't a universal answer. It depends on the workload, traffic patterns, latency requirements, data access, and which component becomes the bottleneck. Autoscaling can add capacity, but it cannot automatically fix inefficient queries, shared state, limited database throughput, or an architecture that cannot run across multiple instances.
First, determine whether the application is suitable for horizontal scaling. Stateless application servers behind a load balancer can often be replicated fairly easily, but the database, cache, file storage, queues, and external services may need separate scaling strategies. The right approach is usually a combination of autoscaling, capacity planning, monitoring, and load testing.
Exactly. Adding more application servers may only move the bottleneck to the database or cache. Autoscaling works best when the app tier is stateless, but teams may still need changes such as query optimization, sharding, better connection management, or asynchronous processing.