I'm testing a system I control and want to send about 100,000 requests in one minute—roughly 1,667 requests per second—using Python 3 and asyncio. My connection used to provide about 25 Mbps up and down, and I upgraded it to 1 Gbps, but the request rate barely changed. I suspect the bottleneck may be my ISP, router, operating system, or the way the client and server are configured. What should I measure and change to identify the limiting factor?
3 Answers
At 100,000 requests per minute, you need to sustain about 1,667 requests per second, or one request every 0.6 milliseconds on average. A single Python process and a single network path may not handle that reliably. Reuse persistent connections, check file-descriptor and ephemeral-port limits, watch for sockets stuck in TIME_WAIT, and profile the event loop and CPU. You may need multiple workers or load-generation machines.
Raw internet bandwidth may not be the bottleneck. The payload size, protocol, connection reuse, destination server, latency, and server-side rate limits all matter. Include the actual request code and measure total bytes, connection setup time, response time, CPU usage, and error rates before assuming the ISP is responsible.
Start by eliminating variables. If the server is local, benchmark it locally first. If it runs in the cloud, run the client from another suitable cloud instance and compare the results. That separates application and server limits from your home router and connection.
The test should also make clear whether the destination is a service you own or have explicit permission to load-test. Otherwise, that volume could look like a denial-of-service attack.