I run a nonprofit educational website built with Django, Gunicorn, and Nginx behind Cloudflare. Around 1:00 AM on August 8, CPU usage began jumping from roughly 20% to 60–90% in repeating spikes about once per minute. The spikes turned out to be request-driven: Gunicorn workers became busy and MariaDB reached around 300% CPU.
Nearly all of the traffic was simple GET requests to the homepage. The source address changed almost every request, with sampled IPv4 and IPv6 addresses belonging to unrelated residential and mobile networks across many countries. Requests generally had no referrer and used Chrome-on-Windows user agents with varying browser versions. The requests arrived in synchronized waves roughly every 60 seconds. In one sample covering the event, there were about 112,000 requests and 92,000 unique IP addresses, making ordinary per-address rate limiting ineffective.
The homepage was relatively expensive because it performed several database queries, including random selections. Adding a 60-second Django full-page cache immediately stopped the CPU spikes and brought load back below 1. I also added a Cloudflare cache rule for the homepage, and Cloudflare is now serving cache hits so most requests no longer reach the origin.
I'm mainly trying to identify the source rather than urgently stop it. Possibilities include a residential proxy network, compromised home routers or other devices, or some kind of distributed scanning or measurement system. The repeated requests for only the homepage do not resemble normal scraping. Has anyone seen a similar combination of one-shot residential IPs, rotating Chrome user agents, and synchronized waves occurring about once per minute?
4 Answers
We saw something broadly similar at a university site: many residential IPs made fewer than ten requests each and then rotated. Caching was already enabled, and blocking a JA4/TLS fingerprint helped reduce it. We didn’t observe the same clearly timed 60-second waves, though, so the timing in your case may point to a different campaign or controller.
The address data supports a genuinely distributed source. A sample from the period contained about 69,700 requests from nearly 61,000 unique IPs, spanning more than 6,700 ASNs and many countries, including Brazil, the United States, Vietnam, Mexico, France, India, Bangladesh, Argentina, and Pakistan. That distribution is much more consistent with residential proxy infrastructure or compromised consumer devices than with a conventional crawler.
This could be activity from a compromised-device or home-router botnet. Residential addresses are useful for this kind of traffic because they tend to have clean reputations and are spread across many networks, making simple IP blocking ineffective. The synchronized waves suggest centrally coordinated behavior rather than normal browsing.
Residential proxies are another strong possibility. A proxy service can rotate addresses after one or a few requests, which would explain the very high unique-IP count and why each address makes almost no repeat requests. Since the homepage was doing expensive database work, this may have been intended to create origin load rather than collect useful content. Caching was exactly the right mitigation here, especially with Cloudflare serving the response at the edge.
I checked a few of the addresses against an abuse database, but there was little or no useful reporting. That doesn’t rule anything out; the addresses may be ordinary devices or short-lived proxy exits rather than previously reported attack infrastructure.