Hey everyone, I've been getting overwhelmed with traffic on one of my smaller websites. In just the last 24 hours, I've seen over 1.25 million unique IPs accessing my site, and this is way more than the usual 500-1000 daily visitors I expect. To give you some perspective, my logs show around 9 million different IPs in the last month, which is concerning. My site doesn't contain anything particularly valuable, like cryptocurrency or sensitive data, and even with all this traffic, my server is holding steady without crashing. I'm curious about what might be causing this: are these bots harvesting content, are there infected devices involved, or could they be AI crawlers? I've enabled CAPTCHA for users from certain regions, but it seems to have led to an increase in random residential IP hits instead. I'd love to hear if anyone else has faced this kind of situation and what solutions worked for you!
5 Answers
Using Cloudflare’s WAF rules, you should block incoming traffic based on geographic regions you’re targeting for business. This simple strategy can greatly alleviate the bot issue without disrupting your genuine visitors.
I’m already doing that for some traffic, but I wish it could cover all random bot activity!
I had a similar problem and managed to tame it using Cloudflare, but it required a lot of tinkering with the settings. If you want to get it right, there’s quite a bit of manual configuration involved.
Yeah, I set up something to show captchas for multiple requests, but it's not perfect when you have millions of different IPs suddenly on your site.
Totally, I think it’s all about identifying patterns and then setting up some rules to avoid them.
It sounds like you're dealing with high volumes of bots. One approach you can take is to set up a honeypot and block known bad IPs. At my workplace, we see a lot of bot traffic targeting specific files, and instead of trying to prevent them, we just block those who hit suspicious endpoints.
That's true! Honeypots can be effective, but many of these bots seem to just hit a valid URL and leave, making them tricky to catch.
For most sites, a WAF like Cloudflare can help manage this. You have to configure it properly though. Some settings can significantly filter out unwanted requests.
As the owner of a bot mitigation service, I suggest disabling IPv6 unless absolutely necessary. It might help narrow down the IP space that your attackers can use. Your focus should be on controlling how your site responds to requests rather than blocking them outright because blocking such a large volume may not be feasible.
That's a risky take. Disabling IPv6 could limit access for legitimate users too. Plus, IPv6 can be more manageable for filtering attacks.
Absolutely! IPv6 offers unique ways to block potential attacks by targeting larger address ranges efficiently.
This hardcore bot activity is unfortunately common when your site is public. I manage a personal server, and while I only use it myself, I often see bots hitting the logs non-stop. If your site is gaining attention, expect this to ramp up!
Exactly! I learned the hard way that any public domain will attract unwanted attention.
Absolutely! I’ve encountered similar international traffic and had to make some adjustments to how I manage my access.