I'm curious about the types of traffic you've had success blocking to enhance security on your site. For example, I know some bots look for vulnerabilities like those involving "/wp-" paths, so you could set a custom rule to block them with something like "(lower(http.request.uri.path) contains "/wp-")". Also, blocking traffic from known data center ASNUMs can be effective. What strategies and rules have worked well for you?
7 Answers
From what I’ve observed, a layered approach works best: start with bot score based rules, then use rate limiting, add JavaScript validation on sensitive pages, use geo controls if applicable, and finally implement custom IP/ASN rules as a last line of defense. Just remember, while community IP blacklists can help, they come with maintenance responsibilities that you should consider based on your long-term needs.
Check out this IP blacklist resource: https://ipbl.herrbischoff.com/. It can help you find malicious IPs to block.
I've had a lot of success using ASNUMs to block scraper bots, which really helped reduce junk traffic on my site.
I simply block Kubernetes.io to keep my boss from thinking it's a good idea to integrate it into our systems.
Another good resource is https://coreruleset.org/. It's useful for security rules.
For my clients, I've set up Cloudflare WAF on the free plan, utilizing geo-blocking and techniques to mitigate bad bots. A couple of years ago, I also used the nginx-bad-bot-blocker, which was pretty effective for me.
I’ve heard some argue that returning a 500 error is better than a 404—what’s your take on that? If you're having issues managing your traffic, consider checking out fail2ban.
How did you compile your list of ASNUMs without accidentally blocking legit traffic?