Hey everyone! I'm relatively new to sysadmin and I'm transitioning from a 25-year development career to managing cloud web applications. I've been dealing with major issues where 60% or more of my server traffic comes from bots and malicious crawlers, leading to high resource consumption. Currently, I'm using the free version of CloudFlare, but I find the paid version hasn't really helped cut down on these malicious connections. I also tried BunkerWeb, but the results were similar, with lots of false positives that waste my team's time. While I feel my main security measures are working fine, it's these relentless attacks that are really causing problems. I included some logs from yesterday and today here: https://imgur.com/a/3HHng6h. By the way, this is my first post here, apologies for any mistakes in English or format!
7 Answers
I'm running Nginx with ModSecurity, which works well for my needs.
I use Fastly as my WAF, and I've been really happy with it. Just keep in mind, any WAF solution requires a good amount of time for testing and monitoring to make it truly effective for your production environments.
I've tried BunkerWeb with a web GUI, but it didn't stand out much for me.
I use a mix of HAProxy and ModSecurity with the Core Rule Set. It's a bit finicky to set up, but super effective once you get it right.
I don't use a traditional WAF, but I rely on HAProxy. It allows me to limit requests and implement sticktables. For example, I track 404 responses, and if a visitor hits more than 5 in 30 seconds, I block them. This method is effective because most bots access pages quickly. I also block URIs that start with a dot or specific sensitive paths. I've found this approach works great!
Thanks for the feedback! I’ve got some blocking rules in CloudFlare for WordPress paths and certain URL extensions, and they’ve helped reduce bot traffic a lot. I also use fail2ban to limit rates on fake pages, which works pretty well!
If configured properly, Cloudflare’s enterprise WAF is really powerful! Definitely recommend checking into it if you can.
We set up Apache as a proxy combined with ModSecurity and the Core Rule Set. It’s a solid free and open-source option.
That sounds promising! I’ll definitely look into scheduling a demo to check it out.