Hey everyone! I'm working on a script to analyze the logs of my online app and identify IP addresses that exhibit suspicious behavior so I can block them with a reverse proxy or firewall rule. I've managed to find the "bad IPs", but I'm struggling with what I like to call "time buckets". Basically, I want to exclude an IP address if it appears 5 times within one minute. I'm having issues with my script, especially with how I'm tracking occurrences and timestamps. Here's the part I've written so far, but I keep encountering errors, and I'm not sure how to proceed or fix it. If anyone has any insights on how I can improve this, I'd really appreciate your help!
4 Answers
I’ve reworked your script with a few fixes and improvements. Check out this version:
```bash
#!/usr/bin/bash
function main {
readonly CONTAINER='my_app'
readonly TEMP_FILE='/home/eric/monitoring/temp'
read -r LOG_FILE < <(docker inspect --format '{{.LogPath}}' "${CONTAINER}")
readonly LOG_FILE="${LOG_FILE:?Log file path is empty, docker command failed?}"
declare -A APPEARED
declare -A APPEARED_AT
tail -F "${LOG_FILE}" | while read -r LINE; do
read -r IP < <(grep -oP "([0-9]{1,3}.){3}[0-9]{1,3}" <<<"${LINE}")
is_valid_public_ip "${IP}" || continue
read -r CURRENT_DATE < >"${TEMP_FILE}"
done
}
main "$@"
```
This script includes error checking and counts occurrences much better. Make sure to test it in a safe environment first!
Looks good, but make sure to test that it doesn't skip valid entries!
Instead of using `$(OCCUR["$IP"])+1`, you should try `$((OCCUR["$IP"]+1))`. This way, it correctly increments the count for each occurrence of the IP.
Thanks for that one!
If you're looking for a tool that might simplify this process, have you considered using fail2ban? It’s really good for IP banning based on patterns. You can find examples of custom filters online to get you started.
Yes, I’ve tried fail2ban, but ran into some issues. I’m actually looking into setting it up with a systemd service to manage log directories.
Here's a suggestion: you could modify your script to store the timestamps of occurrences. Using an array, you can push the current timestamp each time an IP is logged. After that, you can filter to see if there are 5 entries within a minute. Would that help?
Thanks! I'm trying to wrap my head around it, but it sounds like it could really help!
Thanks a lot! This is really helpful. I need to go through it before I implement it.