One of my systems came under a lof of heat recently from basically two different subnets in Asia, which comprised of about 1700 attacking IP addresses. That is, some IPs are harvesting usernames and email addresses from publicly available content, while others randomly attempt to login with this data. Due to the large number of used IP addresses, there aren’t any obvious brute force patterns in the logs, created by the individual IP addresses. Instead, their login attempts are 20 min. apart or 2 hours apart. Standard fail2ban recipes clearly aren’t covering such a scenario at all.
What I’ve done is a bit crude, but turned out effective in this case: I’ve devised a custom fail2ban filter, which collects login attempts over a very long period of time, e.g., 2 or 3 hours. If in this long time period, there are multiple login attempts from the same IP, we check its geographic location. If the geographic location is (in my use case) Germany, then we’re letting it happen (and keep relying on the system’s own flood protection, which does exist, but alas! isn’t quite as sensitive or effective in the described scenario), and otherwise we’re banning for a couple of days or a week, or whatever. This allows for some legitimate use of the service from abroad, but quickly blocks repeated logins.
Since my system employs a proxy, I watch the HAProxy log via an
entry in fail2ban’s jail.local, e.g.:
1
2
3
4
5
6
7[user-login]
enabled = true
bantime = 72h
maxretry = 7
findtime = 150m
logpath = /var/log/haproxy.log
action=geohostsdenyfilter.d/user-login.conf, is not described as there’s nothing
special about it.)
The custom action, action.d/geohostsdeny.conf, looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart =
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop =
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = IP=<ip> &&
/etc/fail2ban/scripts/is_good_country.sh "$IP" ||
(ufw insert 1 deny from "$IP"/24 to any)
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban = IP=<ip> && ufw delete deny from "$IP"/24 to any 2>1 1>/dev/null
It essentially uses a script, scripts/is_good_country.sh, to decide
if the IP should be blocked or not, and then blocks the IP’s entire
class C subnet via ufw (but may as well use iptables directly).
Fortunately, x.y.z.231/24 is automatically translated into
x.y.z.0/24, which then leads to the blocking of the respective
subnet. The unblocking works much the same. (Notice the output
stream redirections in actionunban, which suppress error messages
when the IP to unban wasn’t actually banned in the first place. This
happens, if a German IP address triggers the action, but then ends up
not being banned. A minor ugliness of this method that I have, so
far, found unproblematic in practice. That is, for a while ufw thinks
the IP address is banned, while in reality it isn’t.)
On Debian-based systems, the script, is_good_country.sh, needs the
packages geoip-bin and geoip-database installed and may
look as follows or contain additional, more elaborate conditions:
1
2
3
4
5
6
7
8
9
10
11
12#!/bin/sh
GOOD="DE|Germany|AT|Austria|CH|Switzerland"
IP=$1
IS_GOOD_COUNTRY=`geoiplookup $IP | egrep "$GOOD" | wc -l`
if [ $IS_GOOD_COUNTRY -gt 0 ]
then
# echo "IP IS FROM 'GOOD' COUNTRY"
exit 0
else
# echo "IP IS FROM 'BAD' COUNTRY"
exit 1
fi
Especially in scenarios, where entire networks come attacking, this whole method is very efficient, because a single run of the custom fail2ban action will block hundreds of attacking IPs at once, without adding an endless amount of rules to ufw or iptables, thus potentially slowing down the system if the attack is sufficiently elaborate. And it’s easy to make the banning even more aggressive than just class C subnets.
I should add that the above was inspired by a helpful blog post by
Roland Michael, which adds
offending hosts to /etc/hosts.deny, but which I can’t use all that
often, since many services don’t support it. I needed something more
‘low-level’ than that, which works for all my services equally.
