Saturday, April 26, 2008

Duplicate record remover

I figured I would post this because it's heavily commented. I have a program which monitors /var/log/messages for brute force attempts against sshd and when it sees them, it uses iptables to firewall them and also creates an entry into a blacklist file which is then loaded when my firewall script reloads. Periodically there will be duplicate records in the black list file, so this removes them.

#!/usr/bin/perl
###
# This removes duplicate records from the 'banned-ips' file associated
# with the firewall. The 'banned-ips' file is generated by
# the auto-blacklisting features of the firewall
#
# created by nwo
# 11/1/2007
###

### Needed because I didn't want to use system calls.
use File::Copy;

### Set up the iphash
%iphash = ();

### Define this as something else if you're not using the name 'banned-ips'
$iplist = "banned-ips";

### Get the size of the file for sanity checks later.
$iplist_size = (stat $iplist)[7];

### Random character generator. This is to avoid race conditions in /tmp
@chars = ("A" .. "Z", "a" .. "z", 0 .. 9);
$randfile = join ("", @chars[map{rand @chars} (1 .. 8)]);

### Lets make a backup to work with, rather than modifying the original before
### we know that everything's clean.
copy($iplist, "/tmp/$randfile") or die "File cannot be copied: $!";
### Make sure the file wasn't nuked, or something, after being copied.
if(-e("/tmp/$randfile")) {
### Get the file size to compare with the original. This helps eliminate
### race conditions completely. Only exception would be if someone were to
### create a smaller file then the original and then pad it with garbage
### to make the size match. But this, in combination with the random
### characters would make it very difficult.
$tmplist_size = (stat "/tmp/$randfile")[7];
if($iplist_size != $tmplist_size) {
print "File size mismatch between $iplist and /tmp/$randfile.\n";
print "This could be an accident, or a race condition exploit attempt.\n";
print "Review the /tmp/$randfile file for inconsistancies.\n";
exit(1);
}
}

### Ok, now that we know everything is cool, lets open up the backed
### up copy of the file.
open(F, "/tmp/$randfile") || die "$!";

### Lets also make sure the file contains IP's. I'm sure there's a much fancier way ### of doing this.. but....
while() {
if(/^(\d+\.\d+\.\d+\.\d+)/) {
$iphash{$1}++;
}
}
### We can now delete the original.. we have two copies. One in /tmp, and one in
### memory (iphash).
unlink($iplist);

### Open the file handle IPS for writting
open(IPS, ">>$iplist") || die "$!";

### Go ahead and print each entry in iphash to the file.
foreach $line (keys %iphash) {
print IPS "$line\n";
}

### Close our file handles...
close(F);
close(IPS);

### Lets open the newly created ip list and make sure it looks ok.
open(IPS, "$iplist") || die "$!";
while() {
print "$_";
}
close(IPS);


### Well, does it?!?
print "Does this look right? (N/y): ";
chomp($ans = );
$ans = uc($ans);

### Assume it's not if no answer is given.
if($ans eq "") { $ans = "N"; }
### Restore the backed up version from /tmp
if($ans eq "N") {
unlink($iplist);
copy("/tmp/$randfile", $iplist);
print "/tmp/$randfile was restored to $iplist\n";
exit(1);
}

### Everything looks ok..
if($ans eq "Y") {
print "Ok. Deleting the original in /tmp\n";
unlink("/tmp/$randfile");
exit(0);
} else {
### In case the user puts in something other than "N" or "Y"
print "No idea what you typed. Restoring original.\n";
unlink($iplist);
copy("/tmp/$randfile", $iplist);
print "/tmp/$randfile was restored to $iplist\n";
exit(1);
}

No comments: