Tuesday, April 29, 2008

RAFB paste download

I wrote this using 'lynx -source', but I probably should have used LWP.

Anyway, this grabs all of the code from rafb.net and stores each of the pastes into their own individual directories based on the code the pastes are written in. I use this primarily to have a big repository of code that I can reference when I want to see examples of a function, or something.


#!/usr/bin/perl
chdir("/var/www/htdocs/code");
%dirs = (
"Plain Text" => "plain",
"VB" => "vb",
"SQL" => "sql",
"Ruby" => "ruby",
"Python" => "python",
"PHP" => "php",
"Perl" => "perl",
"Java" => "java",
"C89" => "c89",
"C++" => "cpp",
"C\#" => "csharp",
"C" => "c",
);

open(URL, "lynx -source \"http://www.rafb.net/paste/results.html\"|") || die "$!";
while() {
if(/^.*?\;\"\>.*?\<\/td\>\(.*?)\<\/td.*?\/p\/(.*?)\.html\".*/) {
$lang{$2} = $1;
}
}

foreach $line (keys %lang) {
chomp($line);
print "language: $lang{$line} code source - $line\n";
if($dirs{$lang{$line}} ne "") {
print "This will go into the $dirs{$lang{$line}} directory.\n";
if(-e("$dirs{$lang{$line}}/$line.txt")) {
print "This already exists.\n";
next;
} else {
system("wget -P $dirs{$lang{$line}} \"http://www.rafb.net/p/$line.txt\"");
}
} else {
print "We don't want this code.\n";
}
}

No comments: