Tuesday, April 8, 2014

Exporting firewall rules to a CSV

Sometimes it can be useful to export and analyze rules in a CSV type format. This comes in especially handy when working with long and complex firewall policies.

I came across the perl script below that takes firewall policies from a text file and performs the CSV conversion for you.

Syntax: csvparse.pl rules.txt

<rules.txt> should be in the following format:
config firewall policy
     edit 1
         set srcintf "internal"
         set dstintf "wan1"
             set srcaddr "all"
             set dstaddr "all"
         set action accept
         set schedule "always"
             set service "ANY"
         set logtraffic-app disable
         set webcache enable
         set nat enable
     next
end
And here's the Perl script.
#!/usr/bin/perl
#

my $output = "policies-out.csv";

my $policyid = 0;
my $setting = "";
my %policies;
my %seen;
my $in_policy_block = 0;
my @order_keys;
my $order_key = 0;

open(OUTFILE,">$output") || die "Can't open file $output: $!\n";

while (<>) {
	if ($in_policy_block) {
		if (/^\s*edit\s+(\d+)/i) {
			# start of new policy
			$policyid = $1;
		} elsif (/^\s*set\s+(\S+)\s+(.*)$/i) {
			# it's a setting
			my ($key,$value) = ($1,$2);
			$value =~ tr/\"\015\012\n\r//d;
			$order_keys[$order_key++] = $key unless $seen{$key}++;
			$policies{$policyid}{$key} = $value;
		} elsif (/^\s*end/i) {
			$in_policy_block = 0;
		}
	} elsif (/^\s*config firewall policy/i) {
		$in_policy_block = 1;
	}
}

# print out our header
print OUTFILE "id";
foreach my $key (@order_keys) {
	print OUTFILE ",$key";
}
print OUTFILE "\n";

# now print out each record
foreach my $policy (sort keys %policies) {
	print OUTFILE "$policy";
	foreach my $key (@order_keys) {
		if (defined($policies{$policy}{$key})) {
			print OUTFILE ",$policies{$policy}{$key}";
		} else {
			print OUTFILE ",";
		}
	}
	print OUTFILE "\n";
}


close(OUTFILE);

17 comments:

Schubert Baliza said...

Great script Sebastian. Tks for sharing!

Francois Ropert said...

oh my! perl devs still exists in 2014??? :)

Alexander N said...

Thanks for sharing!

This brings up another question I never managed to figure out...

How do you list the rule order in cli (or from a configuration backup file) ?

Thank you in advance!

Simon said...

That's really cool...but for me we have a lot of clients with identity based profiles...and this cause the script to break, I guess because of the extra "end" statements.

I may try and take a look sometime as I love the concept, but I'm no Perl expert.

Thanks for sharing.

thomaaz said...

Nice script Sebastian.

I ported it to Python and made it public on github : https://github.com/maaaaz/fgpoliciestocsv

I also took the liberty of putting your version in the same repo.

I tested both scripts against several production configuration files. No bug so far. Feel me to report some.

Jay ward said...

I know this can help me soon, so I already bookmarked it :)
firewall appliance is very important specially for the sake of my kids.

kate reid said...

I've already bookmarked this post, thanks! I am also using utm firewall appliance to help me protect My pc.

aalia lyon said...

Its Awesome blog ,, if any firewall problem click it here its helps you to solve your peoblem.
windows firewall error 1068 windows 7
Thanks
Aalia lyon

Unknown said...

Hi, thank you for great script, but i need little help with making the sort of rules in original sequence and i'm not strong in perl - if i delete the 'sort' in 'foreach my $policy ..' it will output rules in random order, so i need to put some index (something like $linenum) instead of policy id in hash and i cannot figure out, how to do it. Thank you ...

Unknown said...

Sorry, being unknown, i'm Eric ;)

Unknown said...

Well, i finally figure out, how to sort it in sequential order.

It's pobably not so elegant, but it works for me.

here is diff ...

12a13
> my $policyseq = 1;
15c16,17
<
---
> $key = "id";
> $order_keys[$order_key++] = $key;
20a23
> $policies{$policyseq}{$key} = $policyid;
26c29
< $policies{$policyid}{$key} = $value;
---
> $policies{$policyseq}{$key} = $value;
28a32,33
> } elsif(/^\s*next/i) {
> $policyseq++;
36c41
< print OUTFILE "id";
---
> print OUTFILE "seq";
43c48
< foreach my $policy (sort keys %policies) {
---
> foreach my $policy (sort {$a <=> $b} keys %policies) {


It shows 'seq' along with 'id' at the beginning.

PR said...

Has anyone tried mapping firewall objects to IP addresses?

Pierrot said...

Bonjour, est il possible de prendre en compte l'affichage des VDOM avec ce script ?

Merci

Pierre

Erik Ruwalder said...

I updated it a bit to make it work for identity based policy (and keep the order)
seems to work for me, hope for someone else too

this is my diff to the original file

6a7
> my $id_policyid = 0;
11a13
> my @pol_list;
20a23
> push @pol_list, $policyid;
26a30,45
> } elsif (/^\s*config identity-based-policy/i) {
> ID_POL: while (<>) {
> if (/^\s*edit\s+(\d+)/i) {
> # start of new policy
> $id_policyid = $1;
> push @pol_list, "$policyid-$id_policyid";
> } elsif (/^\s*set\s+(\S+)\s+(.*)$/i) {
> # it's a setting
> my ($key,$value) = ($1,$2);
> $value =~ tr/\"\015\012\n\r//d;
> $order_keys[$order_key++] = $key unless$seen{$key}++;
> $policies{"$policyid-$id_policyid"}{$key} = $value;
> } elsif (/^\s*end/i) {
> last ID_POL;
> }
> }
43c62
< foreach my $policy (sort keys %policies) {
---
> foreach my $policy (@pol_list) {
54d72
<

Maxime Harel said...

At line 56, I replaced:
^\s*end
by:
^end

then AMAZING!!!

Thx a lot for this script :)

Augustinus Hipponens said...

Could you please upload a final script? I'm not familiar with diff files.
Thanks.

Mohammad Safe said...

Hey guys
Does it possible for any one to share video of how to do it ??


Tnx