23 Feb 11 Writing Better Perl
I’ve recently started using a few tools to help clean up my Perl. The first of these tools I use for standardised formatting of my Perl scripts. The second I use to analyse my Perl source code.
Perltidy
The first of these tools is called perltidy, which is a perl script “indenter and reformatter”. Essentially it allows you to keep a universal formatting standard across all your Perl scripts, and even better, across all Perl scripts written by others you work with too if you can convince them of the need!
Perltidy has many options, but I tend to stick to a pretty basic set of requirements:
perltidy -ce -b <file>.pl
The -ce flags above are short for –cuddled-else, which means we want to have else and elsif’s following immediately after the curly brace belonging to the previous block.
The -b tells perltidy to create a backup of the file as <file>.pl.bak, and modify the file in place. The default behaviour is to writeout a new file to <file>.pl.tdy.
Perlcritic
For analysing my perl I use perlcritic. The following extract is taken from the perlcritic man page:
“the executable front-end to the Perl::Critic engine, which attempts to identify awkward, hard to read, error-prone, or unconventional constructs in your code. Most of the rules are based on Damian Conway’s book Perl Best Practices.”
29 Jun 10 OpsView check_snmp_runningproc Not Matching Process List
I was adding a few checks for oracle processes using check_snmp_runningproc and noticed a few problems. It seems that when the check returns a list of processes from a remote host it lowercases them all and attempts to strips out .e .ex and .exe extensions. The extensions used in the regular expression weren’t escaping the dot which resulted them in stripping out any character followed by an e, ex or exe.
In my case this meant it was matching against “orac” instead of oracle. I also noticed that the user input wasn’t undergoing the same translations as the returned processes (both the conversion to lower case and the stripping of extensions). I fixed this by cleaning up the regexp, and using an lc() function and the same regexp for user input. I submitted the patch which has been included in SVN. The most up to date version of this check is here:
http://svn.opsview.org/opsview/trunk/opsview-core/nagios-plugins/check_snmp_runningproc
16 Jun 09 Renaming Files Using Perl
I finally got around to writing a script to rename all my music files to the same convention. That being either <track>-<song>.mp3 or <song>.mp3 with no spaces (I prefer underscores instead) and all lowercase. I know, it looks so simple! But the problem is all those occasions where you have something like <track>_-_<SONG>_-_(<remix by>).mp3 which just looks horrible! What this script does could be done in a lot less lines than I have here, but I thought it might be a good way for someone who is unsure of regular expressions to get a grasp on the topic.
So heres the script:
#!/usr/bin/perl -w
use strict;
# Open 'find' process to list files recursively with paths
open(FIND, "find |");
while(<FIND>) {
# remove leading / trailing whitespace
chomp;
# Don't rename ourself
next if $_ eq $0;
# create temp file (windows wont allow to rename in place from uppercase to lowercase)
my $name = $_;
my $tmp = $_.'~';
rename($name, $tmp);
# make lowercase
$name = lc($name);
rename($tmp, $name);
my $newname = $name;
# remove apostrophes
$newname =~ s/[']//g;
# remove round brackets and replace with hyphens
$newname =~ s/[()]/-/g;
# remove spaces and replace with underscores
$newname =~ s/ /_/g;
# remove where in sequence there is underscore, hyphen, underscore and replace with a hyphen
$newname =~ s/_-/-/g;
$newname =~ s/-_/-/g;
# where there are one or more digits followed by an underscore change the underscore to a hyphen
$newname =~ s/(d+)_/$1-/g;
# remove all ampersands and replace them with '_and_'
$newname =~ s/&/_and_/g;
# remove underscores where there are 2 or more, and replace with a single underscore
$newname =~ s/_{2,}/_/g;
# write out the changes
rename($name,$newname);
}
close(FIND);
12 Nov 08 Renaming Files As Lowercase Using Perl
This is just a handy script I use for renaming my music collection. It got to be too much trouble having the first letter of each word capitalised, getting new music with the names all in caps, or all in lowercase…so I decided to make everything lowercase and write a script to do it for me!
#!/usr/bin/perl -w
use strict;
# open 'find' process to list files recursively with paths
open(FIND, "find |");
while(<FIND>) {
chomp;
# don't rename ourself if script in same as executing
next if $_ eq $0;
# first move the file to $name~ and then back to the lowercase original to allow for fat32 ignoring case,
# and therefore claiming that a file with this name already exists
my $name = $_;
my $tmp = $_.'~';
rename($name, $tmp);
rename($tmp, lc($name));
}
close(FIND);
To start off the script just opens a handle on the linux find command with an output pipe and calls it…well…FIND. The while loop will then iterate through each line the find command returns and rename using the rename() function.
The rename function takes in two arguments, the old name (as returned by find) and the new name. As the current returned value we are looking at is stored in the special variable $_, we can pass this as the old file name. We can then just use the lc() function to convert the old mane to lowercase by putting $_ as the argument for lc(). Finally we close the filehandle on FIND. And voila!
12 Nov 08 CsvSQL Man Page
NAME
Csvsql – use SQL queries to access information in CSV files
DESCRIPTION
csvsql enables you to access a CSV file as if it were a table in a database. This means you can use SQL queries, with each ’common seperated
value’ as part of a column.
Traditionally in order to access specific information from a CSV file it can take considerable use of regular expressions, awk and sed amongst
others. What if you only wanted to take out a handful of lines from a large file? It nearly becomes easier to do it manually than to figure out
the expressions needed otherwise.
SYNOPSIS
Csvsql [-h] [-v] [-c "command"] [-s seperator] [-p filename] [-l logging]
COMMAND LINE OPTIONS
-h This help message
-v Prints Csvsql version number
-c Run this command and exit. Command should be in “”
-s User defined separator (defaults to , ) if using a space please use ” ”
-p Copy output to a specified file. This option can also be specified in interactive mode.
-l Switch logging on or off. 1 = on, 0 = off (defaults to 1)
INTERACTIVE OPTIONS
SUPPORTED SQL QUERIES
At present the supported queries are:
SELECT
SELECT [ * | select_expr ]
[ FROM file_name [ WHERE where_condition ]
[ LIMIT row_count ] ]
INSERT
INSERT INTO file_name
(col_name,col_name,…)
VALUES (expr,expr,…)
UPDATE
UPDATE file_name
SET col_name=expr
[ WHERE where_condition ]
DELETE
DELETE FROM file_name
WHERE where_condition
CREATE
CREATE file_name
(col_name,col_name,…)
WITH VALUES (expr,expr…)
WHERE
The WHERE clause, if given, indicates the condition or conditions that rows must satisfy to be selected. where_condition is an expression
that evaluates to true for each row to be selected. The statement selects all rows if there is no WHERE clause.
The WHERE clause can be used for selections that include an AND or an OR operator, where AND requires for both conditions to be true, and OR
requires either one or the other.
NON-SQL COMMANDS
open [file_name]
Opens a file and read it’s contents into memory
close [file_name]
Closes the specified file and release all locks. If no file specified will default to the currently opened file.
describe [file_name]
Display a files column headings.
set pipe = [file_name | off]
Will copy output to a specified file. This option can also be specified as a command line option
ls [path]
List specified directory
dir [path]
Same as above
clear
Clear screen
cls Same as above
version
Print version number
dump
Dump a listing of data in memory
SPECIAL VALUES
isnull
In order to check for, or insert a null value, you should use isnull in place of ’’ Example: SELECT * FROM file_name WHERE col_name = isnull
LICENCE
Copyright (C) 2008 Killian Faughnan
CsvSQL is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Soft‐
ware Foundation, either version 3 of the License, or (at your option) any later version.
You should have received a copy of the GNU General Public License along with CsvSQL. If not, see <http://www.gnu.org/licenses>
Author: Killian Faughnan <killian [at] killianfaughnan [dot] com>
<http://www.killianfaughnan.com>
SEE ALSO
For further documentation please see <http://docs.killianfaughnan.com/csvsqldocs>
12 Nov 08 CsvSQL
About CsvSQL
Csvsql is a project I started for my BSc’s final year software project. It is written in Perl and can be used to access information in csv files in the same way you would access a table in a database. This means you can use SQL queries, with each common seperated value as a field in a column.
Traditionally in order to access specific information from a csv file it can take considerable use of regular expressions, and awk commands. What if you only wanted to take out a handful of lines from a large file? It nearly becomes easier to do it manually than to figure out the expressions needed in awk.
CsvSQL Download
You can download CsvSQL here. If you have and questions, suggestions, criticisms or comments please let me know as I would be interested in hearing them. And if you do decide to use this program for something, let me know how you get on!
12 Nov 08 Printing In Perl
Printing A String
Printing a string in Perl couldn’t be easier, you simply use the print function. If you want to print a line in Perl you simple need to write the following:
print "Hello World!";
Output: Hello World!
Similarly you you print a variable as follows:
my $string = "Hello World!"; print $string; Output: Hello World!
12 Nov 08 Reading From A File In Perl
Opening a file
eading from a file in Perl is pretty simple. After all, the language is pretty much built around dealing with text. The first step is to open the file, in which case you just need to use the following:
open(INFILE,"< myfile.txt") or die "Can't open file: $!";
In this statement we are using the open() function with a filehandle and filename as arguments. We will use this filehandle for all further actions on the file. The rest of the statement will print the error “Can’t open file: $!” where $! is the filename, should the script be unable to open the file.
Reading an open file and printing it’s contents
To read a file which already has an open filehandleyou can use the following:
while(<INFILE>){
print $_;
}
This while loop will read through the file line by line, printing the contents, until it reaches the last line where it will terminate. In Perl $_ is a special character. It represents the “default input and pattern matching space”, which in the context above means that each line of the file is assigned to $_ in turn as the file is being read. So when we use print $_ we are printing the current line, and with each iteratation of the loop that line changed to the next.
12 Nov 08 Perl Loop Basics
For loop
The below will print “Hello!” 10 times:
for ($i = 0; $i < 10 ; $i++){ print "Hello!n"; }
Output: Hello!
Foreach loop
For iterating through an array it can be handy to use a foreach loop. In the example below the foreach loop will access each variables in the array and allow you to use it as $i:
@array = ('one','two','three');
foreach $i (@array) { print "$i, "; }
Output: one, two, three,
12 Nov 08 Perl Array Basics
Initialising or clearing an array
To create a new array it is as simple as declaring it as shown below. This method can also be used to clear an existing array, though you will have to drop the my.
my @array = ();
Creating an array with predefined elements
To create a new array with elements just declare the array with a comma delimited list in brackets after it.
my @array = (hendrix,santana,young,clapton);
Adding elements to the end of an array with push()
In this example we will add new elements to an array using the push() function. To do this we will use a loop to push elements from an existing array @array into our new array cleverly named…@newarray. The variable $item represents the value of the currently accessed element in @array
my @array = (hendrix,santana,young,clapton);
my @newarray = ();
foreach my $item (@array) {
push( @newarray,$item);
}
