Perltidy

The first of these tools is called perltidy, which is a perl script “indenter and reformatter”.  Essentially it allows you to keep a universal formatting standard across all your Perl scripts, and even better, across all Perl scripts written by others you work with too if you can convince them of the need!

Perltidy has many options, but I tend to stick to a pretty basic set of requirements:

perltidy -ce -b <file>.pl

The -ce flags above are short for –cuddled-else, which means we want to have else and elsif’s following immediately after the curly brace belonging to the previous block.

The -b tells perltidy to create a backup of the file as <file>.pl.bak, and modify the file in place.  The default behaviour is to writeout a new file to <file>.pl.tdy.

Perlcritic

For analysing my perl I use perlcritic.  The following extract is taken from the perlcritic man page:

“the executable front-end to the Perl::Critic engine, which attempts to identify awkward, hard to read, error-prone, or unconventional constructs in your code.  Most of the rules are based on Damian Conway’s book Perl Best Practices.”

So heres the script:

#!/usr/bin/perl -w
use strict;

# Open 'find' process to list files recursively with paths
open(FIND, "find |");
while(<FIND>) {
 # remove leading / trailing whitespace
 chomp; 

 # Don't rename ourself
 next if $_ eq $0;       

 # create temp file (windows wont allow to rename in place from uppercase to lowercase)
 my $name = $_;
 my $tmp = $_.'~';
 rename($name, $tmp);

 # make lowercase
 $name = lc($name);
 rename($tmp, $name);

 my $newname = $name;
 # remove apostrophes
 $newname =~ s/[']//g;
 # remove round brackets and replace with hyphens
 $newname =~ s/[()]/-/g;
 # remove spaces and replace with underscores
 $newname =~ s/ /_/g;
 # remove where in sequence there is underscore, hyphen, underscore and replace with a hyphen
 $newname =~ s/_-/-/g;
 $newname =~ s/-_/-/g;
 # where there are one or more digits followed by an underscore change the underscore to a hyphen
 $newname =~ s/(d+)_/$1-/g;
 # remove all ampersands and replace them with '_and_'
 $newname =~ s/&/_and_/g;
 # remove underscores where there are 2 or more, and replace with a single underscore
 $newname =~ s/_{2,}/_/g;

 # write out the changes
 rename($name,$newname);

}
close(FIND);

#!/usr/bin/perl -w
use strict;

# open 'find' process to list files recursively with paths
open(FIND, "find |");

while(<FIND>) {

  chomp;
  # don't rename ourself if script in same as executing
  next if $_ eq $0;

  # first move the file to $name~ and then back to the lowercase original to allow for fat32 ignoring case,
  # and therefore claiming that a file with this name already exists
  my $name = $_;
  my $tmp = $_.'~';

  rename($name, $tmp);
  rename($tmp, lc($name));
}
close(FIND);

To start off the script just opens a handle on the linux find command with an output pipe and calls it…well…FIND. The while loop will then iterate through each line the find command returns and rename using the rename() function.

The rename function takes in two arguments, the old name (as returned by find) and the new name. As the current returned value we are looking at is stored in the special variable $_, we can pass this as the old file name. We can then just use the lc() function to convert the old mane to lowercase by putting $_ as the argument for lc(). Finally we close the filehandle on FIND. And voila!

DESCRIPTION
csvsql enables you to access a CSV file as if it were a table in a database. This means you can use SQL queries, with each ’common seperated
value’ as part of a column.

Traditionally in order to access specific information from a CSV file it can take considerable use of regular expressions, awk and sed amongst
others. What if you only wanted to take out a handful of lines from a large file? It nearly becomes easier to do it manually than to figure out
the expressions needed otherwise.

SYNOPSIS
Csvsql [-h] [-v] [-c "command"] [-s seperator] [-p filename] [-l logging]

COMMAND LINE OPTIONS
-h This help message

-v Prints Csvsql version number

-c Run this command and exit. Command should be in “”

-s User defined separator (defaults to , ) if using a space please use ” ”

-p Copy output to a specified file. This option can also be specified in interactive mode.

-l Switch logging on or off. 1 = on, 0 = off (defaults to 1)
INTERACTIVE OPTIONS
SUPPORTED SQL QUERIES

At present the supported queries are:

SELECT
SELECT [ * | select_expr ]
[ FROM file_name [ WHERE where_condition ]
[ LIMIT row_count ] ]

INSERT
INSERT INTO file_name
(col_name,col_name,…)
VALUES (expr,expr,…)

UPDATE
UPDATE file_name
SET col_name=expr
[ WHERE where_condition ]

DELETE
DELETE FROM file_name
WHERE where_condition

CREATE
CREATE file_name
(col_name,col_name,…)
WITH VALUES (expr,expr…)

WHERE
The WHERE clause, if given, indicates the condition or conditions that rows must satisfy to be selected. where_condition is an expression
that evaluates to true for each row to be selected. The statement selects all rows if there is no WHERE clause.

The WHERE clause can be used for selections that include an AND or an OR operator, where AND requires for both conditions to be true, and OR
requires either one or the other.

NON-SQL COMMANDS

open [file_name]
Opens a file and read it’s contents into memory

close [file_name]
Closes the specified file and release all locks. If no file specified will default to the currently opened file.

describe [file_name]
Display a files column headings.

set pipe = [file_name | off]
Will copy output to a specified file. This option can also be specified as a command line option

ls [path]
List specified directory

dir [path]
Same as above

clear
Clear screen

cls Same as above

version
Print version number

dump
Dump a listing of data in memory

SPECIAL VALUES
isnull
In order to check for, or insert a null value, you should use isnull in place of ’’ Example: SELECT * FROM file_name WHERE col_name = isnull

LICENCE
Copyright (C) 2008 Killian Faughnan

CsvSQL is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Soft‐
ware Foundation, either version 3 of the License, or (at your option) any later version.

You should have received a copy of the GNU General Public License along with CsvSQL. If not, see <http://www.gnu.org/licenses>

Author: Killian Faughnan <killian [at] killianfaughnan [dot] com>

<http://www.killianfaughnan.com>

SEE ALSO
For further documentation please see <http://docs.killianfaughnan.com/csvsqldocs>

Csvsql is a project I started for my BSc’s final year software project. It is written in Perl and can be used to access information in csv files in the same way you would access a table in a database. This means you can use SQL queries, with each common seperated value as a field in a column.

Traditionally in order to access specific information from a csv file it can take considerable use of regular expressions, and awk commands. What if you only wanted to take out a handful of lines from a large file? It nearly becomes easier to do it manually than to figure out the expressions needed in awk.

CsvSQL Download

You can download CsvSQL here. If you have and questions, suggestions, criticisms or comments please let me know as I would be interested in hearing them. And if you do decide to use this program for something, let me know how you get on!

download csvsql

Printing a string in Perl couldn’t be easier, you simply use the print function. If you want to print a line in Perl you simple need to write the following:

  print "Hello World!";

  Output: Hello World!

Similarly you you print a variable as follows:

  my $string = "Hello World!";
  print $string;

  Output: Hello World!

eading from a file in Perl is pretty simple. After all, the language is pretty much built around dealing with text. The first step is to open the file, in which case you just need to use the following:

  open(INFILE,"< myfile.txt") or die "Can't open file: $!";

In this statement we are using the open() function with a filehandle and filename as arguments. We will use this filehandle for all further actions on the file. The rest of the statement will print the error “Can’t open file: $!” where $! is the filename, should the script be unable to open the file.

Reading an open file and printing it’s contents

To read a file which already has an open filehandleyou can use the following:

  while(<INFILE>){
    print $_;
  }

This while loop will read through the file line by line, printing the contents, until it reaches the last line where it will terminate. In Perl $_ is a special character. It represents the “default input and pattern matching space”, which in the context above means that each line of the file is assigned to $_ in turn as the file is being read. So when we use print $_ we are printing the current line, and with each iteratation of the loop that line changed to the next.

The below will print “Hello!” 10 times:

  for ($i = 0; $i < 10 ; $i++){   print "Hello!n"; }

  Output: Hello!

Foreach loop

For iterating through an array it can be handy to use a foreach loop. In the example below the foreach loop will access each variables in the array and allow you to use it as $i:

  @array = ('one','two','three');

  foreach $i (@array) {   print "$i, "; }

  Output: one, two, three,

To create a new array it is as simple as declaring it as shown below. This method can also be used to clear an existing array, though you will have to drop the my.

my @array = ();

Creating an array with predefined elements

To create a new array with elements just declare the array with a comma delimited list in brackets after it.

my @array = (hendrix,santana,young,clapton);

Adding elements to the end of an array with push()

In this example we will add new elements to an array using the push() function. To do this we will use a loop to push elements from an existing array @array into our new array cleverly named…@newarray. The variable $item represents the value of the currently accessed element in @array

my @array = (hendrix,santana,young,clapton);

my @newarray = ();

foreach my $item (@array) {

  push( @newarray,$item);

}