Perl

Files and Directories
Pattern Matching

Files and Directories

Operating on a file requires a file handle

Opening a file

open(FILE, "/home/4user3/~bob/orders.log");    # read
open(FILE, "</home/4user3/~bob/orders.log");   # read (< is optional)
open(FILE, ">/home/4user3/~bob/orders.log");   # write
open(FILE, ">>/home/4user3/~bob/orders.log");  # append

Closing a file
```
close(FILE);
```
By convention, file handles are in upper-case

Error Handlers

When opening a file, you should be prepared to deal with errors (cannot read from/write to file)

Use an error handler to catch these exceptions

open(FILE, ">>/home/4user3/~bob/orders.log")
  || &errorMessage("cannot open file orders.log");
sub errorMessage {
  my $message = shift @_;
  print "Content-Type: text/html\n\n";
  print $message;
  exit;	# exit from the script
}

How it works
- Call to open returns 0 if it fails
- In that case, the second part of the OR condition is executed
Whether to include the "Content-Type" line depends on your script (could use a flag and test it)

Writing to a File

Use the file handle as first argument to print

open(FILE, ">/home/4user3/~bob/orders.log");
print FILE "Order $oid was shipped $date.\n";  # writes to FILE
print "Your order was shipped today.\n";       # writes to STDOUT

Gaining Exclusive Access

When you write to a shared file, it is important to gain exclusive access to it

Perl provides the flock function for this purpose

open(FILE, ">>/home/4user3/~bob/orders.log");
flock(FILE, 2);  # obtains lock
print FILE"Order 123 shipped 01/18/2000.";
flock(FILE, 8);  # releases lock

Reading from a File

You can read from a file in multiple ways
- Read file one line at a time
- Read contents of the file into an array

Read file one line at a time

open(FILE, "/home/4user3/~bob/orders.log");
while (<FILE>) {
  # process the line
  # each line will be stored in the variable $_
}
close(FILE);

Read contents of the file into an array

open(FILE, "/home/4user3/~bob/orders.log");
@messages = <FILE>;
close(FILE);

The first technique is generally preferrable
- files can be large (memory-efficient)
- technique allows processing of input as it is read

Renaming and Removing a File

Files can be renamed:

$oldName = "orders.log";
$newName = "messages.log";
rename($oldName, $newName);

Files can be removed:

$fileName = "orders.log";
unlink($fileName);

Checking the Status of a File

You may want to check the status of a file (e.g. if it exists) before trying to access it
if (-option fileName) {
...
}
Options:
- -e to check if file exists
- -r to check for read permission
- -x to check for execute permission
- -w to check for write permission
- -d to check if a directory

Accessing Directories

Opening a directory

opendir(DIR, "/home/4user3/bob");
opendir(CURRENT_DIR, ".");
opendir(PARENT_DIR, "..");

Closing a directory
```
close(DIR);
```

Reading from a Directory

You can read from a directory in multiple ways
- Read directory one line at a time
- Read contents of the directory into an array

Read directory one line at a time

opendir(DIR, "/home/4user3/bob");
$file = readdir(MYDIR);
while ($file) {
  # processing for that directory
  # read next file or directory name
  $file = readdir(MYDIR);
}
close(DIR);

Read contents of the directory into an array

opendir(DIR, "/home/4user3/bob");
@files = readdir(DIR);  # names of files and directories
close(DIR);

Changing Directories and Creating New Ones

Default working directory is the one that contains the script
If you need to access several files in a different directory, you may want to change the working directory
```
chdir("../data");
```
Create a new directory with
```
mkdir("catalog", 0777);
```
Remove an existing directory with
```
rmdir("tmp");
```

Sending Data to another Program

To send data to another program write it via a pipe

The following example shows how to send mail from Perl

open(MAIL, "| /usr/bin/sendmail -t");
print MAIL "To: $to\n";
print MAIL "From: $from\n";
print MAIL "Subject: $subject\n";
print MAIL "$contents\n";
close(MAIL);

Pattern Matching

Perl provides powerful operators for analyzing form input
Three basic operations:
- Match (check for a pattern)
- Substitute (check for a pattern, and replace it by something else)
- Split (search for a delimiter and split up the data)
Pattern matching capabilities based on regular expressions that describe the pattern to match against

Matching

To find a pattern in a text use the matching operator
Syntax of the matching operator:
```
$text =~ m/pattern/[modifiers]
```

Optional modifiers:

$text =~ m/pattern/i  # ignores case
$text =~ m/pattern/m  # treats text as multiple lines (with \n)
$text =~ m/pattern/s  # treats text as single lines
$text =~ m/pattern/g  # matches all possible occurrences (iterator)
$text =~ m/pattern/o  # interpolates only once (optimization)

The match returns true as long as the text contains the pattern (it can also contain other data)
If the pattern uses the "/" character itself, replace it by some other character that isn't used
The "m" can be omitted, as long as the slashes ("/") are used
```
$text =~ /pattern/
```

Matching (Example)

This example determines the type of browser from the HTTP_USER_AGENT environment variable

$browser = $ENV{'HTTP_USER_AGENT'};
if ($browser =~ /MSIE/) { 
  $isMicrosoft = 1;
} elsif ($browser =~ /Mozilla/) {
  $isNavigator = 1;
}

Substitution

To find a pattern in a text and replace it by something else use the substitution operator

Syntax of the substitution operator:

$text =~ s/pattern/replacement/[i][m][s][g][e]

Optional modifiers (m, s, and o as above):

$text =~ s/pattern/replacement/i   # ignores case
$text =~ s/pattern/replacement/g   # replaces all occurrences
$text =~ s/pattern/replacement/e   # evaluates replacement

Substitution (Example)

Remove all occurrances of the IMG tag from a feedback field received via a form
```
$feedback = $q->param('feedback');   # CGI.pm
$feedback =~ s/<IMG[^>]*>//ig;
```
Analysis:
- Ignore case, i.e. "IMG" and "img" will match
- Uses a simple regular expression that matches all occurrences of "<IMG" up to the nearest ">"
- Substitute each of these by the empty string

Splitting

Use the split operator to divide up a text at a given delimiter and assign the result to an array
Syntax for the split operator:
```
split(/pattern/, $text);
```
The pattern defines the delimiter that separates the elements in the input

Splitting (Example)

Example provincial sales tax file:
```
BC:6.25
ON:7.00
QC:9.00
NB:5.00
```

Code to retrieve the sales tax for a given province:

sub salesTax {
  my $p = shift @_;
  open(FILE, "/home/shop/salestax");
  while (<FILE>) {
    my ($province, $tax) = split(/:/, $_);
    if ($province eq $p) { return $tax; }
  }
  return 0.0;
}

Regular Expressions

Patterns are described through regular expressions
```
^\d{3}-\d{4}$   # phone numbers (e.g. 520-2600)
```
Regular expressions are made up of
- atoms like a, [a-z], or \d, and
- operators like sequence, repetition, and alternation

Matching a Single Character (Atom)

a	# matches the letter a
\$	# matches the special character $
\n	# matches newline
\t	# matches tab
.	# matches any character except \n
\.	# matches a period

Character Classes (Atom)

Specify a class of characters to match within [ and ]

[abc]      # matches a, b, or c
[a-z]      # matches a range of characters
[a-zA-Z]   # matches all lowercase and uppercase
[a\-z]     # matches just a, -, or z

To match a character that is not in a given class begin the class definition with ^
```
[^0-9\-]   # matches everything except 0-9 and -
```
Often it is simpler to define which characters should not appear in a text, rather than which ones should

Several shorthands are predefined for common classes

\d  # matches any digit (same as [0-9])
\D  # matches any non-digit
\w  # matches any letter, digit or underscore
\W  # matches any character not in \w
\s  # matches any space, \t, \n, \r or \f
\S  # matches any character not in \s

Sequence (Operator)

The sequence operator is implicit

Used to match a series of characters, in that order

abc     # matches abc
ab?c    # matches ac or abc
(ab)+   # matches ab, abab, etc.
a(b|c)d # matches abd or acd

Repetition (Operator)

There are four repetition operators
- ? (optional)
- * (zero or more)
- + (one or more)
- {m,n} (at least m, at most n)

Examples:

ab?c	# matches ac or abc
ab*c	# matches ac, abc, abbc, etc.
(ab)+	# matches ab, abab, etc.
ab{2,4}	# matches abb, abbb, and abbbb
(ab){3} # matches ababab
a{3,}   # matches aaa, aaaa, etc.

Alteration (Operator)

The alteration operator (|) defines a choice between different options within a pattern
```
ab|cd   # matches ab or cd
a(b|c)d # matches abd or acd
```

Anchors (Operator)

To anchor a pattern at the beginning of a line, use ^
```
^http:
```
To anchor a pattern at the end of a line, use $
```
\.gif$
```
Use \b to indicate that a pattern must begin or end at a word boundary
```
\b[A-Z][a-z]*\b   # matches any capitalized word
```

Remembering Matches

To store the result of a match in variables
- Enclose the parts of your pattern in parentheses that you want to store
- Count the number of left parentheses to determine which variable to use
Use $1, $2, ... (as determined by number of left parentheses) to refer to the match result from outside the pattern
```
$header = "Subject: Your Order";
$header =~ /$(\w+):\s+(\w*)/;
$tag = $1;    # Subject
$value = $2;  # Your Order
```

The result of matching can also be assigned to an array

$header = "Subject: Your Order";
($tag, $value) = $header =~ /$(\w+):\s+(\w*)/;

Input Space

Many of the above examples can be expressed more succinctly using the special variable $_
$_ is the input space, on which many standard operations such as print and pattern matching operate
```
$_ = "Subject: Your Order";
($tag, $value) = /$(\w+):\s+(\w*)/;
```
As pointed out earlier, for example, $_ is set when reading from a file using a while loop

For example, the following code reads from a password file and checks if the user name and password were correct

sub verifyPassword {
  my ($user, $password) = @_;
  open(PWD, "/home/alice/password");
  while (<PWD>) {
    if (/^$user:$password$/) { return 1; }
  }
  return 0;
}

Iterating through Matches

As noted earlier, the g modifier allows you to iterate through all occurrences of a pattern in a string
For illustation, we could implement the following simple style checker that counts words you should avoid
```
while ($text =~ /but/) {
  buts$++;
}
print "Your text contained $buts buts.";
```

Things to Watch Out For

Regular expression operators are greedy

They will match the largest element they can find, at the first position possible

$_ = "Greetings, planet Earth!";
/G.*t/	    # matches Greetings, planet Eart
	
$_ = "This 'test' isn't successful";
/'.*'/	    # matches 'test' isn'

Things to Watch Out For (Fixed)

A fix for the first example is to tell the matcher, not to be greedy, by adding a "?" to the greedy operator
```
$_ = "Greetings, planet Earth!";
/G.*?t/     # matches Greet
```
A fix for the second example is to exclude single quotes from the pattern (i.e. making it more specific)
```
$_ = "This 'test' isn't successful";
/'[^']*'/   # matches 'test' only
```

Readings

Read Chapter 5 in CGI Programming with Perl