Coderholic

There is an article over a DZone today “What if your contact form fails?”. The article discusses what to do if the contact form on your website fails to send a message. The approach they suggest is to display a message to the user saying it failed, and presenting them with a mailto link so they can send you an email themselves.

This is one approach to the failed contact form problem, but it isn’t going to help much if the problem is actually with your email address. For example, you may have exceeded your disk quota, or you might have a badly configured mail server.

Another problem with the mail solution is that it requires the user to perform another action. They’ve gone to the trouble of filling out a contact form. If that fails they might not bother to send to send you an email too (although the suggested approach in the article does reduce the effort required).

An Alternative Approach

So what other approaches are there? What I do is store all contact form information in a database. Below is the SQL to create the basic messages table:

CREATE TABLE messages (
    id int auto_increment NOT NULL,
    sent timestamp NOT NULL,
    sender varchar(255) NOT NULL,
    message text,

    PRIMARY KEY(id)
);

Then when a contact form gets submitted you can store the information in the database before sending an email. Below is some sample PHP code, using PDO to access the database:

// Get the form data
$sender = $_POST['sender'];
$message = $_POST['message'];

// Add the message to the database
$query = “INSERT INTO messages(sender, message) VALUES (:sender, :message);
$stmt = $pdo->prepare($query);
$stmt->bindParam(’:sender’, $sender, PDO::PARAM_STR);
$stmt->bindParam(’:message’, $message, PDO::PARAM_STR);
$stmt->execute();

// Send the mail
mail(”me@my.domain”, “Contact Form Message”, $message, “From: {$sender}”);

All that is left to do is create an admin page that lists all messages, and allows us to delete them. We could even include a link to the admin page in the emails we send to ourselves. By using this approach we never lose the message content, even if the mail call fails, because they all get stored in the database. All we need to do is check the admin page every so often and we’ll soon see if there are any new messages there.

Of course, this approach doesn’t have to be used in isolation. We can combine it with the approach suggested in the DZone article, and present the user with a mail link if sending the mail does fail. If the user decides not to send a mail though, we haven’t lost anything.

If you want to make changes to a single image, such as resizing or converting from one file format to another, then you’ll probably load up the image in an editor and manually make the required changes. This approach is great for a single image, but it doesn’t really scale past more than a few images, at which point it becomes time consuming, not to mention boring!

This is where Python and the Python Imaging Library (or PIL) come in, allowing you to write scripts that process images in batch. The PNG2GIF converter I wrote back in August is just one example of batch image processing using Python and PIL. In this post I’m going to explore some other uses, and provide lots of example code.

Batch Processing

No matter what processing we want to do on our images the script outline will be the same: loop over all provided arguments, and perform the processing. The code to do this is shown below:

#!/usr/bin/env python
import sys
import Image

# Loop through all provided arguments
for i in range(1, len(sys.argv)):
	try:
		# Attempt to open an image file
		filepath = sys.argv[i]
		image = Image.open(filepath)
	except IOError, e:
		# Report error, and then skip to the next argument
		print “Problem opening”, filepath, “:”, e
		continue

	# Perform operations on the image here

This will allow our script to be called in all of the following ways:

batch_process.py image.gif
batch_process.py image1.png image2.gif image3.jpg
batch_process.py *.png
batch_process.py image1.gif, *.png

Processing

With the help of PIL the image processing is really simple. Below is a collection of just some of the operations we could perform. For more details see the PIL documentation.

# Resize an image
image = image.resize((width, height), Image.ANTIALIAS) 

# Convert to greyscale
image = image.convert('L')

# Blur
image = image.filter(ImageFilter.BLUR)

# Sharpen
image = image.filter(ImageFilter.SHARPEN)

Saving

Once we’ve processed the image we need to save the changes. In some instances we might just want to save over the original filename. If that’s the case we can just call the save method of image with its original filename. If we want to save the file under a different name, or as a different filetype we need to do a little more work:

	# Split our original filename into name and extension
	(name, extension) = os.path.splitext(filepath)

	# Save with "_changed" added to the filename
	image.save(name + '_changed' + extension)

	# Save the image as a JPG
	image.save(name + '.jpg')

Notice in the last example how PIL takes care of the conversion to JPG for us. All we do is provide the file extension and PIL does the rest for us.

A Complete Example

Each of the steps above can easily be put together to create a batch image processing script. Below is a complete script which creates thumbnails of all provided images, and saves them as PNG images, not matter what their original format:

#!/usr/bin/env python
# Batch thumbnail generation script using PIL

import sys
import os.path
import Image

thumbnail_size = (28, 28)

# Loop through all provided arguments
for i in range(1, len(sys.argv)):
	try:
		# Attempt to open an image file
		filepath = sys.argv[i]
		image = Image.open(filepath)
	except IOError, e:
		# Report error, and then skip to the next argument
		print “Problem opening”, filepath, “:”, e
		continue

	# Resize the image
	image = image.resize(thumbnail_size, Image.ANTIALIAS)

	# Split our original filename into name and extension
	(name, extension) = os.path.splitext(filepath)

	# Save the thumbnail as “(original_name)_thumb.png”
	image.save(name + ‘_thumb.png’)

PyRadio 0.1 is now available! PyRadio is a curses based Internet radio player that makes it really simple to play music via the console. It’s written in Python, and uses mplayer for the media playback. The full source code is available via GitHub.

I use Amarok for managing my media collection, and it comes with excellent support for Internet radio. However, Amarok can be fairly resource intensive. That’s why I developed this extremely lightweight console based radio player to make it easy to listen to music without slowing my system down.

For the next release I’m planning to support easy addition and removal of radio stations. Currently you have to edit the actual code if you want to do that. If anyone finds this app useful and would like to request a feature then feel free to leave a comment, either here or on the PyRadio project page.

Boids are a way to simulate swarm behaviour using 3 simple rules. I’d previously implemented Boids in Python, and was really impressed with the results. When I heard about the javascript vector graphics library Raphaël I thought it would be a good opportunity to port my original code to javascript. If you just want to see the boids in action you can jump right to the boids demonstration.

At the heart of the boids implementation are the boid objects. The constructor function is shown below, and basically just sets up a starting position, size, and velocity, and creates an Raphaël circle:

var Boid = function(x, y, size) {
	this.x = x;
	this.y = y;

	this.xVelocity = 1;
	this.yVelocity = -1;

	this.circle = paper.circle(x, y, size).attr({fill: '#FF0000'});
}

Here’s the boids move away function, which makes sure that the boids don’t crowd each other:

Boid.prototype.moveAway = function(boids, minDistance) {
	var distanceX = 0;
	var distanceY = 0;
	var numClose = 0;

	for(var i = 0; i < boids.length; i++) {
		var boid = boids[i];

		if(boid.x == this.x && boid.y == this.y) continue;

		var distance = this.distance(boid)
		if(distance < minDistance) {
			numClose++;
			var xdiff = (this.x - boid.x);
			var ydiff = (this.y - boid.y);

			if(xdiff >= 0) xdiff = Math.sqrt(minDistance) - xdiff;
			else if(xdiff < 0) xdiff = -Math.sqrt(minDistance) - xdiff;

			if(ydiff >= 0) ydiff = Math.sqrt(minDistance) - ydiff;
			else if(ydiff < 0) ydiff = -Math.sqrt(minDistance) - ydiff;

			distanceX += xdiff;
			distanceY += ydiff;
		}
	}

	if(numClose == 0) return;

	this.xVelocity -= distanceX / 5;
	this.yVelocity -= distanceY / 5;
}

The move closer function, which causes the boids to attract one another:

Boid.prototype.moveCloser = function(boids, distance) {
    if(boids.length < 1) return				

	var avgX = 0;

	var avgY = 0;

	for(var i = 0; i < boids.length; i++) {

		var boid = boids[i];
		if(boid.x == this.x && boid.y == this.y) continue;
		if(this.distance(boid) > distance) continue;

		avgX += (this.x - boid.x);

		avgY += (this.y - boid.y);
	}

	avgX /= boids.length;
	avgY /= boids.length;

	distance = Math.sqrt((avgX * avgX) + (avgY * avgY)) * -1.0
	if(distance == 0) return;

	this.xVelocity= Math.min(this.xVelocity + (avgX / distance) * 0.15, maxVelocity)

	this.yVelocity = Math.min(this.yVelocity + (avgY / distance) * 0.15, maxVelocity)
}

And the move with function, which causes all the boids to generally move in the same direction:

Boid.prototype.moveWith = function(boids, distance) {
    if(boids.length < 1) return
	// calculate the average velocity of the other boids
	var avgX = 0;
	var avgY = 0;
	for(var i = 0; i < boids.length; i++) {
		var boid = boids[i];
		if(boid.x == this.x && boid.y == this.y) continue;
		if(this.distance(boid) > distance) continue;

		avgX += boid.xVelocity;
		avgY += boid.yVelocity;
	}

	avgX /= boids.length;
	avgY /= boids.length;

	distance = Math.sqrt((avgX * avgX) + (avgY * avgY)) * 1.0
	if(distance == 0) return;

	this.xVelocity= Math.min(this.xVelocity + (avgX / distance) * 0.05, maxVelocity);
	this.yVelocity = Math.min(this.yVelocity + (avgY / distance) * 0.05, maxVelocity);

}

With just those simple three rules some quite complex and realistic looking swarm behaviour emerges!

View the boids in action (tested on Firefox 3).

Download the source code: boids.js.

It can be absolute nightmare if you find out on Monday morning that your web server was down all weekend, and all your potential visitors would have been greeted with an error message! This has happened to me a couple of times recently (fortunately not on this site), so I came up with an automated checker. The shell script is below:

#!/bin/bash

# Query a supplied URL and output an error message if
# the status code was not 200. Can be used as a cron
# task to check if a server is up or not.

# Ben Dowling - http://www.coderholic.com

# The URL to query
if [ $# -ne 1 ]
then
	echo “Usage: $0 <URL>”;
	exit;
fi

url=$1;
response=$(curl -s -I -L $url | grep HTTP); 

status=${response#* }; # Strip off characters up to the first space
status=${status:0:3}; # Just use the 3 digit status code

if [ "$status" != "200" ]
then
	echo “Error fetching $url. Status code ‘$status’”;
fi

I then set this up as a cron task to run every 15 minutes, and to email me if the site doesn’t respond with a HTTP 200 OK status:

*/15 * * * * /path/to/checkserver.sh "http://www.coderholic.com" | mail ben@coderholic.com

Profiling is a important part of software development, allowing you to find bottlenecks in your application and discover areas that would benefit from optimisation. In PHP a common technique is to calculate the total time to render a page. This can be achieved using the microtime function:

<?php
$startTime = microtime(true);

// normal page code here

$time = microtime(true) - $startTime;
echo "<!-- Page generation time: {$time} seconds -->";
?>

This simple approach is a a great way to find out how long certain pages are taking to load. Sometimes, however, you want more granular profiling information. That’s why I created a Profile class, which records the execution time of a given method. Here’s how to use it:

$p = new Profile();
$time = $p->profile("myClassName", "methodName", array("arg1", "arg2"));

You can also pass in an optional fourth argument, which specifies how many times to run the given method. After running it multiple times you can call the Profile::printDetails() method to view the total time, average invocation time, and the maximum single invocation time. Below is a simple example:

<?php
include("Profile.php");

/**
 * Test class just for the purposes of demonstration
 */
class Test {
	public function add($a, $b) {
		// add the numbers together 100 times
		for($i = 0; $i < 100; $i++) {
			$c = $a + $b;
		}
	}
}

$p = new Profile();
$p->profile(”Test”, “add”, array(1, 2), 1000);
$p->printDetails();
?>

If you run the above code you’ll see the following output:

Test::add(1, 2) was invoked 1000 times
Total duration: 0.0447s
Average duration: 0s
Worst duration: 0.002s

The code listing for the Profile class is below, with some of the comments removed for the sake of brevity. A fully commented version is available for download.

<?php
/**
 * Class to time the execution of method calls.
 *
 * @author Ben Dowling - www.coderholic.com
 */
class Profile {
	/**
	 * Stores details about the last profiled method
	 */
	private $details;

	/**
	 * @param classname string
	 * @param methodname string
	 * @param methodargs array
	 * @param invocations int The number of times to call the method
	 * @return float average invocation duration in seconds
	 */
	public function profile($classname, $methodname, $methodargs, $invocations = 1) {
		if(class_exists($classname) != TRUE) {
			throw new Exception("{$classname} doesn't exist");
		}

		$method = new ReflectionMethod($classname, $methodname);

		$instance = NULL;
		if(!$method->isStatic()) 		{
			$class = new ReflectionClass($classname);
			$instance = $class->newInstance();
		}

		$durations = array();
		for($i = 0; $i < $invocations; $i++) {
			$start = microtime(true);
			$method->invokeArgs($instance, $methodargs);
			$durations[] = microtime(true) - $start;
		}

		$duration["total"] = round(array_sum($durations), 4);
		$duration["average"] = round($duration["total"] / count($durations), 4);
		$duration["worst"] = round(max($durations), 4);	

		$this->details = array(	“class” => $classname,
							   	“method” => $methodname,
							   	“arguments” => $methodargs,
						 		“duration” => $duration,
								“invocations” => $invocations);

		return $duration["average"];
	}

	/**
	 * @return string
	 */
	private function invokedMethod() {
		return “{$this->details["class"]}::{$this->details["method"]}(” .
			 join(”, “, $this->details["arguments"]) . “)”;
	}

	/**
	 * Prints out details about the last profiled method
	 */
	public function printDetails() {
		$methodString = $this->invokedMethod();
		$numInvoked = $this->details["invocations"];

		if($numInvoked == 1) {
			echo “{$methodString} took {$this->details["duration"]["average"]}s\n”;
		}

		else {
			echo “{$methodString} was invoked {$numInvoked} times\n”;
			echo “Total duration:   {$this->details["duration"]["total"]}s\n”;
			echo “Average duration: {$this->details["duration"]["average"]}s\n”;
			echo “Worst duration:   {$this->details["duration"]["worst"]}s\n”;
		}
	}
}

?>

I’ve been using Linux for over 10 years now, but I ran into a seemingly simple problem today that had me stumped. I wanted to change directory whose name started with a “-” character (a hypen, or dash). Here’s what happened:

bmd /dir: ls
-folder
bmd /dir: cd -folder
-bash: cd: -f: invalid option
cd: usage: cd [-L|-P] [dir]
bmd /dir: cd \-folder
-bash: cd: -f: invalid option
cd: usage: cd [-L|-P] [dir]
bmd /dir: cd “-folder”
-bash: cd: -f: invalid option
cd: usage: cd [-L|-P] [dir]

I wasn’t having much luck! It turns out solution is to pass “--” (two hypens) as the first argument to cd:

bmd /dir: cd — -folder
/home/bmd/dir/-folder
bmd /-folder:

I think I remember having to use this trick in the past, but it’s such an infrequent problem that I obviously forgot about it. Hopefully by posting about it I’ll have a better change of remembering it for next time!

If you don’t already know about it, Dia is an open source diagram drawing application, similar to Microsoft’s Visio. It makes it really simple to create all sorts of diagrams, including UML diagrams, flowcharts, and network diagrams. It also allows diagrams to be exported to a number of formats, including PNG images (which can easily be converted to GIF images using my handy PNG2GIF utility).

Using a third party tool tedia2sql, you can automatically generate SQL schema for a number of databases, including Oracle, MySQL, Postgres, based on your database diagram. In this post I’ll describe exactly how.

The diagram

To create the database diagram we’ll be using Dia’s UML tools. Classes represent database tables, and associations between classes represent foreign key constrains.

In this post I’ll go over a simple example database with just three tables: Film, Film_Actor and Actor.

To create our database diagram we’re going to use Dia’s UML shapes. When you first start Dia it defaults to displaying the “Assorted” set of shapes, so you’ll need to change the drop down to “UML”

Then we can click on the UML class shape, which is at the top left.Once we’ve done that we should have a diagram that looks like the one below:

Double clicking on the class will bring up a properties window, which has several tabs. On the first “Class” tab we can set the name. For the first table that is “Film”. On the “Attributes” tab we can enter the rows that this table will have. Setting the attribute visibility to protected signifies that it is a primary key.

After adding all three tables and attributes we end up with a diagram like the one below:

At this point we could already generate a database schema, but it wouldn’t contain any foreign key contraints. If you’re not interested in adding them then you can skip straight to the generation part.

Foreign key contraints must be modelled using the UML aggregation tool: The line with the white diamond on one end. Select that tool, and then click on one table and drag and drop onto another table. When you let go of the mouse button a link should be drawn between the two. Double clicking on that line brings up a properties dialog.

Enter the foreign key in one end, and the row to which the foreign key referrs. The link between the Film_Actors table and the Films table is shown above, where the link is from film_id to id.

After adding a similar link between the Film_Actors and Actor table we end up with our complete diagram:

Generating the SQL

Now that our database diagram is compete we can generate the SQL commands to create the database. The command to generate the code for MySQL InnoDB is:

tedia2sql -i diagram.dia -o schema.sql -t innodb -f

If we want to generate SQL for a different database (such as Oracle) then we just need to change the -t argument. The generated SQL contains lots of comments, but the main sections are shown below:

-- Film
create table Film (
  id                        int not null,
  title                     varchar,
  year                      int,
  constraint pk_Film primary key (id)
) type = InnoDB ;

-- Actor
create table Actor (
  id                        int not null,
  name                      varchar,
  dob                       date,
  constraint pk_Actor primary key (id)
) type = InnoDB ;

-- Film_Actors
create table Film_Actors (
  film_id                   int not null,
  actor_id                  int not null,
  constraint pk_Film_Actors primary key (film_id,actor_id)
) type = InnoDB ;

alter table Film_Actors add constraint film_Actors_fk_Film_id
  foreign key (film_id)
  references Film (id)  ;
alter table Film_Actors add constraint film_Actors_fk_Actor_id
  foreign key (actor_id)
  references Actor (id)  ;

Other Dia tools

In this post I described how easy it is to generate SQL from a Dia diagram using tedia2sql. There are lots of other great third party tools to automatically generate output based on your diagram. The official Dia links page lists many of them.

I previously blogged about Ubuntu Linux Codenames, right after it was announced that the 8.10 release would be known as Intrepid Ibex. At the time I discussed possible names for the 9.04 release, my favourite being Jiggly Jellyfish. It wasn’t to be! A recent email on the Ubuntu mailing list revealed:

As we approach the launch of Ubuntu 8.10, it’s time to create space for future plans, and so I’m writing to introduce you to The Jaunty Jackalope

I’d never heard of a Jackalope before, and quickly looked it up on Wikipedia. It turns out it is a fictional animal that is a cross between a jackrabbit and an antelope. How strange!

I think this is going to be the first release of Ubuntu named after a fictional animal. Personally I think they should have stuck to real animals. What’s wrong with a normal jackrabbit?

As for the next release? I think Kicking Kangaroo would be brilliant! Based on the Jackalope release though, I’d say Kooky Kinnara is more likely.

Below is a collection of 10 great Python programming books that are available online in full, completely free of charge:

Dive into Python

This is a fantastic book that is also available in print. It covers everything, from installing Python and the language’s syntax, right up to web services and unit testing. This is a good book to learn from, but it’s also excellent to use a reference. I frequently find myself visiting the site! If you only read one book on this list make it this one.

An Introduction to Tkinter

Tkinter is a popular cross-platform Python GUI toolkit, and this book provides a good introduction with lots of examples. If you want to learn GUI development with Python then this book is a great place to start.

How to think like a Computer Scientist

This book uses Python to explain some Computer Science principals. It is full of examples, and each chapter has a collection of exercises for the reader to perform.

The Standard Python Library

This book provides a detailed description and usage examples for all of the modules in Python’s standard library. This fantastic reference is available in print.

Invent Your Own Computer Games with Python

This book is aimed at novice programmers who are interested in writing simple computer games. Each chapter describes a different game and goes through the stages of design and development. A great book for beginners, but the book lacks more advanced content and doesn’t include any information about Python game development libraries such as PyGame.

The Django Book

Django is a Python web framework (similar to Ruby’s Rails) which “encourages rapid development and clean, pragmatic design”. This book covers the framework in detail, and is a great place to start if you’re thinking about creating a dynamic website using Python. This book is also available in print

The Pylons Book

Pylons is another Python web framework. The book is structured in a very similar way to the Django book, and is worth reading if you’re looking for an alternative framework. This book is available in print too.

Data Structures and Algorithms with Object-Oriented Design Patterns in Python

This book provides a thorough introduction to data structures and algorithms using Python. This is definitely not a book for those wishing to learn Python. However, if you want to learn about data structures and algorithms, or you just wish to learn about their implementation in Python, I can’t recommend this book enough.

Building Skills in Python

This book consists of 42 chapters, each packed full of exercises designed to help you build Python programming skills. The book “includes six projects from straight-forward to sophisticated that will help solidify your Python skills”. A great book for those that already know Python, but would like to become more proficient.

Building Skills in OO Design

Written by the same author as the book above, this book follows the same format. This book focuses on object-oriented application design (using Python, of course) to help you to develop better programs.

More free books…

Above are just some examples of the books that you can find online. Back in January I posted a list of free computer science books, and I’ve managed to find lots more since then! I’ll post about them soon, so be sure to subscribe to my RSS feed.