Coderholic

I came up with the following shell script recently to monitor code changes in a subversion repository. On the first run it will emails out the 10 most recent changes. After that the script mails out all changes since the last time it was run. You can set it up to run as a daily cron job which mails you all changes made to you favourite open source project!

It wouldn’t take much to get it working with other version control systems such as Git or Bazaar, or to do some nice formatting of the output instead of outputting the raw svn log as-is. Let me know if you find it useful!

#!/bin/bash
# Shell script to email the latest changes in an SVN
# repsitory to a specified email address.
# Ben Dowling - wwww.coderholic.com

svnUrl="http://anonsvn.wireshark.org/wireshark/trunk/"
lastRevisionFile="./.last-revision"
mailto="ben@coderholic.com"

function getCurrentRevision {
  # Get the current SVN revision, eg. "r4670"
  currentRevision=$(svn log "$svnUrl" -r HEAD 2>/dev/null | head -n2 | grep -v -- "-------" | awk '{ print $1 }')
  # Strip off the 'r'
  currentRevision="${currentRevision:1}"
  echo "$currentRevision"
}

currentRevision=$(getCurrentRevision)

# If we've run this program before then we've stored the SVN revision at the time
if [ -f "$lastRevisionFile" ]
then
  lastRevision=$(cat "$lastRevisionFile")
  #  Check what the current revision is, and exit if there
  # haven't been any changes since we last checked
  if [ $currentRevision -lt $lastRevision ]
  then
      echo "No changes since last check"
      exit
  fi
else
  # We haven't run this program before, so set the last revision to the current revision - 10
  lastRevision=$(echo "$currentRevision - 10" | bc)
fi

# Mail the SVN changes
svn log "$svnUrl" -r "HEAD:${lastRevision}" | mail -s "SVN changes for $svnUrl" $mailto

# Store the current revision + 1 as the last revision
revision=$(echo "$currentRevision + 1" | bc)
echo "$revision" > "$lastRevisionFile"

Today has seen the release of 4 major releases for some great software that I regularly use. There must be something special about June 30 that I didn’t know about. It makes me wish I held out on the release of PyRadio 0.2 for a few more weeks, and then it’d have been 5 great releases is one day!

PHP 5.3.0

The long awaited release of the latest version of the PHP programming language was finally released today. I can’t wait to start using some of the new language features in my PHP code, especially closures and late static binding. Namespaces are a feature I was looking forward to, but due to the awkward syntax I’m not sure I’ll be compelled to use them over PEAR style class naming conventions, which do a pretty good job of emulating namespaces.

Firefox 3.5

I’ve been using the beta version for a while, but today Firefox 3.5 was officially released. Along with lots of performance improvements 3.5 also includes lots of new features. My favorite is an improvement to the “restore previous session” dialog, where you can now select specific tabs that you do or do not want to restore.

Virtualbox 3.0

The open source visualization software was also released today. A great tool for testing sites in multiple browsers, or trying out new operating systems.

Wireshark 1.2

As is 3 major releases today weren’t enough, version 1.2 of the popular network protocol analyzer was also released today! An absolute must for anyone developing any network code. The new version includes lots of bug fixes and GeoIP integration.

PyRadio 0.2, the curses based Internet radio player which I develop, is now available!

Changes since the last release include:

• External radio stations file
• Page up/down to move up and down 5 stations at a time
• The selected station remains highlighted when you move about
• Volume up/down keys
• Additional radio stations

The latest code available via GitHub, and you can download the 0.2 release from the project page.

Thanks to everyone who left feedback and suggestions about the 0.1 release!

If you find any bugs or would like to request any features then please leave a comment.

I spend quite a bit of time over at reddit, reading subreddits such as programming (aka proggit), php, python, and compsci. It’s a great place to find new and interesting programming related content on the web.

There was a post on proggit today called the big programming related subreddits round up which listed over 30 different programming related subreddits! Far too many to keep up with, you might think. A comment was left describing an amazing reddit feature I (and apparently many others) didn’t know about: You can create a multi-reddit by combining several subreddits together. The following multi-reddit includes both the PHP and Python subreddits:

http://www.reddit.com/r/php+python

What a great feature! Here are some multi-reddits that I’m now following:

LAMP (includes PHP, MySQL, AJAX, Javascript, CSS, Linux)
Scripting (includes Python, Ruby)
Functional (includes Functional, Lisp, Scheme, Clojure)
Dev (includes Programming, CompSci, CodeProjects, Agile and OpenSource)

With the configuration option “log_errors” PHP will log all errors to the Apache error log file, allowing you to see all of the errors that have occurred while people have been using your site. Given the snippet of code below:

<?php
	// Ensure logging is enabled. Ideally this will be set in php.ini
	ini_set('log_errors', 'On');
	// Call a non-existant function to generate an error
	doesNotExist();
?>

You’ll see something like the following entry in the error log, which includes a description of the problem and the file and line number of where the problem occurred:


[Tue May 26 19:54:53 2009] [error] [client 127.0.0.1] PHP Fatal error: Call to undefined function doesNotExist() in /var/www/index.php on line 6


I’ve written a small bash script to mail me all of the PHP errors that it finds in the log. The results are sorted and piped through the “uniq” command to remove any duplicate errors. I’ve set it up to run as a cron task once a day, so if there are ever any problems it isn’t long before I know about them. The code is below:

#!/bin/bash
# Mail out PHP errors that are in the apache error log.
# Note PHP's log_errors must be turned on
# Ben Dowling - www.coderholic.com

errorLog=/var/log/apache2/error.log # Error log location
email=you-email-address@example.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat $errorLog | grep PHP | awk -F'] ' '{print $4}')
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$errors" ]
then
	echo "$errors" | mail "$email" -s "PHP Errors"
fi

If you need to it should be quite easy to modify the script for a slightly different use, such as printing out all the fatal PHP errors that are in the log.

Hope you find it useful!

If you’ve ever got to work with javascript and IE6 then you might want to read the guest post I wrote for Six Revisions: Debugging Javascript in Internet Explorer 6.

The post describes how to configure IE6 so that instead of displaying subtle and cryptic error messages when you run into a Javascript issue it will instead allow you to debug the problem with the help of Visual Studio’s powerful debugger. I’ve put together a small slideshow that includes a summary of the content:

At work we’re experiencing some fairly rapid growth, and our single production server is starting the feel the strain. I’ve been doing a lot of investigation into how we can scale the site, and thankfully there is lots of information out there.

The “Do you Scale” presentation I saw at PHP London a couple of months ago gave a good high level overview of scalability issues, and included some useful techniques to help you scale.

I think I’ve found the ultimate scalability presentation though: “Real World Web: Performance & Scalability”. The 189 slides contained within this presentation cover almost everything I’ve read elsewhere, and it’s packed full of practice advice!

MySQL has a kill command which lets you kill a specific query or connection. This is useful it situations where you’ve accidentally issued a really complex query, or an application created a persistent connection and forgot to close it.

Killing lots of connections can be tedious though. Say an application has created lots of persistent connections and not closed any of them, you’d have to go through the following steps:


mysql> SHOW FULL PROCESSLIST; /* shows all connections */
mysql> KILL <process_id>; /* for EVERY process you want to kill */


To make this process much simpler I wrote a little bash script that will kill all connections for a given MySQL user.

#!/bin/bash
# Kill all MySQL connections for a given user
# Ben Dowling - Apr 2009
# www.coderholic.com

user="root" # MySQL super user
pass="" # Above user's password
kill_user="myapp" # User who you want to kill all the connections for

# Get the connection IDs for all connections from the user $kill_user
processList=$(mysql -u$user -p$pass -s -e "SHOW FULL PROCESSLIST" | grep $kill_user | awk '{print $1}')
# Loop through all the connection IDs and kill the connection
for id in $processList
do
       result=$(mysql -u$user -p$pass -s -e "KILL $id")
       echo $result
done

The MySQL superuser account name and password, along with the user you want to kill all the connection for, are embedded within the script. It would be quite simple to pass these in as command line arguments if you wanted to though.

JSON data is commonly used as a simple way to send data back to client side javascript from the server. For example, a PHP script may output the following JSON to confirm that an action was taken:

{success: true, message: "It worked!"}

or in the case of failure:

{success: fase, message: "It didn't work!"}

Below is a basic javascript callback function to process the JSON data, displaying the returned message:

function callback(data) {
    if(data.success) alert(data.message);
    else alert('ERROR:' + data.message);
}

Sometimes it’s actually useful to send back javascript code to in the JSON data, to be run on the client. For example, if an Ajax request is made that updates some account information we could return javascript code to update several parts of the page with the new information. Here’s PHP code that outputs JSON data which includes javascript code:

$response = array('success' => 'true',
    'code' => "jQuery('#id').html('Success!');");
echo json_encode($response);
// Outputs {success: true, code: "jQuery('#id').html('Success!');"}

Note how in the returned JSON data “code” is actually a string, so it won’t be possible to run the code simply by doing “data.code()” on the client. Instead we must explicitly tell javascript to evaluate the string as code. Below is an example of how we can do this using eval:

function callback(data) {
    if(data.success && data.code) eval(data.code);
}

If we wanted to update the page with HTML more complicate the the simple “Success!” message in the previous example we must be careful to escape any quotes in the HTML correctly. If we wanted to set the content to: “Javascript”, “json”, “PHP” then we’d need to do the following:

$response = array('success' => 'true',
    'code' => "jQuery('#id').html('\"Javascript\", \"json\", \"PHP"');");
echo json_encode($response);

And It’s easy to imagine a far more complicated example. The code becomes less readable, and if you accidentally forget to escape one of the quotes, or escape of the quotes that shouldn’t be escaped, then the whole thing breaks. A nice alternative is ty store the HTML as a separate JSON property and then to reference it, avoiding the need to escape the quotes and making the code easier to read:

$response = array('success' => 'true',
    'html' => '"Javascript", "json", "PHP"',
    'code' => "jQuery('#id').html(this.html);");
echo json_encode($response);

However, to make the “this.html” reference work the clients side callback function will need to changed, as eval() runs the code in global scope. This means that “this.html” will refer to a global html variable rather than the one in our JSON data. We can get around this problem by using the apply function as follows:

function callback(data) {
    if(data.success && data.code) {
        var f = new Function(data.code);
        f.apply(data);
    }
}

And here’s a complete example which uses jQuery to perform the Ajax call:

jQuery.get('update.php', {} function(data) {
    if(data.success && data.code) {
        var f = new Function(data.code);
        f.apply(data);
    }, 'json');

Whether your a full time freelancer or someone who does projects on the side for a bit of extra cash the web is a great place to find new clients and exciting projects. Below is a list of ten of the best freelance development job sites on the Internet:

1. All Dev Jobs
Mainly web development projects.

2. Authentic Jobs
A fairly active job board that attracts some great project postings.

3. Freelance Switch
The freelance switch job board is very active, and contains a wide range of development jobs. There is a catch though: It costs $7 a month to reply to any of the postings.

4. Javascript Ninja Jobs
This is a job board for freelance javascript jobs from John Resig, the author of the jQuery library.

5. MetaFilter Jobs
Freelance development projects, mostly consisting of web and mobile application development.

6. No Agencies Please
Primarily a site for UK based work, but there are lots of remote projects. Includes a whole range of different development projects, and isn’t just limited to web development.

7. Programmer Meet Designer
A site that aims to introduce programmers to designers. Many of the projects that get listed are for skills exchanges (eg. you code my site I’ll design yours. There are occasionally paid jobs though, so it’s worth checking the site every so often.

8. Search Web Jobs
Lots of web development jobs, primarily involving CSS, HTML, PHP and MySQL.

9. Smashing Jobs
The job board of the online web development magazine.

10. Web Directions Jobs
Web development jobs. Mostly PHP and Javascript.

So that’s 10 of the best places to find freelance development work online. Checking those 10 sites every for new projects isn’t easy though! That’s why I developed the next site.

Bonus!

11. Plasis Jobs
Plasis Jobs aggregates freelance development jobs from 8 of the above 10 sites, so instead of having to check lots of different sites everyday to make sure you don’t miss out on a great new project you only need to check one! It also filters the jobs so that only freelance development jobs are shown, rather than full time jobs, or jobs for designers.

Suggestions?

Are there any freelance development sites you use that aren’t listed here? Please leave a comment!