Being an avid reader of his online stuff, and having watched various talks of his I knew the book would be well written and informative, but I thought it’d probably just repeat much of what I’d already read, without providing much new information or insights. To a certain extent this was true, the book does reiterate what’s said in many of his online articles and talks, but the book is absolutely amazing for a different reason: The code examples.

The book is extremely succinct. There’s no padding, and very little dialog to join one section to the next. There is a common thread thoughout the book though, and that’s the code. Examples of good coding practices are repeated, and functions written in earlier chapters are often reused in later ones. So although the book appears to be presenting one feature at a time with a small code example you’re actually building up more and more complex javascript applications, and the book ends with a full JSON parser!

Crockford’s coding style seems to match his writing style: succinct and to the point. There’s some really great code in the book, that I think even non-JavaScript developers could appreciate. Here’s my favourite, from the chapter on regular expressions:

var parse_url = /^(?:([A-Za-z]+):)?(\/{0,3})([0-9.\-A-Za-z]+)(?::(\d+))?(?:\/([^?#]*))?(?:\?([^#]*))?(?:#(.*))?$/;
var url = 'http://www.ora.com:80/goodparts?q#fragment';
 
var result = parse_url.exec(url);
var names = ['url', 'scheme', 'slash', 'host', 'port', 'path', 'query', 'hash'];
var blanks = '       ';
var i;
for (i = 0; i < names.length; i += 1) {
    document.writeln(names[i] + ':' + blanks.substring(names[i].length), result[i]);
}

Which outputs:

url:    http://www.ora.com:80/goodparts?q#fragment
scheme: http
slash:  //
host:   www.ora.com
port:   80
path:   goodparts
query:  q
hash:   fragment

I also love the explanations Crockford gives as to why he avoids using certain parts of JavaScript, and limits himself to a “good” subset of the language. He doesn’t simply say “This is bad and you shouldn’t use it”, it’s usually accompanied by a story about a time when Crockford had used that feature and got caught out, which is much more compelling.

I think it’s fair to say that JavaScript: The Good Parts is one of my favourite programming books: It’s succinct, packed full of great code examples and best practices for writing maintainable and bug free code. It’s a fantastic JavaScript reference, but much of what the book talks about is relevant to programming in general. Highly recommended!

After forming a group most of Friday evening and Saturday morning was spent discussing ideas around the original pitch of twitter calendar integration. We discussed a whole range of ideas including an event broker service, calendar availability widgets, and a twitter/google calendar mashup. It took until lunch time on Saturday to finalise our idea. We’d settled on a a service that would automatically work out the date of an event mentioned in a tweet and keep track of these events. There were now only 36 hours left!

I got straight to work on the code, using Django. I knew we’d need to pull in tweets and then analyse the dates. Using the python-twitter library I wrote a management command to pull in all tweets containing some specific hashtags every minute.

The next stage was to work out a date from the tweets. A Stack Overflow question suggested two options, parsedatetime library, and some pyparsing example code. I tried the parsedatetime library first, and it gave some fairly good results right away. It wasn’t so good at more complicated dates though. I tried the pyparsing example, but unless given just the date string (eg. “Next week” instead of “See you next week”) failed to work out a date at all. I did briefly investigate using the NLTK to extract the date from a tweet, but worried about running out of time I gave up on investigating further and stuck with parsedatetime.

While I’d been busy programming Guillume had come up with a great name for our service: tweevents. He’d then set about registering the domain name, setting up a twitter acccount and facebook fan page. Once we had the name sorted I setup an online logo competition offering $25 to the winner, and an hour later we had our logo.

Gabriel worked hard overnight to produce some HTML/CSS for the site, so on Sunday morning I worked on integrating that into the Django project. We worked on adding features to the website, such as links to add events to your google calendar, hCalendar markup, and the ability to filter events by twitter username, and the rest of the day was spent putting a presentation together.

So tweevents is now up and running. Within 54 hours we’d gone from a rough idea into a working website. The presentation we gave gives some details about what’s next from the business side, but there’s also some things I’d like to get done on the development side. Most importantly improved date parsing. I plan to give the python-dateutil library a try, and failing that go back to look into NLTK in more detail. Guillume’s also working hard on improving the library we’re already using. Should none of the Python options work I’ve also come across some great date parsing libraries for other languges, such as Chronic for Ruby and Datejs for Javascript. A Rails or Node.js rewrite might be on the cards! There are also features that we could add, such as Facebook event creation, or filtering the list of events to just your twitter followers, or people you follow.

A huge thanks to Damien, James and Franck who organised and helped out at the event, and made it all so much fun. Of course, tweevents wouldn’t be much without the rest of the team. Thanks also to my girlfriend, who was left on her own with our 2 month old daughter for the entire weekend!

I posted the problem to the OAuth user group but didn’t get any response. I got in touch with Yahoo. After quite a bit of back and forth we finally figured out the problem, which I’m posting here to try and save others from months of frustration!

The Yahoo guys noticed that with Python2.5 the HTTP host header was being sent through as as “fireeagle.yahooapis.com:443″, whereas 2.6 sends “fireeagle.yahooapis.com”. The inclusion of the port results in an invalid OAuth signature, because the signature is generated assuming the port isn’t included. I dug into the Python2.5 httplib code and came across this:

 813    if self.port == HTTP_PORT:
 814        self.putheader('Host', host_enc)
 815    else:
 816        self.putheader('Host', "%s:%s" % (host_enc, self.port))

In Python 2.6 the comparison on line 813 is done with self.default_port instead of HTTP_PORT, which prevents the port from being added with HTTPS requests. I noticed that later on in the code that if you pass in your own host header it prevents one being created for you:

 875     def _send_request(self, method, url, body, headers):
 876         # honour explicitly requested Host: and Accept-Encoding headers
 877         header_names = dict.fromkeys([k.lower() for k in headers])
 878         skips = {}
 879         if 'host' in header_names:
 880             skips['skip_host'] = 1

So the fix turns out to be really simple – explicitly set the http header. That’s exactly what I’ve done in my fork of the fireeagle library (see the fix). I’ve also sent a push request, so hopefully this fix will make it back into the original library. Thanks to Arnab Nandi and Anand S from Yahoo for helping to debug things their end.

The code is based on Matt Biddulph’s screenshot-tng script, but heavily modified to be more user friendly and provide more options. It uses embedded mozilla for rendering, and therefore requires the python-gtkmozembed package.

You can specify a resolution to take the screenshot at, and also a resolution for the thumbnail. When generating the thumbnail the aspect ratio will be preserved. You can also specify a delay, so that the screenshot is only taken so many seconds after loading the page. Here’s an example of running PyWebShot with 3 URLs, and the resulting images:

$ ./pywebshot.py -t 500x250 http://www.coderholic.com http://geomium.com/update/598/ http://jobs.plasis.co.uk
Loading http://www.coderholic.com... saved as www.coderholic.com.png
Loading http://geomium.com/update/598/... saved as geomium.com.update.598..png
Loading http://jobs.plasis.co.uk... saved as jobs.plasis.co.uk.png

It you have a huge list of URLs you’d like to generate screenshots for you can put them all into a file and generate images for them all with the following command:

$ cat urls.txt | xargs ./pywebshot.py

For more details and the source code see the PyWebShot project page on GitHub.

Getting Starting with Clojure

Getting up and running with Clojure was made easy to to the wealth of documentation. There’s a great getting started guide, and a guide specifically for Clojure on Ubuntu. In terms of programming environment there’s a round up of Clojure IDEs. I stuck to Vim, but I didn’t take it as far as this guide, which describes turning Vim into a fairly comprehensive Clojure IDE.

Clojure has a REPL, which I always find makes learning a new language easier. When you want to find something out just type it in a see what the result is! The default REPL doesn’t support arrow navigation or pressing up to run previous commands though, so it can be a little frustrating. There are guides on enhacing the REPL with this functionality.

One of the first things I did was put together the following shell script which either runs the specified Clojure script, or gives you a REPL if no script was specified.

#!/bin/sh
CLOJURE_JAR="/opt/clojure-1.1.0/clojure.jar"
if [ -z "$1" ]; then 
        java -jar "${CLOJURE_JAR}" 
else
        java -jar "${CLOJURE_JAR}" "$@"
fi

Writing Code

I decided to write a port scanner in Clojure, which would introduce me to command line argument handling, the network API, and parallelization.

One of the great things about Clojure is that despite being such a new language there is so much example code available on the web. I was able to find a Clojure network scanner by Travis Whitton which detailed all of the network related code I’d need to my port scanner. Travis uses Clojure’s agents for parallelization, which is something else I borrowed from his script. I was amazed at how easy it was to parallelize the lookups. So much simpler than threading. If there is only one thing I take away from Clojure it’ll be its interesting approaches to parallelization.

So without further ado here is my Clojure port scanner:

(import '(java.io IOException)
        '(java.net Socket)
        '(java.net InetSocketAddress)
        '(java.net SocketTimeoutException)
        '(java.net UnknownHostException))
 
(if (== (count *command-line-args*) 1)
  (def hostname (first *command-line-args*))
  (
    (println "Usage: scanner <hostname>")
        (System/exit 1)
  ))
 
(defn port-open? [hostname port timeout]
  (let [sock-addr (InetSocketAddress. hostname port)]
    (try
     (with-open [sock (Socket.)]
       (. sock connect sock-addr timeout)
       port)
     (catch IOException e false)
     (catch SocketTimeoutException e false)
     (catch UnknownHostException e false))))
     	<li>
(defn host-port-open? [port]
  (port-open? hostname port 5000))
 
(def port-list (range 1 1024))
 
(def agents (for [port port-list] (agent port)))
 
(println (str "Scanning " hostname "..."))
 
(doseq [agent agents]
  (send-off agent host-port-open?))
 
(apply await agents)
 
(doseq [port (filter deref agents)]
       (println (str @port " is open")))
 
(shutdown-agents)

I’m sure it is far from an idiomatic solution, so any suggestions for improvement are welcome. Running the scanner with my bash script gives the following output:

$ ./clj.sh scanner
Usage: scanner <hostname>

$ ./clj.sh scanner github.com
Scanning github.com...
22 is open
80 is open
443 is open

What’s next?

I’ve barely scratched the surface of Clojure, but I’ve certainly become more aware of some of the concepts and idioms used by the language, which is what I was hoping for. I’ll be looking for more projects in the future where I can make use of it. For now though I need to move on to another language as part of my challenge. As I’m running behind I plan to go with one that I don’t think will be too unfamiliar, either Go or Fantom. I’ll keep you posted!

The Art of Computer Programming
TAOCP is mentioned throughout Coders at Work. Some of the developers had read it all the way through, and really got stuck into the maths. Others, such as Doug Crockford had only used the book as a reference. Almost everybody agreed that this book should be on your shelf though.

Programming Pearls
A book about good programming practice with examples in C, this classic has stood the test of time.

Beautiful Code
The tagline for this book is “Leading Programmers Explain How They Think”. The book contains lots of example code, and descriptions and commentary from the code’s author. While this book is only recommended once in Coders at Work, almost all of the interviewees stress the importance of reading code, and this book can help with that.

Structure and Interpretation of Computer Programs
SICP is recommended several times in the book. It’s supposed to be a very in depth book, it looks at programming from a functional perspective, using Scheme. Zawinski says despite being “Lispy”, it provides a great introduction to programming without teaching a language.

Purely Functional Data Structures
Describes data structures from a functional programming point of view, rather than the more common imperative perspective (such as Java and C++ data structure books).

Higher Order Perl
Brad Fitzpatrick recommends this book, which is about functional programming techniques in Perl. The reviews on Amazon seem to suggest this this is an excellent programming book, and should even be interesting for non-Perl developers.

Code Complete
Jamie Zawinski describes this as “the book you wish your idiot coworker had read”. It goes through the whole development lifecyle, including design, desbugging, testing and refactoring.

The Practice of Programming
Co-authored by one of the creators of the C programming language, this book aims to teach you good programming practices to help you write faster and more maintainable code.

The list above is just a sample of some of the books recommended in Coders at Work. The Joshua Block interview also mentions lots of interesting Computer Science papers.

The interviews are really great. Most aren’t particularly technical, but it is interesting to read about how these well-known programmers work, and to get some insight into some of the projects they’ve worked on. Coders at Work is well worth a read.

Clojure

Clojure is a Lisp dialect for the JVM. I’ve done some reading on Clojure and had a play with the REPL, so this year I’m finally going to get round to writing some code with it. From what I’ve seen so far it looks quite powerful, and the integration with the Java libraries seems to work nicely.

Factor

Factor is a concatenative programming language, meaning it uses a stack instead of named variables to pass data around. Seems like a strange concept to me. This will definitely be an interesting one!

Go

Google announed their Go language in late 2009, describing it as a cross between C++ and Python. It was co-created by by Ken Thompson, who also co-created Unix and worked on the C programming language.

Haskell

Haskell is a purely functional strongly typed language. It also uses lazy evaluation. It’s got a bit of a reputation for being difficult, but I’m hoping I’ll at least be able to pick up some of the main concepts and put a few small programs together.

Erlang

Erlang is a concurrent programming language designed for fault tolerant real time applications. I’m looking forward to learning how Erlang deals with concurrency.

Scheme

I did some Scheme, a Lisp dialect, while at university. I haven’t really touched it since though, so I’m planning to revisit it and maybe dig a little deeper. I’m thinking about trying The Little Schemer book, or perhaps the online fixnum days tutorial.

Fantom

Previously called Fan, Fantom claims to be portable across the JVM, .NET CLR, and JavaScript in the browser! It’s on object oriented language with C like syntax, so there shouldn’t be a huge learning curve here. It seems to have some very interesting features though, so I’m looking forward to trying it out.

Scala

Another JVM language, Scala is a strongly typed language that “smoothly integrates features of object-oriented and functional languages”.

OCaml

Objective Caml is a statically typed, object-oriented and functional. I’m sure I’ll pick up some new techniques and ideas from this language.

Ruby

Ruby is a scripting language similar in many ways to Python. I’ve never looked at it in any detail before because I always thought it was too similar to Python. I’d like to learn it mainly to give the Rails web framework a try and to see how it compares to Django.

Lua

A small and portable scripting language, Lua is very popular in the game programming world where it is often used for the scripting of complex 3D engines.

Prolog

I did a little bit of Prolog at University and I remember being blown away by it. It’s a declarative logic programming language, often used in the field of AI.

It’s going to be a tough challenge, learning 12 new languages in 12 months. But hopefully this time next year I’ll have picked up some interesting techniques, and have a few extra tools in my programmer’s toolbox. I’ll be blogging about my progress throughout the year, posting my thoughts on these languages along with any code that I come up with. If you’re interested in the updates then you can subscribe to get them automatically. If you’ve got any comments about any of the languages or about the challenge in general then I’d love to hear them!

I have been using Python for quite a few years, but mostly for writing one off sysadmin scripts, command line utilities, and of course PyRadio. Most of my web development work has been done with PHP. The language gets a lot of bad press, some deserved and some not so much. I’ve had my own gripes, but all-in-all I’ve been fairly happy with PHP.

Several months ago, though, I thought I’d give Django a try, a python based web framework. I was completely blown away!  Compared to the PHP frameworks I’d worked with, such as Cake, it was just so much more of a pleasure to work with. So here are 8 reasons why you should give Django a try yourself if you haven’t already. You won’t be disappointed!

1. Great Documentation

The Django documentation is well written, extremely comprehensive, and up to date. The official documentation contains details API references, loads of relevant examples, and tutorials for those getting started. If that isn’t enough there’s also a whole book that’s available for free online.

2. It’s Python

The fact that it’s Python is a huge plus point for me. It’s a great language that doesn’t suffer from many of the well documented inconsistencies that PHP does, and includes some nice features such as decorators and first-class functions. Going back to PHP you soon start to miss the little things, such as the ability to assign multiple values at once, and the simplicity of slicing lists.

3. The ORM

Django’s object relational mapper completely abstracts away the database, meaning you don’t need to worry about your database schema or constructing SQL queries. If you’re using to writing SQL queries then the QuerySet API takes a little getting used to, but it’s really worth the effort. Projects like South make the ORM even more powerful, allowing you to make schema changes and data migrations automatically.

4. Built in Development Server

Where PHP really shines is on its ease of deployment. Setting up a local development server can be a bit of a pain though, especially if you’re working on several different sites. Django comes with a built in development server though, so you can be up and running within minutes! From your project’s root directory you just do

./manage.py runserver

and access your django website from http://localhost:8000 – awesome!

5. The Admin Interface

Django’s built-in admin interface is practically a full blown CMS, allowing you to add, delete or update your data. It’s pretty much all automatic, but it’s also fairly configuration. See the documentation to see what it can do!

6. Reusable Applications

Django projects are broken up into “applications”, and there are lots of existing reusable applications that you can use for your own projects, such as those for user registration, facebook integration, blogging, and many many more.

Existing applications are great, but the whole project/application distinction also forces you to think about your own project structure and therefore more likely to make reusable components that you can use in more of your own projects, or even share for others to use.

7. Templates

I’ve always been a bit dubious about the merits of PHP template engines such as Smarty. The Django template layer is great though. The inheritance model works well, and the restrictive language really forces you to have a very clean separation of presentation and logic.

8. Forms

I usually find dealing with user input one of the most boring parts of web development. It takes time to get it right, and its repetitive. The Django Form API really simplifies things. You can define your form class, include and validation rules, and simply add a few lines to your template and few lines to your view and you’re done!

So those are my 8 reasons why you should give django a go. If you’re already a django user let me know if you have any points to add!

You can view the page source to see what’s going on behind the scenes, but for full details check out the December edition of JSMag. Give it a try for yourself, generate your own tree!

One feature I miss all the time is late static binding, or LSB. The lack of LSB means that you can’t tell tell which class in your class hierarchy was invoked when calling a static method. Here’s a simple example:

class class0 {
  public static function getName() {
      echo __CLASS__ . "\n";
  }
}

class class1 extends class0 {}

class class2 extends class1 {}

class0::getName(); // -> "class0"
class1::getName(); // -> "class0"
class2::getName(); // -> "class0"

Notice that all three method calls output “class0″, the name of the base class. There is no way to tell which class the static method was called on, and in some situations that is information we need to know.

So what can we do? One obvious solution is to re-implement the method in every subclass. This doesn’t pose much of a problem for a method as simple as the one in our example, but in reality our method is likely to be much more complex, and implementing it in every subclass will lead to lots of duplication. If we ever need to make changes to the method then changes will need to be made to every subclass too. Not fun!

And alternative solution is to have every subclass invoke the parent method, passing in the class name (or whatever variable we’re interested in). It requires only a couple of extra lines per subclass, rather than repeating the whole method. Any changes now only need to be made in one place. Here’s the code:

class class0 {
  public static function getName($class = __CLASS__) {
      echo "$class\n";
  }
}

class class1 extends class0 {
  public static function getName($class = __CLASS__) {
      return parent::getName($class);
  }
}

class class2 extends class1 {
  public static function getName($class = __CLASS__) {
      return parent::getName($class);
  }
}

class0::getName(); // -> "class0"
class1::getName(); // -> "class1"
class2::getName(); // -> "class2"

In PHP5.3 we can use the new get_called_class(), so the code becomes much cleaner:

class class0 {
  public static function getName() {
      echo get_called_class() . "\n";
  }
}

class class1 extends class0 {}

class class2 extends class1 {}

class0::getName(); // -> "class0"
class1::getName(); // -> "class1"
class2::getName(); // -> "class2"