Sunday, 28 December 2008


Working on projects or working products? Which is for you? Both provide for interesting, stimulating work with difficult problems to solve. Which are you personally going to derive the most satisfaction from? Well, I have a theory, or, a way of phrasing the question that has helped others in the past and might help you.

  • When you are working on projects you will sit down with someone with a problem. You'll get to know that person and their problem intimately and personally. Hopefully you'll then solve their problem and leave them in happier and better place. All thanks to the expertise you have imparted.

  • When you are working on products you will not have this personal connection with your customers. Instead you will attempt to imagine how all the possible customers in the world could potentially want to use your product. You'll try to place yourself in an enormous range of situations and attempt to make each of those a bit better. Hopefully, if your product is completed and a success in the market then you will have made the world a better place for a huge range of people. None of whom you'll ever know.

So... satisfaction from helping just a few people you know well, or satisfaction from helping a huge range of people you'll never know? Of course, if you do choose to work on projects then you're guaranteed to help people, whereas products succeed far less often.


Neil Gaiman

Like Wrath of a Mad God this is fantasy too: but this is a completely different proposition. This is good, very, very good. Good enough that I will recommend this to non-fantasy reading friends. Ahhh... A breakout hit - the dream of fantasy authors the world over. Well, here's a tip: instead of sucking, try writing high quality, original, funny and genuninely moving stories, like, say, this.

The poignancy. It's not cloying, there's no preaching. Not even any condescension, patronisation or pity. This is a tale of those who fall through the cracks. Those you don't notice around you; the other nation outside, in the words of Billy Bragg, sleeping in the street. A tale of the disenfranchised, the dispossessed. told so well. So clearly, so directly, with no pathetic efforts to tug at the heart strings that, for me, this became the most moving story since Greene's The Quiet American.

But it's fantasy. How can a fantasy novel seriously be mentioned in the same breath as Graham Greene? Well, I'm going to have to try to justify that. On the surface, and the back cover, Neverwhere is a fantasy adventure set in a strange, fantastical world at once beneath and entwined within everyday London. This world intersects with London through the streets and the homeless. Richard Mayhew is pulled from our world into this other place. Forced onto a quest all he really wants is to be able to return home.

Viewed as a fantasy creation, this other world is a joy. Full of magic, grand quests and the most imaginative etymologies for major London landmarks: I certainly wished I knew London better. To get the right feeling I was able to transplant Sydney in place of London. Enough wandering in the City, Surry Hills, Pyrmont and Balmain and you have the feeling that there is history, and history on history here. And beyond that, it's dark. Frighteningly, unexpectedly dark.

Like Midnight's Children though, I read Neverwhere in two ways. As well as the straight forward fantasy interpretation, you could also see this as a story told by an unreliable narrator. What if the weird, fantastical world beneath London's streets doesn't exist? I mean, not even within the world of the book? What if that entire world is inside of Richard Mayhew's mind and he just doesn't know it? And for me, that possibility made this a touching, poignant story. A story genuinely of those who fall through the cracks; into a world that is both magical, frightening and very dangerous.

Unfortunately, to make you believe I'll have to cite specifics. Without spoiling, I'd point at the third quest for the Blackfriars. When you read that scene think about alternate explanations.

Sunday, 7 December 2008

Automatic Deployment for Rails

For the Rails applications we're building at work, as well as all the standard continuous integration features, we also automatically deploy our applications. That is, every time we submit code a central server is automatically updated with a new release. Before running tests.

We're pretty happy with this set up. It's already found a couple of bugs in some plugins we're using. More on that in an upcoming post. Here's how we made our automatic deployment work. We're using Capistrano for our deployment scripts, we're deploying to Phusion Passenger running under Apache on FreeBSD and our continuous integration server runs an Ant script.

These instructions describe how to set up a Apache 2.2 web server with Phusion Passenger on FreeBSD; the Ant script to automatically deploy and how to configure a Rails app to be deployed like this.

This will give you two new environments for your apps: DEVTEST and UAT. UAT is a user acceptance testing environment, our system testers and analysts use and own this environment. We don't automatically deploy to here, we release to here. DEVTEST is the environment we automatically deploy to.

Setting up Your Server

Installing Phusion Passenger

Installing Phusion Passenger on a FreeBSD server is no different to installing anywhere else:

$ sudo gem install passenger
$ sudo passenger-install-apache2-module

Configuring Apache

At the end of the second step, the installer tells you to add some config to the end of your Apache config. On FreeBSD, edit this with:

$ sudoedit /usr/local/etc/apache22/httpd.conf

And then add the following at the end:

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3
PassengerRuby /usr/local/bin/ruby18
NameVirtualHost *:80
<VirtualHost *:80>
    ServerAlias devtest
    DocumentRoot /usr/local/www/rails/devtest
    <Directory "/usr/local/www/rails/devtest">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    RailsEnv "devtest"
<VirtualHost *:80>
    ServerAlias uat
    DocumentRoot /usr/local/www/rails/uat
    <Directory "/usr/local/www/rails/uat">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    RailsEnv "uat"

Unless you want to use two different servers for the two environments, you'll need to use named virtual hosts, and ask your friendly administrator to add CNAME records to your DNS server pointing devtest and uat at the same physical server. They'll know what you mean.

Create a Local User

You'll need a local user on your server. This is the user that will run the automatic deployments.

$ sudo adduser
Username: deploy-robot
Full name: Deployment Robot
Uid (Leave empty for default):
Login group [deploy-robot]:
Login group is deploy-robot. Invite deploy-robot into other groups? []: www
Login class [default]:
Shell (sh csh tcsh zsh nologin) [sh]: 
Home directory [/home/deploy-robot]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username   : deploy-robot
Password   : ****
Full Name  : Deployment Robot
Uid        : 1001
Class      :
Groups     : 
Home       : /home/deploy-robot
Shell      : /usr/local/bin/sh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (deploy-robot) to the user database.
Add another user? (yes/no): no

Deployment Directories

Set up the directories to hold your applications.

$ sudo mkdir -p /usr/local/www/rails/devtest
$ sudo mkdir -p /usr/local/www/rails/uat

These are the web roots for each of the environments, but applications will not be deployed here. Instead, symlinks will be created from here to where the applications are actually deployed.

$ sudo mkdir -p /usr/local/app/rails/devtest
$ sudo mkdir -p /usr/local/app/rails/uat

These last two directories, and everything under them should be owned by the deployment user you created above.

$ sudo chown -R deploy-robot:www devtest uat


Finally, there are some gems you'll need installed on the target deployment server. Some of these depend on FreeBSD ports.

$ cd /usr/ports/comms/ruby-termios
$ sudo make install clean

And then just a couple of gems.

$ sudo gem install termios
$ sudo gem install capistrano

And that's it for initial server configuration. There will be some more configuration when first deploying an application.

Preparing Your Application

Capistrano Config

Capify your application:

$ cd app
$ capify .

Edit your capistrano rules in deploy.rb. You'll want them to look something like the following. These rules use no source control system to get the code. Our continuous integration server takes care of checking out the code, so it's easier to deploy from the local code copy. And, this way we can be sure each deployment only contains one changelist.

# Overall config
set :use_sudo, false
# Application config
set :application, "app-name"
set :default_env, "production"
set :rails_env, ENV['RAILS_ENV'] || default_env
# Deployment source and strategy
set :deploy_to, "/usr/local/app/rails/#{rails_env}/#{application}"
set :deploy_via, :copy
set :scm, :none
set :repository,  "."
# Target servers
set :default_server, "localhost"
set :dest_server, ENV['SERVER'] || default_server
role :app, dest_server
role :web, dest_server
role :db,  dest_server, :primary => true
# Phusion Passenger specific restart task
namespace :deploy do
    desc "Restart Application"
    task :restart, :roles => :app do
        run "touch #{current_path}/tmp/restart.txt"

Environment Configuration

Set up the two new environments for your application.

$ cp config/environments/production.rb config/environments/devtest.rb
$ cp config/environments/production.rb config/environments/uat.rb

Somewhere inside both those files you'll need to set the RAILS_RELATIVE_URL_ROOT as the application will be running at a sub-URI on your server and Rails needs to know that. Something like:


The two new environments will also need to be described in your database.yml file. This of course depends on your specific database server setup, so I'll leave that bit to you.

Server-side Application Setup

Apache needs to know about the applications, and there needs to be symlinks from the web root to the application deployment folder. This setup only needs to be done once for each application.

To add the application to Apache, edit /usr/local/etc/apache22/httpd.conf again, and in the VirtualHost section for the devtest environment, add a line like the following:

RailsBaseURI /app-name

Now, set up the symlink:

$ ln -s /usr/local/app/rails/devtest/app-name/current/public /usr/local/www/rails/devtest/app-name

And you're done with the application configuration.

Ant Deployment Scripts

Our company has an in-house continuous integration server. We'd be too embarrassed at cocktail parties if we didn't have our own. Yes, yes, I know this is completely ridiculous. And to make it even worse, it only runs Ant scripts. Sigh. Anyway, here's how you make Ant automatically deploy an application to devtest.

In a file called definitions.xml:

<project name="definitions_rake">
    <macrodef name="rake">
        <attribute name="app" />
        <attribute name="target" />
        <element name="variables" optional="true" />
            <exec executable="rake" dir="@{app}" failonerror="true">
                <arg value="@{target}" />
                <variables />
    <macrodef name="capistrano">
            <attribute name="app" />
            <attribute name="environment" />
            <attribute name="task" />
                <exec executable="cap" dir="@{app}" failonerror="true">
                    <env key="RAILS_ENV" value="@{environment}" />
                    <env key="SERVER" value="${project.server}" />
                    <arg value="@{task}" />
                    <arg value="-s" />
                    <arg value="user=${project.user}" />
                    <arg value="-s" />
                    <arg value="password=${project.password}" />
    <macrodef name="deploy">
        <attribute name="app" />
        <attribute name="environment" />
            <capistrano app="@{app}" environment="@{environment}" task="deploy:setup" />
            <capistrano app="@{app}" environment="@{environment}" task="deploy:migrations" />
    <macrodef name="test">
        <attribute name="app" />
            <rake app="@{app}" target="db:migrate" />
            <rake app="@{app}" target="test" />
            <rake app="@{app}" target="spec" />

Ant macros, while quite insane, are generally a better way to define new tasks than the complete insanity of trying to write a whole Ant plugin in Java. These macros define low-level tasks to run rake and capistrano tasks, and then use these to build up higher level tasks like test and deploy. All these tasks assume that Ant has been run from the directory immediately above your Rails app directory.

In a file called, set your server, user name and password. Having the password here is unfortunate, but it is a local account, with limited privileges on an internal server. Your call.


In a file called build.xml:

<project name="aegean" default="build">
    <import file="./definitions.xml" />
    <property file="" prefix="project" />
    <!-- Sample application.
         To add a new application:
         1. Copy the following targets.
         2. Replace 'depot' with your Rails app name.
         3. Add the 'app name' target as a dependency of the target 'build'.
    <target name="depot.deploy.devtest">
           <deploy app="depot" environment="devtest" />
    <target name="depot.test">
           <test app="depot" />
    <target name="depot" depends="depot.deploy.devtest, depot.test" />
    <target name="example.deploy.devtest">
           <deploy app="example" environment="devtest" />
    <target name="example.test">
           <test app="example" />
    <target name="example" depends="example.deploy.devtest, example.test" />
    <target name="build" depends="example" />

The large comment block is just helpful for other developers trying to add another application. From here, to try this out:

$ ant

It should run the deployment, and then run the test suites. If that works as you expect, then just configure your continuous integration server to run Ant over that file on every submit.

Hopefully this is of use to someone. Though this is how our environment is configured, I have written this all from memory, so I might have missed a critical step somewhere. Please let me know if there's anything that needs to be changed.

Tuesday, 2 December 2008

Reading News

Previously, I've been a Google Reader fan for my RSS news reading needs. Now that I'm a proper Apple fan boi with an iPhone and a MacBook Pro, I've switched to NetNewsWire. Waaay better. The Google Reader iPhone app was what really drove me away. I'm probably going to have to turn off my blog for this, but desktop applications are frequently better than web applications. Heresy, I know. Google's iPhone Reader app has two specific problems:

  1. It refreshes the page after you close a tab. This is pretty irritating. Particularly if, like me, you only show unread items. Things disappear while I'm still reading them. Aargh!

  2. The big one: they 'mobilize' web pages. That is, instead of linking to the original version of every item Google has decided to link to a rewritten version of the item. Supposedly this version will be more readable on the iPhone. Well, the iPhone actually has a really good browser. But they've actually significantly broken something: the iPhone web browser recognises YouTube URLs and opens them in the built-in YouTube app. Because the iPhone web browser can't play YouTube movies. The rewriting means that this doesn't work. Thank you Google, thank you.

Anyway, there is one feature that I miss from Google Reader: sharing items. But there's the whole desktop application thing going on. I'm now posting items I would have shared to my Twitter feed: gga, look for items tagged #feed.

So how do I this? A pretty simple piece of AppleScript:

tell application "NetNewsWire"
        set t to title of selectedHeadline
        set u to URL of selectedHeadline
end tell
tell application "Twitterrific"
        post update t & ": " & u & " (#feed)"
end tell

A single click from NetNewsWire and I've posted an item to Twitter. If you think you might be interested in items I've previously shared, follow me on Twitter.