overwatering

Moving Home

2009-03-27T00:55:00.001+11:00

Yet another long period of silence from me. A lot has been going on. Most of it not anything I'm planning on blogging about. But, I am here to say that as a supposedly all Web 2.0 type person, I finally have a proper blog on a proper domain where I can host my own proper applications:

overwatering.org

And with that, this blog is now closed. All my posts have been migrated over, and I am now posting new writing over there. Please update your subscriptions.

As a teaser, I've got an application running there now: mutual. What does it do? Have a read.

Review Catch-up

2009-01-25T22:26:00.001+11:00

I've been falling very far behind on my book reviews. I have actually been reading, I just haven't been reviewing. And, well, once the backlog of books gets more than about four high it's pretty hard to write proper reviews.

I'm cheating. I'm going to catch up by writing short reviews of all the books I've read in the last six months or so. And from there I should be able write real reviews for books again.

Without further ado, here's six months worth of books in three sentences, or less.

On Her Majesty's Secret Service, Ian Fleming. Part of the book club, I wanted to get a feel for actual Ian Fleming Bond books, before reading Faulks'. Fun, enjoyable, if you can avoid hurling the book across the room in the first 20 pages out of frustration over the blatant misogyny. I managed - just - and found it got better.

Devil May Care, Sebastian Faulks. The actual book club book - a Bond story, set in the '60s, but written just last year by Faulks, in the style of Ian Fleming. Less misogynistic and generally offensive, but a lot less enjoyable. I frequently got bored and would put the book down, forgetting to pick it up again for a little while.

The Road, Cormac McCarthy. Another book club book - this one was brilliant, some thought it was depressing, but I found it uplifting. The ash and the grey bleakness practically leaches onto your fingers out of the page, which is nothing on the handful of images in this book that you will probably never forget. It's a fantastic book, but be warned.

The End of Mr Y, Scarlett Thomas. A potential contender for most pretentious book I've ever read, possibly even beating Virginia Woolf's Orlando, but don't let that put you off, it's actually pretty good. It a tour through literary criticism and modern physics with a significant dash of metaphysics tossed in - it felt inspired by Pynchon. Quite original though, and recommended.

One Day in the Life of Ivan Denisovich, Aleksandr Solzhenitsyn. Another book club book - chosen because, well, he'd just died. An absolutely great book, and a deserved classic, I have essentially no complaints and instruct you all to read it - it's short, funny and a very easy read. However, apparently this book inspired many in the west to embrace communism, and that I just can't see.

A Clockwork Orange, Anthony Burgess. A book club companion book, for One Day In the Life..., chosen because it was a banned book, and coincidentally it features a lot of Slavic inspired slang, without any explanation - which was actually surprisingly cool. Unfortunately, I haven't seen the famous movie. The book was a good, but a little weak.

Seize the Day, Saul Bellow. Because of all the short books, I went for another companion book - this one was a 'day in the life' story. Fellow book clubbers felt that our last two books (The Road and One Day in the Life of Ivan Denisovich) were very depressing - but this, this is depressing. Every single character is deeply detestable, not just in nature and behaviour but also in past: this is a book to attack your opinion of your life and make you doubt everything. Be warned.

Stranger in a Strange Land, Robert Heinlein. A monster sci-fi classic from the golden age of science fiction, regarded as serious, deep and important. Also, utterly hilarious, and quite irritating. The funny comes from Heinlein's sexism: he simply could not conceive of any kind of female equality that wasn't some kind of weird submissive promiscuouity. That and the long discourses on various aspects of science are also very irritating: please don't put incidental exposition in dialogue, it's trite.

Twilight, **Stephenie Meyer"". Book club again - vampire chick-lit was the required genre and this hit it. Very readable, but I was hoping that something would happen. I guess I was never a teenage girl.

The Graveyard Book, Neil Gaiman. Excellent: just the right line between a fun story and something that felt just a little darker and deeper. It's a re-writing of Kipling's The Jungle Book, though this is subtle. While it is a 'young adult' novel, read it and enjoy it, a very good book.

Odd and the Frost Giants, Neil Gaiman. A very short $2.50 novel that I read in 45 minutes. Cute.

The Virgin Suicides, Jeffrey Eugenides. Wow, one of the best books I've read in a very, very long time. It's different, it draws you in, you become part of the story; in a very engaging way. Shortly after reading I saw the movie: and also wow, a very faithful to the spirit rendering.

Still Life, Louise Penny. Wow, one of the worst books I've ever read. Seriously, this is absolutely abysmal. Murder-mystery in genre, but pure rubbish in execution. All the way through the book I had to keep putting it down to avoid the hurl-across-the-room feeling. For example, first chapter identifies the murder victim; second chapter goes back in time a couple of days, to the victim talking to a friend in a café, she reveals that she saw a crime. And then without any pretense, the description the crime is skipped. I mean, come on! Gee, do you think that could have something to do with her death? But then, in a few pages you find out what happened anyway. And! In the end, that crime has nothing to do with the murder. Christ. After this and The Blind Assassin Canadian literature is dead to me. Oh, and this was a book club book as well.

Altered Carbon, Richard Morgan. A tip: if you read something really bad, read something light that you know you'll enjoy very quickly afterwards or your brain will start to tell you that the hours you have to put into a book are a bad investment. This was a good counter: a really cool sci-fi noir story. Most interestingly, this was a novel centred around a highly socially disruptive technology, but in the window before the tech becomes ubiquitous and available to all. That window is interesting. There are also some Banks-ian characters, without quite the same detail in the characterisation, please read if you like sci-fi.

Pomegranate Soup, Marsha Mehran. Again, thanks to the book club, this was a simple story, and just plain nice. It wasn't particularly well written, there wasn't a great deal that happened and the characterisation was just plain atrocious, but in the end I enjoyed reading it and I found the story was... nice. Apart from the transparently good vs evil characters, a major criticism is the lack of direction: there are frequent, unexpected changes in direction. She almost redeems herself with a glimpse into the past of the main villain, but it just doesn't seem to go anywhere. Still, ... nice.

And the funny thing about all that? It seems to be much easier to write something about bad books than good books. That would say something the reviewer, I think. I shall work on that.

Finding Mutual Follows

2009-01-19T21:52:00.001+11:00

When you're a Twitter'er you will often be in a situation where someone follows you, and you're wondering, 'Who is this person? Do I know them?' Well, I can't answer that question for you. But, I have found that one thing that tells you about your new follower is who they follow that you also follow. Follow?

I want to be able to ask the question 'Who do we know in common?', in short. A useful question, but one that can take quite a while to answer using the web site. I asked the lazy twitterverse if there was already an app for this, but my twitterverse is too small to get an answer. So, I wrote my own script. I don't have any handy web space to run this from, so you'll have to grab it and run it yourself. You will need to install the twitter4r gem first:

sudo gem install twitter4r

Then paste the following code into a Ruby file, and run. It takes two parameters, the names of the two users for who you want to find common ground.

require 'rubygems'
require 'open-uri'
require 'rexml/document'
require 'twitter'
class Twitter::User
  def all_friends
    users = friends.map { |f| f.screen_name }
    # If there's more than one page of users, we've already got the
    # first one
    page = 2
    found_users = friends.length
    while found_users >= 100
      found_users = 0
      open("http://twitter.com/statuses/friends/#{screen_name}.xml?page=#{page}") do |f|
        users_doc = REXML::Document.new(f.readlines.join(''))
        users_doc.elements.each('/users/user/screen_name') do |friend_name|
          users << friend_name.text
          found_users += 1
        end
      end
      page += 1
    end
    users
  end
end
def in_common(my_friends, other_friends)
  my_friends.select { |m_n| m_n if other_friends.member? m_n }
end
def main(me, other)
  c = Twitter::Client.new
  me_friends = c.user(me).all_friends
  other_friends = c.user(other).all_friends
  in_common(me_friends, other_friends).each do |f|
    puts "  #{f}"
  end
end
main(ARGV.shift, ARGV.shift)

Enjoy, and please let me know how it works out for you, or if you make any changes. And by the way, *this* is why RESTful APIs rock.

The Kite Runner

2009-01-18T21:13:00.001+11:00

The Kite Runner
Khaled Hosseini

You would have heard of the movie for this one. It achieved some fame when the two Afghani child actors had to be smuggled out of Afghanistan for their own protection. Apparently, acting in a rape scene put their lives at risk. I haven't seen the movie, I'd be interested to hear what people thought of it.

This book is in two parts. The first part is a child's impression of living in a relatively stable and developed third-world, feudal country before the rest of the world decided to use that particular patch of ground as a World War-by-proxy. The first part continues with a story of poor outsiders attempting to make a new life in a very different world. This part of the novel is very, very good: it's a charming view into a destroyed world that we don't hear much about it, and certainly nothing good. Continued with a very real feeling tale of making the best of a potentially unpleasant world, and building a new life there.

My recommendation is to read this first part and then stop. I'll tell you what, send me your copy of the book, I'll remove the second part and then it back to you. Because the second part is just plain terrible.

The second part is a long sermon on how damaged Afghanistan is now. I don't have a problem with being told this. I do actually think us cozy, safe residents in front of our TVs need more confrontation of the destruction done on our behalf. And I'm not uncomfortable with placing blame: the entire first world and all the individual citizens therein, are responsible. But, oh my God! Is this ever preaching! Yes, Afghanistan is in a very bad state. We see that, we know that. The real effect and impression of the damage done comes not from preaching, but from the contrast of what we hear and see with what we read in the first part of this book. Beyond the preaching, there is also a monotonous tone and freaky coincidences to wear you.

My advice again, read the first part and then stop. Pretend the book is over. You'll enjoy it better that way, trust me.

Perhaps You Shouldn't Get Involved in Free Software

2009-01-03T00:39:00.001+11:00

Say you're a bright young kid at Univeristy and you've decided that computer science is what you want to do with your life. What should you start doing with yourself to live that dream? One piece of advice you will frequently hear is 'Get involved with an open source/free software project.' Should you?

Short answer: yes, with an if. Long answer: no, with a but.

If you want to be working in computer science, you'll probably be programming. A lot. And in fact, this is probably what you want to do. Like all professional skills, programming takes a large amount of practice before you become truly proficient. The commonly cited figure is 10,000 hours of practice to become proficient in your chosen profession. The sooner you get started on those 10,000 hours, the sooner they'll be over.

Becoming involved with an open source project is a great way of getting your practice in. You could also practice on your own project of course. But, you'll get more satisfaction from contributing to an open source project that others use. To become involved though, you'll need some level of proficiency. So hack around with your own projects; write a game; then start reading the mailing lists of an open source project you use and pick a simple bug from the tracker. There's lots of advice out there on how to become involved, I'll leave that up to you.

That's a pretty good reason to become involved with an open source project. And if that is your reason, head on out there kid, you'll do great.

That's the 'if'. Now for the 'but'.

Notice above that I referred to getting involved with an open source project? Not the open source community? If you're a particularly bright kid in your class, then please let me beg you to think deep and long before becoming involved in the community.

Why are computers important? Step outside of your beloved field for a moment and ask yourself, why do you think computers have been an important and interesting invention? Why is writing software for expensive, abstract machines to be used by the middle-class of the first world a noble endeavour? How can better computer science help people? I mean really help people. Not allow them to quickly find a cheaper price on that expensive gadget they don't need, but do the really important things: cure malaria, educate the third world, connect with strangers, fall in love. Whatever it is you think needs solving.

Personally, I do think computer science has the potential to make the world a better place. And I believe you don't need to be working in some research lab to achieve that. My definition of making the world a better place is simple: find someone with a problem, solve it, and leave them to find other problems in need of solutions. As an organic, growing, network every individual has the ability to change the world for the better by focusing on this question: where do I see problems that my expertise can help solve?

And here lies my issue with the open source and free software communities: being able to recompile the kernel to your operating system is a problem that only other programmers have. The rest of the world see computers and software as tools to solve their own sets of problems. Anytime a computer or program does not do what they want, a problem goes partially unsolved. And that particular unsolved problem is a problem you're simply unqualified to solve. You, as a programmer, can only ever indirectly help with those problems.

Lifting our heads outside of our insular worlds of programmers and computer scientists, we can see that the rest of the world has a problem with computers. These supposedly powerful devices, that the first world has invested a huge amount of resources in improving over the last 20 or 30 years just don't seem to have reached their potential. Instead, rather than constant improvement with the goal of improving the lives of real people, computing has become insular. Insanely insular. Programmers either solve problems only other programmers have, or they simply duplicate the work of other projects, poorly. Very poorly. And this is most obvious in the free software communities. In my opinion, the world does not need another poor implementation of a 38 year old operating system. The world does not need another poor duplicate of photo editing software. The world does not need another poor duplicate of office productivity software.

The world needs the potential of computers applied to new problems. Or, at the least, original solutions to already solved problems. And in my opinion the free software community, in its current incarnation, will never deliver on either of those. The free software community is tackling what they see as a moral and ethical problem: source code wants to be free. Their current solution to this problem is to duplicate every popular piece of software with a suitably free license. So entrance into the free software community requires accepting this, and then duplicating existing commercial software. To me, that sounds like a complete waste of my brain.

And if you're as bright as I think you are, then it sounds like a complete waste of your brain as well. Please, choose to advance our industry, take it in new and interesting directions. Start your own company, work for a large company. These are not immoral decisions. You will be advancing the sum total of human knowledge, you will be solving real problems of other people. People without the expertise to solve these problems. Expertise you have.

And this is without even getting into the ethics of duplicating someone else's work.

The free software rhetoric can also damage both computing and free software. Personally, I found the moral aim of the One Laptop Per Child program repugnant. But, their efforts to re-imagine what computers could be was exciting and full of potential. But, eventually all for naught as the program has been tried, found guilty and executed in the court of free software. Why? Because they chose to use a non-free (gasp!) wireless network driver and because they chose to allow Windows to run on the device. The project was found wanting, and abandoned. Why does the choice of underlying operating system matter that much? If the project believes their new approaches were of value, why not try to improve Windows? It's not as if these computers would be powerful enough to re-compile Windows anyway, even if the code was available.

You're young, you have brains, you have energy. Please think about it before deciding that the free software community is where you want to devote your energy. Do not write off working for Google, Silverbrook Research, ThoughtWorks, or even Microsoft. There is nobility there, I dare say more than there is to be found in holding back our industry.

Satisfaction

2008-12-28T18:35:00.001+11:00

Working on projects or working products? Which is for you? Both provide for interesting, stimulating work with difficult problems to solve. Which are you personally going to derive the most satisfaction from? Well, I have a theory, or, a way of phrasing the question that has helped others in the past and might help you.

When you are working on projects you will sit down with someone with a problem. You'll get to know that person and their problem intimately and personally. Hopefully you'll then solve their problem and leave them in happier and better place. All thanks to the expertise you have imparted.
When you are working on products you will not have this personal connection with your customers. Instead you will attempt to imagine how all the possible customers in the world could potentially want to use your product. You'll try to place yourself in an enormous range of situations and attempt to make each of those a bit better. Hopefully, if your product is completed and a success in the market then you will have made the world a better place for a huge range of people. None of whom you'll ever know.

So... satisfaction from helping just a few people you know well, or satisfaction from helping a huge range of people you'll never know? Of course, if you do choose to work on projects then you're guaranteed to help people, whereas products succeed far less often.

Neverwhere

2008-12-28T18:15:00.001+11:00

Neverwhere
Neil Gaiman

Like Wrath of a Mad God this is fantasy too: but this is a completely different proposition. This is good, very, very good. Good enough that I will recommend this to non-fantasy reading friends. Ahhh... A breakout hit - the dream of fantasy authors the world over. Well, here's a tip: instead of sucking, try writing high quality, original, funny and genuninely moving stories, like, say, this.

The poignancy. It's not cloying, there's no preaching. Not even any condescension, patronisation or pity. This is a tale of those who fall through the cracks. Those you don't notice around you; the other nation outside, in the words of Billy Bragg, sleeping in the street. A tale of the disenfranchised, the dispossessed. told so well. So clearly, so directly, with no pathetic efforts to tug at the heart strings that, for me, this became the most moving story since Greene's The Quiet American.

But it's fantasy. How can a fantasy novel seriously be mentioned in the same breath as Graham Greene? Well, I'm going to have to try to justify that. On the surface, and the back cover, Neverwhere is a fantasy adventure set in a strange, fantastical world at once beneath and entwined within everyday London. This world intersects with London through the streets and the homeless. Richard Mayhew is pulled from our world into this other place. Forced onto a quest all he really wants is to be able to return home.

Viewed as a fantasy creation, this other world is a joy. Full of magic, grand quests and the most imaginative etymologies for major London landmarks: I certainly wished I knew London better. To get the right feeling I was able to transplant Sydney in place of London. Enough wandering in the City, Surry Hills, Pyrmont and Balmain and you have the feeling that there is history, and history on history here. And beyond that, it's dark. Frighteningly, unexpectedly dark.

Like Midnight's Children though, I read Neverwhere in two ways. As well as the straight forward fantasy interpretation, you could also see this as a story told by an unreliable narrator. What if the weird, fantastical world beneath London's streets doesn't exist? I mean, not even within the world of the book? What if that entire world is inside of Richard Mayhew's mind and he just doesn't know it? And for me, that possibility made this a touching, poignant story. A story genuinely of those who fall through the cracks; into a world that is both magical, frightening and very dangerous.

Unfortunately, to make you believe I'll have to cite specifics. Without spoiling, I'd point at the third quest for the Blackfriars. When you read that scene think about alternate explanations.

Automatic Deployment for Rails

2008-12-07T22:31:00.001+11:00

For the Rails applications we're building at work, as well as all the standard continuous integration features, we also automatically deploy our applications. That is, every time we submit code a central server is automatically updated with a new release. Before running tests.

We're pretty happy with this set up. It's already found a couple of bugs in some plugins we're using. More on that in an upcoming post. Here's how we made our automatic deployment work. We're using Capistrano for our deployment scripts, we're deploying to Phusion Passenger running under Apache on FreeBSD and our continuous integration server runs an Ant script.

These instructions describe how to set up a Apache 2.2 web server with Phusion Passenger on FreeBSD; the Ant script to automatically deploy and how to configure a Rails app to be deployed like this.

This will give you two new environments for your apps: DEVTEST and UAT. UAT is a user acceptance testing environment, our system testers and analysts use and own this environment. We don't automatically deploy to here, we release to here. DEVTEST is the environment we automatically deploy to.

Setting up Your Server

Installing Phusion Passenger

Installing Phusion Passenger on a FreeBSD server is no different to installing anywhere else:

$ sudo gem install passenger
$ sudo passenger-install-apache2-module

Configuring Apache

At the end of the second step, the installer tells you to add some config to the end of your Apache config. On FreeBSD, edit this with:

$ sudoedit /usr/local/etc/apache22/httpd.conf

And then add the following at the end:

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.3
PassengerRuby /usr/local/bin/ruby18
NameVirtualHost *:80
<VirtualHost *:80>
    ServerName devtest.example.com
    ServerAlias devtest
    DocumentRoot /usr/local/www/rails/devtest
    <Directory "/usr/local/www/rails/devtest">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>
    RailsEnv "devtest"
</VirtualHost>
<VirtualHost *:80>
    ServerName uat.example.com
    ServerAlias uat
    DocumentRoot /usr/local/www/rails/uat
    <Directory "/usr/local/www/rails/uat">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>
    RailsEnv "uat"
</VirtualHost>

Unless you want to use two different servers for the two environments, you'll need to use named virtual hosts, and ask your friendly administrator to add CNAME records to your DNS server pointing devtest and uat at the same physical server. They'll know what you mean.

Create a Local User

You'll need a local user on your server. This is the user that will run the automatic deployments.

$ sudo adduser
Username: deploy-robot
Full name: Deployment Robot
Uid (Leave empty for default):
Login group [deploy-robot]:
Login group is deploy-robot. Invite deploy-robot into other groups? []: www
Login class [default]:
Shell (sh csh tcsh zsh nologin) [sh]: 
Home directory [/home/deploy-robot]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username   : deploy-robot
Password   : ****
Full Name  : Deployment Robot
Uid        : 1001
Class      :
Groups     : 
Home       : /home/deploy-robot
Shell      : /usr/local/bin/sh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (deploy-robot) to the user database.
Add another user? (yes/no): no
Goodbye!

Deployment Directories

Set up the directories to hold your applications.

$ sudo mkdir -p /usr/local/www/rails/devtest
$ sudo mkdir -p /usr/local/www/rails/uat

These are the web roots for each of the environments, but applications will not be deployed here. Instead, symlinks will be created from here to where the applications are actually deployed.

$ sudo mkdir -p /usr/local/app/rails/devtest
$ sudo mkdir -p /usr/local/app/rails/uat

These last two directories, and everything under them should be owned by the deployment user you created above.

$ sudo chown -R deploy-robot:www devtest uat

Gems

Finally, there are some gems you'll need installed on the target deployment server. Some of these depend on FreeBSD ports.

$ cd /usr/ports/comms/ruby-termios
$ sudo make install clean

And then just a couple of gems.

$ sudo gem install termios
$ sudo gem install capistrano

And that's it for initial server configuration. There will be some more configuration when first deploying an application.

Preparing Your Application

Capistrano Config

Capify your application:

$ cd app
$ capify .

Edit your capistrano rules in deploy.rb. You'll want them to look something like the following. These rules use no source control system to get the code. Our continuous integration server takes care of checking out the code, so it's easier to deploy from the local code copy. And, this way we can be sure each deployment only contains one changelist.

# Overall config
set :use_sudo, false
# Application config
set :application, "app-name"
set :default_env, "production"
set :rails_env, ENV['RAILS_ENV'] || default_env
# Deployment source and strategy
set :deploy_to, "/usr/local/app/rails/#{rails_env}/#{application}"
set :deploy_via, :copy
set :scm, :none
set :repository,  "."
# Target servers
set :default_server, "localhost"
set :dest_server, ENV['SERVER'] || default_server
role :app, dest_server
role :web, dest_server
role :db,  dest_server, :primary => true
# Phusion Passenger specific restart task
namespace :deploy do
    desc "Restart Application"
    task :restart, :roles => :app do
        run "touch #{current_path}/tmp/restart.txt"
    end
end

Environment Configuration

Set up the two new environments for your application.

$ cp config/environments/production.rb config/environments/devtest.rb
$ cp config/environments/production.rb config/environments/uat.rb

Somewhere inside both those files you'll need to set the RAILS_RELATIVE_URL_ROOT as the application will be running at a sub-URI on your server and Rails needs to know that. Something like:

ENV['RAILS_RELATIVE_URL_ROOT'] = "/app-name"

The two new environments will also need to be described in your database.yml file. This of course depends on your specific database server setup, so I'll leave that bit to you.

Server-side Application Setup

Apache needs to know about the applications, and there needs to be symlinks from the web root to the application deployment folder. This setup only needs to be done once for each application.

To add the application to Apache, edit /usr/local/etc/apache22/httpd.conf again, and in the VirtualHost section for the devtest environment, add a line like the following:

RailsBaseURI /app-name

Now, set up the symlink:

$ ln -s /usr/local/app/rails/devtest/app-name/current/public /usr/local/www/rails/devtest/app-name

And you're done with the application configuration.

Ant Deployment Scripts

Our company has an in-house continuous integration server. We'd be too embarrassed at cocktail parties if we didn't have our own. Yes, yes, I know this is completely ridiculous. And to make it even worse, it only runs Ant scripts. Sigh. Anyway, here's how you make Ant automatically deploy an application to devtest.

In a file called definitions.xml:

<project name="definitions_rake">
    <macrodef name="rake">
        <attribute name="app" />
        <attribute name="target" />
        <element name="variables" optional="true" />
        <sequential>
            <exec executable="rake" dir="@{app}" failonerror="true">
                <arg value="@{target}" />
                <variables />
            </exec>
        </sequential>
    </macrodef>
    <macrodef name="capistrano">
            <attribute name="app" />
            <attribute name="environment" />
            <attribute name="task" />
            <sequential>
                <exec executable="cap" dir="@{app}" failonerror="true">
                    <env key="RAILS_ENV" value="@{environment}" />
                    <env key="SERVER" value="${project.server}" />
                    <arg value="@{task}" />
                    <arg value="-s" />
                    <arg value="user=${project.user}" />
                    <arg value="-s" />
                    <arg value="password=${project.password}" />
                </exec>
            </sequential>
    </macrodef>
    <macrodef name="deploy">
        <attribute name="app" />
        <attribute name="environment" />
        <sequential>
            <capistrano app="@{app}" environment="@{environment}" task="deploy:setup" />
            <capistrano app="@{app}" environment="@{environment}" task="deploy:migrations" />
        </sequential>
    </macrodef>
    <macrodef name="test">
        <attribute name="app" />
        <sequential>
            <rake app="@{app}" target="db:migrate" />
            <rake app="@{app}" target="test" />
            <rake app="@{app}" target="spec" />
        </sequential>
    </macrodef>
</project>

Ant macros, while quite insane, are generally a better way to define new tasks than the complete insanity of trying to write a whole Ant plugin in Java. These macros define low-level tasks to run rake and capistrano tasks, and then use these to build up higher level tasks like test and deploy. All these tasks assume that Ant has been run from the directory immediately above your Rails app directory.

In a file called project.properties, set your server, user name and password. Having the password here is unfortunate, but it is a local account, with limited privileges on an internal server. Your call.

user=deploy-robot
password=deploy-robot-password
server=deployment-server

In a file called build.xml:

<project name="aegean" default="build">
    <import file="./definitions.xml" />
    <property file="project.properties" prefix="project" />
    <!-- Sample application.
         To add a new application:
         1. Copy the following targets.
         2. Replace 'depot' with your Rails app name.
         3. Add the 'app name' target as a dependency of the target 'build'.
    <target name="depot.deploy.devtest">
           <deploy app="depot" environment="devtest" />
    </target>
    <target name="depot.test">
           <test app="depot" />
    </target>
    <target name="depot" depends="depot.deploy.devtest, depot.test" />
    -->
    <target name="example.deploy.devtest">
           <deploy app="example" environment="devtest" />
    </target>
    <target name="example.test">
           <test app="example" />
    </target>
    <target name="example" depends="example.deploy.devtest, example.test" />
    <target name="build" depends="example" />
</project>

The large comment block is just helpful for other developers trying to add another application. From here, to try this out:

$ ant

It should run the deployment, and then run the test suites. If that works as you expect, then just configure your continuous integration server to run Ant over that file on every submit.

Hopefully this is of use to someone. Though this is how our environment is configured, I have written this all from memory, so I might have missed a critical step somewhere. Please let me know if there's anything that needs to be changed.

Reading News

2008-12-02T21:23:00.001+11:00

Previously, I've been a Google Reader fan for my RSS news reading needs. Now that I'm a proper Apple fan boi with an iPhone and a MacBook Pro, I've switched to NetNewsWire. Waaay better. The Google Reader iPhone app was what really drove me away. I'm probably going to have to turn off my blog for this, but desktop applications are frequently better than web applications. Heresy, I know. Google's iPhone Reader app has two specific problems:

It refreshes the page after you close a tab. This is pretty irritating. Particularly if, like me, you only show unread items. Things disappear while I'm still reading them. Aargh!
The big one: they 'mobilize' web pages. That is, instead of linking to the original version of every item Google has decided to link to a rewritten version of the item. Supposedly this version will be more readable on the iPhone. Well, the iPhone actually has a really good browser. But they've actually significantly broken something: the iPhone web browser recognises YouTube URLs and opens them in the built-in YouTube app. Because the iPhone web browser can't play YouTube movies. The rewriting means that this doesn't work. Thank you Google, thank you.

Anyway, there is one feature that I miss from Google Reader: sharing items. But there's the whole desktop application thing going on. I'm now posting items I would have shared to my Twitter feed: gga, look for items tagged #feed.

So how do I this? A pretty simple piece of AppleScript:

tell application "NetNewsWire"
        set t to title of selectedHeadline
        set u to URL of selectedHeadline
end tell
tell application "Twitterrific"
        post update t & ": " & u & " (#feed)"
end tell

A single click from NetNewsWire and I've posted an item to Twitter. If you think you might be interested in items I've previously shared, follow me on Twitter.

Dealing with Bot Nets

2008-11-26T21:24:00.001+11:00

Currently at work I'm designing a large-scale system that will be susceptible to a certain kind of denial-of-service attack. By way of analogy, imagine that Gmail didn't bother to prevent robots from creating accounts. By the time the first human went to create an account all the reasonable combinations of the top 10,000 human names would have had already been taken, by robots. This would be very irritating to all actual human users.

Our problem is much more serious than simply losing human-preferred free email addresses. But, it is a case of preventing robots from soaking up a finite resource and depriving real humans of using resource.

My approach to large system design is to always get security right first: you can never effectively retrofit it later. And the central question we keep coming back to on security is how to defend ourselves against robots. Our thinking has typically followed certain lines:

To acquire a resource, a user must prove they are human.
All users must have a registered account, so we can identify who is consuming the resource and only have to verify their humanity once: on registration.
The user's account must be protected with a password to avoid a bot misusing a real human's account.
Each account has a threshold of resource acquisition. If the threshold is exceeded than that account is temporarily blocked in some way.

At this point in our thinking we're pretty confident that we've dealt with the risk of a robot creating an account and using that single account to soak up all our resources. We're also pretty certain we've dealt with the issue of a robot creating many, many accounts, using those accounts to soak up resources while staying under the threshold for each.

But. What about bot nets? And by restricting single accounts like this, haven't we just forced attackers to use a bot net? Attackers would want to distribute a bot across the Internet. Each bot would not use its own account, instead it would use the account of the human owning the computer the bot had infected. Once the bot is on the human's computer it can easily grab the credentials, as a key logger or by sniffing around in the browser cookies. In this situation our threshold control hasn't really stopped the attacker, but it has hurt the human. The effective threshold for the human is now much lower.

And it is on this point that our discussions tend to go around and around. How can we prevent bots (who may have acquired a human's account) without negatively affecting the human's experience and without placing prohibitive barriers to use in place?

Thinking about this issue tonight, I wonder if we're not completely wrong in this argument? If a user's computer has been compromised and is now part of a bot net, should we be trying to give that user a smooth experience at all? They've been compromised, shouldn't we identify that, inform the user and then attempt to lock them out completely? There's a question there about when we can let them back in, but I'll leave that now.

My central question is, should web applications actually aggressively make the experience worse for user's who have been compromised? In the case of a bank the answer seems obvious. I suspect we're actually similar.

Wrath of a Mad God

2008-11-26T20:44:00.001+11:00

Wrath of a Mad God
Raymond E. Feist

Pure crack for fantasy geeks and about as high quality. I've been reading Feist since a friend recommended Magician to me when I was nine years old; in grade four, back in 1988. My friend's name was Paul Reid and that was 20 years ago now. It's also long since I realised that I'm pretty much only reading Feist because reading Feist is what I do.

As his books get steadily worse that becomes a weaker and weaker reason. He does have some redeeming features: he doesn't forget where he put the plot; his sagas actually finish; he manages to avoid appearing a total right-wing fascist. After the disappointment of Martin and the betrayal of Jordan those are very good things to a recovering fantasy geek. He is still one of the reasons that I haven't completely given up on fantasy. And of course, Gaiman.

Why am I now so disappointed? His first three books (Magician, Silverthorn and A Darkness at Sethanon) were really great fantasy epics. Magician even managed that rarest of fantasy firsts: a self-contained, single, enjoyable novel. What was so enjoyable? A rich, consistent, well-thought through world, with a deep and fascinating history. The sort of thing that makes Tolkein so popular. Those books sold well, Feist proceeded to mine that world and his characters in countless sequels. And like the fools we are, us fantasy fans lapped those sequels up.

You may think you want the blank spots in the story filled in, you may think that those tantalising glimpses are only a fraction of the glory that is fully formed, but hidden, in the author's mind. But. You are wrong. The back story you build, the worlds you imagine around the glimpses? Those are the real joy in fantasy. Do not burn those worlds to the ground by demanding ad reading endless prequels and sequels. Let the great stories stand alone.

Feist is a great example of this. It turns out that he didn't really have anything to surround those brief histories and as he writes more and more he's starting to change things. Sometimes for the better, but many times the things I've loved have died.

I see two things here: the world is not meant to change, even if it does make things easier for someone; and, you don't want to know your heroes too well. Even if they are only characters in a book.

The Worst Desktop Operating System. Evar.

2008-11-23T22:35:00.001+11:00

I complain a lot about FreeBSD here and on Twitter and, thankfully, I am now about to stop using that horror on my desktop. But why horror?

In the world of desktop computers, anything that is not Windows, is niche.
In that niche, anything that is not Mac OS X is niche.
In that niche, anything that is not Ubuntu Linux is niche.
In that niche, anything that is not Red Hat or SUSE Linux is niche.
In that niche, anything that is not one of the commercial workstation UNIX operating systems, like Solaris, or AIX, or HP/UX is niche.
And down there, in that niche, in that fraction of a fraction of a fraction of a fraction of a percent of the world of desktop computers, FreeBSD is niche.

From a technical point of view it actually has quite a lot to recommend it. The kernel is very well tested and reliable. For a UNIX, it has generally made decisions for correctness over performance. Something Linux certainly can't match. The user land is a consistent space, harking back through over 20 years of tradition. The ports system is a pretty good way to install and manage software.

But. In the whole world there are perhaps 15 people using it (no, not really). Anytime you Google for any problems or issues, you'll find Linux, and just have to hope that you can figure out to translate the instructions.

And this is to say nothing of the complete dearth of available software. To use FreeBSD is to always be several versions behind in Firefox. To have to compile Emacs from CVS source. To have to tweak the source code to your video driver.

FreeBSD may once have had the One True Filesystem layout, but not anymore. Linux is now nearly the king of that hill. Don't use FreeBSD as your desktop. You really don't care about how good the kernel is. You really do care about not having to compile video drivers. Worst Desktop Operating System Evar.

Still Alive

2008-11-20T23:09:00.001+11:00

Yes, this is one of those irritating posts. Where a blog that you thought had quietly retired suddenly reappears with a post. A post that says basically nothing. A very self-indulgent post just promising that there will actually be real work worth reading reappearing soon.

Why couldn't the blogger just leave us all in peace? Why this attempt to appear that he hasn't just gotten bored or too lazy to update? Why this empty post tantalising and teasing with a promise; only to disappoint with more deathly silence.

Yep, this is one of those posts.

But! I actually do promise to post something real soon. No! Really!

And, in a desperate attempt to appear trustworthy, here's a short overview of what's been going on.

Switched from the horror of FreeBSD I now have a brand new MacBook Pro as my primary computer. After nine years I'm finally being paid to use the platform I stayed loyal to throughout the dark years. Hopefully the new computer, well set up, will actually help me write more here. It got me writing this.
New project. Can't talk about it. Cool though. Has inspired some general problem solving that I can talk about though. There will be some technical recipes on here for the first time.
Briefly had a fish tank on my desk at work. It was very nice. The tank did well, but then I had to move desks. Probably worth doing, but you'd want to be more sure of where you were sitting.
Joined a book club. Read quite a few books. And yep, that means reviews. There will be some of those coming soon.
Still annoyed at various parts of my industry, enough to rant.

Hopefully, all that and more to be posted.

On the Nature of my Damage

2008-09-01T22:19:00.001+10:00

Recently I have realised that at a very early age my attitudes towards and interactions with computers were permanently damaged. Like all geeks, I first started programming in primary school. And like many geeks my age, the first computer I had to program was an Apple //e. My Dad had lots of books for the Apple //e, so I had a lot to work through. But once he got a Mac, I wanted to program that as well.

The seduction of more power, I guess.

Well, times were tough in the Northern Territory: the only book I could find even vaguely on programming the Mac was The Apple Human Interface Guidelines. The original edition, by Tog, from the mid-1980's. That was it. And this was, of course, long before general Internet availability.

What was I going to do? That was all I had, so that's what I read. Cover to cover. Twice.

The Apple HIG is a somewhat unusual technical manual. Instead of just documenting all the available possibilities, dispassionately and exhaustively, this book took a very firm position. There was a right way to do things and things must be done the right way. The HIG then set out to list the right ways and the wrong ways, with justifications.

This preaching about the true path was both low-level and high-level: as well as detailed instructions on how to place and label buttons, it was also about how to design whole programs for the smoothest and most consistent interaction with the user.

And there was installed my damage. That book didn't just encourage good UIs, it demanded them. And now it seems that I demand a lot from computers. Computers shouldn't be hard to use, in fact we shouldn't even notice that we're using them at all.

Now every time I have to do something just so the computer knows what's going on (like 'Save') or I have to jump through a hoop because it's easier for me to jump then the programmer to write their software well, I feel a deep sense of annoyance. It doesn't have to be this way, dammit! Computers are meant to free us from drudgery to allow us more time to do the things we enjoy. Or, more cynically, the jobs we're more efficient at. Either way, doesn't matter to me. But, most of all, computers don't have to be this way. It isn't that much harder to do the right thing. We could do the right thing in the 1980's; we can do the right thing now.

As a programmer I could be frustrated and demoralised by the state of my industry. Maybe later. For now I choose to rant and rail against this, and fight. Much to the endless delight of my highly fortunate colleagues.

Where have all the photos gone?

2008-09-01T21:33:00.001+10:00

I've stopped posting my photos on my blog, they go straight to my Flickr.com feed now. It's just easier to post to and have the photos look reasonable. Sorry Google, but Picasa just isn't there yet!

Anyway, here's my feed: overwatering.

And here some recent sets of photos that I kind of liked. Follow the photos for larger sizes and the rest of the sets.

I will post other links to some other photos I like in the future, but if you're interested, probably best to subscribe to my Flickr feed.

after the quake

2008-07-13T19:11:00.001+10:00

after the quake
Haruki Murakami

I'm in a book group again and this is our first book. Funnily enough when we all brought our picks to the first gathering there were two Murakami suggestions - the other being A Wild Sheep Chase. We chose after the quake as our first book (it was short and a short story collection - a slightly commitment-phobic book group) and A Wild Sheep Chase was pushed to the end of the list with a strong suggestion to find a substitute. And now the suggester has left the group! Oooh - scandal!

For all that after the quake was fantastic. It's a collection of short stories each following a single person's life after the Kobe earthquake. None of the characters lives were directly affected by the quake: they didn't live in Kobe, they apparently didn't lose anyone from their lives - but for each of them the quake was there, this huge background event that has shuddered through them all.

The writing is spare, brief, highly evocative and, ultimately, beautiful. Reading this very short collection was an unusual reading experience: it was relaxing, peaceful. There was no urge to understand what was going on, to read deeper - there was just a peaceful journey. Apparently Murakami is to be read very literally and that's how I saw this. It seems to be full of allegory and deeper intent, but I don't think that's what we're supposed to read. It felt like a series of beautifully told stories about ordinary people. People whose lives had been massively disrupted - even though nothing actually happened to them. And thinking on that, there is a strange undercurrent of guilt: as if they should not be feeling pain while there is so much suffering on TV.

I have a theory that there is something that connects together all the stories told in this book. An earthquake is a sudden event following a long build-up of pressure, after the quake the seismic fault lines settle into a new state, one that is hopefully more stable. Unfortunately, for us, it requires this sudden release to jump to the new state. This is reflected in all the stories: the characters' lives were flowing along and suddenly the earthquake kicks them into a new state. With an upheaval of their life. The book as a whole is tied together by the final story, where the characters end up living the life they had always intended. It may sound corny, but hope from the change. And, as it is told quite subtlely, both in message and style, you don't feel the urge to cringe.

Some final comments: I read this immediately after Midnight's Children, the difference in style was very striking. Throughout the book group this was a hit. Even those who initially skeptical (due to cat torture, or overly trendy covers) were won over. I'd recommend it, but don't expect to be grabbed by the collar and hauled on a ride. This is a slow, contemplative book. Read for the enduring feeling of peace.

Blink

2008-07-10T23:18:00.001+10:00

Blink
Malcolm Gladwell

This book is just plain cool and it's actually hard to say precisely why. Humans think and make decisions very quickly without knowing we do this, or even understanding how we can do it.

There are two immediate rammifications:

If you know a field well, and I mean very well. If you've studied it, trained in it, worked and lived in it, then your snap thought process, your 'Blink' is very valuable. You should trust it.
If this isn't your field of expertise though, your brain will find something to react to, some stereotype you aren't even aware of and react to that. Frequently, that stereotype will be "I don't like that because it's different." In these cases your 'Blink' will lead you wildly astray. Don't trust it - it's hard, but dig deeper and take time.

A major flaw may have occurred to you: if you can't understand these instant reactions, how do you know which one you're having? Well, if you're honest with yourself, of course you know. Either you have studied something, or you haven't.

But that doesn't work well for the softer skills like reading people. Every thinks they're good at reading people.

And there's the Dunning-Kruger Effect waiting to bite.

So what can you do? Well to start, read this. It's a truely fascinating study of people and how we think. And, being aware of the decisions you make without thinking is actually a pretty powerful antidote to those times it leads you astray.

You've just met someone. He seems like a pretty good guy and you like him. Your powers of rationalisation will tell you that you like him because he seems confident but easy-going. You also liked his mildly self-deprecating introduction. And if this is social, great! Just go with it! But, if this is an interview and you're on either side of the table, stop and ask yourself. Is that all true, or do I just like him because he's tall?

Seriously. Read the book. Gladwell also wrote The Tipping Point which I will be definitely be reading.

Steve Jobs & the JesusPhone Will Save Us

2008-07-10T23:06:00.001+10:00

Clearly Steve Jobs and the JesusPhones is the ultimate name for a band.

We've had mobile phones in our lives for quite awhile now. First they were enormous, and only tradesmen had them. Then they started to get small, really small. So small you couldn't use them. And then they got bigger again: now swelling with countless features. Torches, cameras, pedometers. Some of the features stayed, but not many. Next was email, and that's been pretty popular. The Internet made its way onto our phones as well, but like video calls didn't really go anywhere.

This Friday the iPhone will launch in Australia. And predictably people are going crazy. When was the last time you knew the launch date of a mobile phone ahead of time? Sure, most of the hype is because it's Apple and everyone loves Apple and isn't it so gorgeous and stylish and Oh My God I've just got to have one. Deep breath. But is there something else going on here?

The core function of a mobile phone is making phone calls. Well, yeah. But there have been countless other features rammmed into them. Haven't some of these taken off as well? Yes. There is one that is on every phone in Australia, most of the phones in the world and used by the overwhelming majority of mobile owners; in some demographics more than phone calls: SMS. But SMS is just a very limited single-person to single-person version of online chat. AOL first released Instant Messenger back in 1997 and it's been huge ever since. IM, with presence, blocking, buddy lists, group chat, location mobility is a far richer chat experience than SMS. So why don't you, yes, you reading this post, have an IM client pre-installed on your phone? Why hasn't SMS gone the way of SIM card addressbooks (remember those?) and been completely replaced by IM?

Firstly though, why is that an interesting question? As I said, we've been carrying mobile phones for a long time. And in that time phones have progressively become more and more powerful. Sure, they've lagged in the power stakes behind standard computers, but I think you'd be surprised by how little. The original iPhone was equivalent at release to a four year old Mac laptop. Four years! I was writing interesting software (including a chat system) on 18 year old Macs! So clearly phones are powerful enough. How come then, given that we have these mini-computers with us more than our real computers there aren't interesting applications for them? How come it's still phone calls and SMS? This is expecially frustrating as these powerful devices have permanent connections to the Internet, everywhere! Something I could only dream of when I was first writing software 15 years ago!

People have tried. Shrinkwrapped application developers, vertical integrators, shareware developers have all tried to make a living writing software for phones. And one by one they've given up. And after much thinking the industry as a whole has come up with a batch of reasons why there has been no success. And a lot these reasons boil down to there is no killer app. There isn't one thing that people want to do with their phones other than make calls or send texts. And I bought that line too. Until I thought of SMS and IM.

So why no IM? Well firstly, you are not Nokia's or Ericsson's customer. You are their product. Telstra is their customer and you are being delivered to Telstra so Telstra will buy mobile network gear off Nokia. Interesting. It may not be true any longer, but Nokia used to make more off that gear than their phones. The phones were a loss-leader to drive sales of equipment.

Why the 160 character limit on SMS? Because SMS messages are squeezed into a gap in the control sequences that phones exchange with the towers to remain connected to the network. In other words, SMS messages are sent anyway, all the time, even if you haven't put anything in them. They are just part of the network! So why do the telcos charge 25c per message? Because they can. Oligopolies are cute like that.

Imagine how many text messages are sent every day. Think about how much is charged per-text. All of that income is pure profit for Telstra and the other telcos. That is an enormous, uncontaminated by overheads revenue stream. That kind of revenue is addictive. And here is the crux of the problem with mobile phones: the telcos became addicted to their existing revenue streams and then, with the handset manufacturers as their willing accomplices, set to work on completely controlling and stifling mobile phones as a platform.

Writing applications for phones is incredibly difficult. I don't want to go into the problems here but the two main issues are the half a dozen different platforms with inconsistent implementations of the same platform across devices and end-user distribution and installation are essentially impossible. This situation did not happen by accident though. The telcos strongly encouraged this situation to emerge. Why? Because they are terrified of just becoming a utility that can only charge for data flowing down the pipe. It may be too late, but this was a very short-sighted fear.

Apple and the iPhone are changing this world. Not because Apple are out to save the world, not because they only care about the user experience, not because their phone is pretty. Nope, that's all hype. The iPhone changes things because for the first time, you the phone buyer are actually the customer of the handset manufacturer. Apple is not trying to sell network equipment, Apple is trying to sell phones. And they decided that to sell phones the phone has got to have a great browser. And the ability to install other applications. And somewhere to buy those apps from.

You are buying the iPhone and you're liking it. Or you're not buying it, but those particular features sound pretty good. Why can't my Nokia have those? And pretty soon the telco's worst fear is realised: they are just a pipe through which we ship packets. And I can guarantee when that happens that 160 characters worth of IM conversation will cost a lot less than 25 cents. Try 0.03 cents. That's 833 times less! At today's rate, no demand discount applied!

So, relegated from giants of the economy to the likes of water and sewage for the telcos. But, it didn't have to be this way. As well as providing the network, telcos had something else: a billing relationship with the consumer.

What if when browsing Amazon on your phone when you bought something you didn't have to enter any credit card details? Instead the web site communicated directly with your phone, used a rolling key from there to sign the invoice and then billed it straight to your phone bill? Gee, sounds pretty convenient to me. And a hell of a lot more secure than handing out credit card details. This can only work with phones, and telcos have only a short window remaining to make this happen before something else comes along. They had their chance to replace the credit card companies. But because of their addiction to their immediate (but ultimately doomed) revenues, their willingness to screw their customers and stifle an entire world of technology for almost two decades they appear to have done themselves out of a future.

I, for one, shall not mourn their passing. And do not mourn for Nokia either. Brainless henchman is not a noble calling.

Midnight's Children

2008-07-06T22:39:00.001+10:00

Midnight's Children
Salman Rushdie

Dense, detailed, loud, intense and, in a way, unrelenting. The world is swirling around you and you've got no idea where to look but you want to look everywhere right now! I've never been there, but this book is what I imagine India is like. I don't think that's unreasonable either as Rushdie seems to be wanting to tell the story of India's birth as a country.

This is another of those literary 'magical realism' novels that I find much easier to describe as fantasy. There is definitely a lot of apparent fantasy in here, but the story has much more to it than those parts.

For me, possibly the most interesting aspect was the realisation that Saleem Sinai was an unreliable narrator. This was just a suspicion at first, he was so desperate to defend everything as true that I started thinking he doth protest too much. And once that seed was planted it became easy to read everything too ways: all the fantasy that Saleem claimed could be explained entirely prosaically.

So I read the book with two interpretations. I don't know which is true, but I do know that for all his transparency Saleem is one of the more intensely realised and interesting characters in fiction.

Ahh, audiophiles

2008-07-06T20:28:00.001+10:00

I've always enjoyed audiophiles; it's pretty hard to find a single group with so much rich potential for mockery. But, through all my laughter at their talk of high quality digital cables (they haven't heard of error correction perhaps?); through all the sniggering over their detailed discussions about bit rates when the Nyquist-Shannon sampling theorem is a mystery unto them (What? Perhaps the CD sound frequency of 44.1kHz being approximately twice the typical highest human-audible frequency is a coincidence?)

Anyway, for all that I've always just thought it was funny: Ahh, aren't they cute? No knowledge of information theory at all, but here they are arguing about transmitting bits. Still cute though. Just a geeky hobby, kind of like theology. Theologians and audiophiles arguing about things that aren't really going to have any effect on their lives, that they don't understand, and in the end are all indistinguishable.

And I've always assumed that on some level audiophiles knew just how ridiculous they were. They'd never admit it, but there was always something in there that would prevent them from doing something really stupid. But, no!

Behold! The $500 Ethernet Cat-5 cable! And it's not even blue, like a proper one! And they're available used! Some idiot actually bought one of these!

Oh, and please, please, please can an audiophile attempt to defend this? I won't respond, but it's always amusing to listen to.

Time for a New Desktop

2008-05-24T11:45:00.001+10:00

I'm now in the process of switching my main desktop from Windows XP to FreeBSD. And man, all I can say is that if you're still on Windows or a Mac, now's the time to go Free baby!

My Windows desktop has two DVI LCD displays, and a Microsoft (hiss!) Natural Keyboard. I'll still be using Windows reasonably often but I don't want to be rearranging keyboards, mice and monitors on my desk constantly. An extra monitor on my desk, and a USB-DVI KVM switch, and all that's nicely arranged. Sure, when I switch to FreeBSD sometimes the mouse is all messed up: moves the wrong direction, won't send clicks, buttons confused. But this doesn't happen often, and if I just switch back to Windows a couple of times, it sorts itself out.

After I'd recompiled and installed my editor, and installed some fonts from source, the next thing to get right were having the two screens display a different image: they were defaulting to clone mode (same image on both screens.) That was pretty easy to fix up in the end.

Oh, but before I did that there was also the kernel that needed to be recompiled, because I needed to be able to run Java. Sure, I've only got JDK1.5, but that's pretty close to up-to-date, and I really don't know why you'd need anything more recent.

Somewhere in here I had to get some assistance from the sysadmin. While installing those fonts up above, one of the packages had complained that an existing package hadn't been installed correctly, it very helpfully told me how to de-install and re-install it. Nice. That package had a 'delete' dependency on KDM, the login window for KDE, and then on re-install KDM didn't come back, so I had to get the sysadmin to reinstall KDE. It hardly took him anytime at all. People who design packages really should get their dependencies correct.

So, I've got my desktop working again, my editor running, and some nice TrueType fonts available. Emacs can't see any of those TrueType fonts, but I'm sure with a few more recompiles with different configure options it will all come right. The one remaining problem was getting FreeBSD to use both screens in something other than clone.

Configuring Xinerama wasn't the right way to go: the web recommended that and a colleague already had that working. But it wasn't right for me. I had to download the source code for my video card driver from nVidia (after I found someone with the same machine as me, running Windows, and asked them what video card they had.) I hacked the source to the driver, as it complained that FreeBSD 7.0-CURRENT wasn't supported, although I was running FreeBSD 7.0-STABLE. A few small changes to the source and it compiled straight away. Then I ran the nVidia config tool, and restarted. That very nicely controlled both monitors by turning one off and the other on. But right there on the nVidia site were instructions on how to enable TwinView. Perfect! After I'd rewritten /etc/X11/Xorg.conf based on their instructions and what I knew about my video card and monitors, I restarted again.

And... perfect! Two independent monitors that I can arrange windows on to my heart's content!

So, I really don't know what you're waiting for, you really should dump those Macs and Windows computers and switch straight to FreeBSD! You have no idea how good the kernel is. Sure, it isn't as good as Solaris, but it's miles better than Linux.

The Mythical Man-Month

2008-05-12T20:53:00.001+10:00

The Mythical Man-Month
Fred Brooks

Everyone knows this book; everyone knows the core points and Brooks' recommendations and laws, even if not everyone has read it. This is one of the few true classics of computing. I'm not going to waste anyone's time repeating those assertions.

The Mythical Man-Month is now, unfortunately, hilariously anachronistic. And, the anachronisms are starting to damage the book: the core ideas are getting buried beneath 40 years of development technology advances. Engineers each get their own computer (or even two!) now, we don't need to share debugging time anymore. Surprisingly, I'm a little hesitant to recommend this book now. Beneath the anachronisms there is plenty of good advice: the point he tries to make about planning your debugging time and keeping track of what happened afterwards still applies, for example. But, you have to be prepared to dig, to see through all that to what he's really trying to say. If you do choose to read, skim the original parts and dwell more on his 20^th anniversary additions:

There are two things I will say:

'Build one to throw away' is wrong. Brooks comes out very clearly against that, even though he originally popularised it. Don't do it, plan to prototype and grow organically. This suits me just fine and leads to:
Brooks is the original agilist. Time and time again the things he values are competent, pro-active people and high-visibility, high-efficiency, fast-turnaround development processes. The corner stones for the agile family of methodologies.

Dreaming in Code

2008-04-09T22:44:00.001+10:00

Dreaming in Code Scott Rosenberg

Rosenberg is one of the co-founders of the online magazine Salon.com, a magazine I've been reading on and off since 2000. After having a bad experience with internal software development, he became interested in how, after more than 50 years experience, we still find, in the words of Donald Knuth, that 'software is hard.'

In the interests of disclosure, I will point out that I got a copy of this book for free from the author after he offered copies to bloggers if they agreed to mention the book. So, this is me (gratefully, happily) fulfilling my end of the deal.

This book is a story, a story of a collection of great developers attempting to write an amazing piece of software. It is not a retrospective story either. It begins shortly after the project has started, but unfortunately couldn't follow the project to completion: the project still isn't done yet.

The Open Source Applications Foundation (OSAF) was formed by Mitch Kapor (of Lotus 1-2-3 fame) to build a new form of personal information management software: Chandler. Dreaming in Code tells the story of how they tried and (so far) failed. With many informative diversions into the theory of software engineering in an effort to discover why software seems to so consistently be delivered late (if at all) and buggy (if it even gets close to meeting the original vision.)

Rosenberg does an impressinve amount of research into the theory. No appropriately read professional software engineer will find any revelations here: this is all stuff you should already know. But Rosenberg is not claiming to invent or discover anything new. In fact, he goes out of his way to disclaim that there is any original research contained in this book. He is a journalist, and as a journalist he has produced a detailed literature survey. The summary in two well-written chapters is useful even for experienced software engineers. I'm sure non-software engineers will find this all very interesting; assuming they are interested in how software gets written.

For software engineers, there is something very interesting here. The internal mechanics of a a team building a piece of software is a very secret thing. Companies are secretive and companies and open source projects want to protect their reputations. Most software engineers only ever see how a project they are working on proceeds, and then they're too close, plus there's no nice summary of what happened. Dreaming in Code is something very valuable to our field: an accurate story of the human side of a software development project. With both the clarity of distance and the accuracy of events recorded at the time they happened.

It was simply astounding how familiar this story was. OSAF and Chandler get some things spectacularly wrong, but then in other cases they do things very right. It's easy to point at the things they stuffed up and claim that you would never make those mistakes, but it's a little too easy to ignore the things that were done right. The good decisions end up forgotten and never noticed.

So what did I think that got wrong? Firstly, and by far the biggest: Analysis Paralysis. This is a mistake that I've seen projects make over and over again. In fact, I'd go as far as to say that it's more common to see this affliction than not. Projects just can't seem to make a decision, stick to it and then start building. The fear of the future locks them solid: what if we make the wrong decision? What if someone blames me for the wrong decision? In the end, ikf the decision wsas so egregiously wrong that it can be traced back to just one person, then everyone else around at the same time is just as culpable for allowing that decision to happen. Everyone, please just get in the habit of making decisions. Rely on those around you to spot a bad direction: that's what they're there for.

Second big mistake for Chandler? People. No surprises there, it's the big and obvious disaster. If you believe that software is hard and you care about your software then clearly you should only work with the best. That's easy to say, but pretty hard to achieve. And in Chandler's case the people problem exhibited in a couple of interesting ways. Before I talk about the people problems that I saw Chandler as having I need to say that any attempt to judge is based solely on what is described in this book. I (obviously) didn't work on this project, I don't know the people, there's only so much I can say. Having said that, I'd also like to say that people problems kill most projects and if we hope to advance we need to get over this fear of talking about the problems with people. Maybe then we can find some solutions. Or maybe that's just an overly analytical geek talking. Why can't everyone just be a nice reducable puzzle, dammit?

There was a permanent employee of the OSAF who was hired quite early and ended up in quite a senior technical position, and this employee was unfortunately the very soul of Analysis Paralysis. Any story about some interminable technical discussion has this particular employee at its heart. He was extraordinarily conservative and wary of making decisions, but ultimately many of the technical decisions came down to him. After a few of these stories, I was left shouting 'Do something about him!' at the book. And I've seen precisely this problem face-to-face too many times to ignore. Once again people, make decisions! You probably won't get it too far wrong. In fact, in this case, the inability to make a decision led to one of their few definite cases of over-engineering.

The second people problem: OSAF planned to run themselves as an open source project that encouraged volunteers. Predictably enough, the only people who could volunteer for any extended period tended to be ex-Apple and -Netscape employees who had already made their fortune and no longer needed to work for an income. From the outside these volunteers presented an interesting problem. They were all brilliant and had done amazing work in the past, but they didn't see this project as any kind of meal ticket. There was no drive for them to get this project finished and out the door. In the end, they come across as people partially on the side-lines, commenting endlessly but never really pushing the project forward. Instead, they were offering endless advice on how things could be done better. Software projects are always cursed with people like this; encouraging volunteers just seems to guarantee it.

But back to the book, I think I've already made clear that I really enjoyed it. I also think the opportunity to see inside another project is infinitely valuable for software engineers and software engineering. Aside from all that, I will say that sometimes the brief newspaper style of the book was a little irritating. On occasion I felt a topic or anecdote could have done with some more depth before moving on.

Software engineers will get a lot out of this; non-software engineers who care about how software (upon which, our civilisation is built) is written will get even more out of this. Oh, and as a professional programmer and wannabe-amateur writer-slash-blogger, software is definitely written - then organically edited into shape.

To finish on a positive note, something Chandler got right? Python. Choosing to write their software in an expressive high level language was clearly a win. There is no question now that Python is fast enough for desktop software, and that's really the only doubt. Hopefully Chandler can be used as an example of choosing better languages.

If you choose to read this, or not, in the end people! Make a decision!

Atonement

2008-04-07T20:32:00.001+10:00

Atonement
Ian McEwan

This is simply an excellent book. It doesn't go in for any special literary tricks, there's no special effort to make some obvious point: it's a really good story, told very well. There's some intimations of other layers, but feel free to ignore those. One thing that this book does pull off is an unsympathetic main character who I actually managed to not hate in the end. I didn't want to hurl the book across the room; always a worthwhile achievement.

A couple of points: I always find it vaguely amusing to see novelist characters in books written by professional novelists; even the best write what they know. The characterisation and the imagery are what really grabbed me. I was there on that hot, summer day in 1935. I knew Robbie Turner, and I knew Cecilia?Tallis.

There is much that could be said about the effect of fantasy and the blurry line between a clearly seen artificial vision and reality, especially in relation to the powerful imagery that provides this story.

But instead, I'll just say this is a great novel, read.

Computers Hate Me

2008-04-06T21:12:00.001+10:00

It's true, they do. Possibly something of a disadvantage in my chosen career, but I get by, carefully. Don't believe me? Hear my tales of woe, come cry with poor, poor me.

April, 1997 - Still at University, just started my second year of a computer science degree. I save up the cash and buy the first computer of my very own: a Performa 6360. I get it home; I set it up, including copying all my work off the family computer: there must have been 80 megs of data! I'm about to go downstairs and delete everything off that shared computer, whe I stop. Nah, I'll do that tomorrow. I shut my brand new Mac down, I go to sleep. I wake up in the morning and one of the first things I do is turn my new Mac on. To be confronted with the dreaded flashing disk icon. My Mac couldn't find a disk to start from. Uh oh... Even booting from a system CD showed nothing. In the end this wasn't even a disk crash, there was a bug in the disk driver. It completely lost everything meaningful off the disk. Nice. Good thing I hadn't deleted my backup. Words to live by.

July, 1998 - For our third year project we decided to write a TCP peer-to-peer IM system for Apple's up coming new OS: Rhapsody. The beta didn't run on my 6360, so I sold that and bought a Power Mac G3, one of the original (or so I thought) beige ones. Turns out it wasn't quite 'original' enough: I scored a motherboard rev that wouldn't boot the developer seed of Rhapsody that I had access to. Argh! We still got our IM system working: we wrote it using the cross-platform environment Apple released for Windows NT. Everyone else in the class wrote Access databases.

March, 2004 - Time to finally upgrade the now ancient G3, so I order an iLamp G4. It arrives, complete with a nice line of purple pixels all the way down the screen. Fortunately, it was declared DOA and a complete replacement was sent.

Hmmm... all Macs so far. Why do I keep buying these?

July, 2005 - We've now started a startup. We know .NET, so that's what we're writing it in. I buy my first Windows PC - a Compaq Presario. It came with XP Home, so I buy an upgrade to XP Pro at the same time. At home that night, running the upgrade - and it just stops. No upgrade for me. And even better, it deleted the old XP Home installation, leaving me with an unbootable PC. Sound familiar? I got my computer back with a clean installation of XP Pro. Except that didn't include any hardware drivers at all. Instead of just using VGA 640x480 resolution on my 19in LCD monitor, I spent the evening finding, downloading and installing all the right drivers. That is also my only experience of trying to convince someone I had willingly bought software from that I was not a criminal. Thanks, Microsoft Software Activation. This computer lasted barely two years before a very fatal disk crashed, ended that incarnation.

So, it's not the computers, it really seems to be me. Those are the only computers that I've bought. Seriously, no other computers were hidden away in there. I've also had zip drives inexplicably and suddenly give the click of death, lamps leave scorch marks on my desk, monitors catch fire (really! there was smoke), printers refuse to power on and USB devices make my machine reboot right now. Maybe I just have a special relationship with hardware? I've definitely got a reputation for it... But, I'm a software guy, and there are uncountable software disasters tucked away in there.

Now, it's that time again: I need to replace my four year old iMac. I'm planning on getting a laptop, hopefully a MacBook Pro. Doesn't sound dangerous to me, what could possibly go wrong?

The Player of Games

2008-04-04T23:57:00.001+11:00

The Player of Games Iain M. Banks

That 'M' is important and very distinctive. This is a completely different author to Espedair Street; even though both books list all Iain Banks and Iain M. Banks books. No? Don't believe me? Well, yeah. His fiction is published as Iain Banks and his sci-fi as Iain M. Banks. Strange, but that's the way he does it.

His sci-fi is some of the best I've read since Philip K. Dick. And as he doesn't produce anywhere near as much as Dick, it averages a lot better. Though without some of the crazed inventiveness. But that sounds like damning Banks with faint praise: his sci-fi really is that good. There are fantastic ideas and a very plausible feel to everything. He doesn't shoot himself in the foot by trying to explain how everything works: the technology is just there and it works.

But his strongest points are actually his characterisations and story. You get involved, you believe, and most importantly, you care. And on top of that, the story is usually about the growth and life of a character - sometimes a descending spiral with no apparent way out; sometimes a broadening and opening of a character you initially dislike.

This book is fascinating for the first real peek inside the Culture, instead of the view of a mercenary looking from the outside, in.

Orlando

2008-04-01T22:48:00.001+11:00

Orlando
Virginia Woolf

Another literary fantasy novel. AFter the disappointments of Jordan, Martin and, most of all, Feist, I'm happy to be looking to Marquez, Updike and now Woolf for my fantasy fix. As I've said before, any story is by definition a fantasy, so why restrict your scope to only the events that can take place in this prosaic world we are trapped in? Sure, there's a place for the great everyday; but fantasy can be so much fun!

And given how dry Woolf is, it's surprising to see how fun Orlando can be. There are two key elements of fantasy here: Orlando (the character) lives for a very long time, and there's a second question of gender... The age question is handled interestingly. There's never a discussion of this, Orlando just keeps on living, aging at a different rate to everyone else around.

This disconnect from reality creates a dreamy, flowing world: the story reads like a lyric poem: drifting from image to image guided by your narrator, Orlando. And then towards the end it starts to coalesce on but two points. But slowly, like a willow emerging from the mist. Left wondering if those were always there, you float past.

Startups as the Future of Technology

2008-03-31T00:08:00.001+11:00

It's very fashionable in geek circles to attack Paul Graham at the moment, particularly after his essay You Weren't Meant to Have a Boss. I've been reading his essays for a few years now and I've wavered between agreement and an undefinable sense of unease. Now I believe I can finally pin this down.

The central point of Graham's Boss essay seems to be that over a certain size organisations become rigidly hierarchical and once the hierarchy sets in the creativity of programmers is significantly and fatally constrained: over a certain size an organisation will be incapable of producing original software. This is a continuation of a theme through much of Graham's writing. Startups do interesting work and software development will migrate exclusively to startups.

Well, I think that's something I can disagree with. Of course, it's pretty obvious that I have a vested interest. An iconoclast like Paul Graham will always get the most vicious response from those he seeks to help. Allow me to give my personal background, in the interests of disclosure.

My entire career has been in software development. In my first job out of Uni I fell into the 'hero' programmer archetype. In every job after that I've been in some form of senior position: tech lead, architect, team lead. I've also been an independent consultant, co-founded my own startup (we failed; be very careful about selecting your co-founders) and now I'm working for a startup. Yep, as an employee with a boss and all.

I spent sometime this morning going through all the startups listed on the Y Combinator website. For each startup I've tried to classify them according to the current big themes in web sites.

Social Networking: Reddit, Loopt, Flagr, LikeBetter, JamGlue, Scribd, I'm In Like With You, SocialMoth, Anywhere.FM, Disqus, Reble, AddHer, Inkling, Draftmix

Advertising & Sales: ClickFacts, Adpinion, Bountii, Octopart, Auctomatic, TextPayMe, TipJoy

Apps on the Web: Snipshot, Wufoo, YouOS, Thinkature, Weebly, Buxfer, Heysan, Versionate, Fuzzwich, RescueTime, 8AWeek

Other: Virtualmin, Justin.TV, Xobni, Webmynd, Heroku,

Dead: Shoutfit

The 'other' category is the interesting one: into that bucket falls a server admin dashboard, a web-based TV channel and a plugin for searching Outlook email. But, there are a lot less of those than the social networking and web-based desktop application startups.

I'm sure many, particularly the founders, will disagree with my classifications. But, these are mainly right, especially if you read 'Social Networking' as 'Social Networking around Common Interest X.' And in the end you'll find the exact classification is not important.

All of these startups share a few things in common. They were all launched quickly and they're all pure software development, often running on someone else's eco-system. There is a place for development like this, but if this is the future for computer science I believe the field will be significantly poorer for it.

I am working for a company that by pretty much every definition is a startup. By the time we left stealth mode in March last year the company had been around for 13 years and had grown from a core group of computer scientists to a company of over 300, including chemists and physicists. Oh, and we invented a new type of printer. A startup like ours simply doesn't fit into the Y Combinator model. We also don't fit into the small company with no bosses model: building hardware takes time and a lot of people, you simply can't avoid either.

This is my concern. Is all future computer science productisation and development really going to be latest and cool ad-funded mobile social networking site for parrots? Because, excuse me if I'm not excited by that future. I am still excited by the potential for computing and the Internet in particular, but that potential is better served by longer-term thinking and grander plans than refinements of what everyone else is doing.

This criticism may actually run deeper. A continuing trend in computing is to make programming easier for all. This has had the effect of pushing some tasks out of the realm of the programmer and back to expert users. This has been a good thing. Users have more control and programmers are free to work on interesting problems. The web has also been fantastic at improving human to human communication. Recently, the potential of the web as a mechanism for computer to computer communication is becoming more apparent. Many of the Y Combinator startups very effectively exploit this: improved experiences and convenience by combining the information on eBay with the blogosphere. It also appears that these startups are surfing a wave. The gap between technology becoming easy to use and becoming easy to program. In other words, I suspect this style of startup is not long for this industry. An historical aberration, automatic arbitrage for company startup, as it were.

This is not the end of startups of course. There will always be startups, however there will have to be some interesting, risky and difficult technology behind the curtain. Originality will once more be prized. In this world, You Weren't Meant to Have a Boss will make a lot less sense.

Personally, I fully expect to do the startup ride at least once more. And I'm looking forward to doing that in a world that once more demands innovation rather than just another social network. After all, I really do want to add something to the world and I just don't see that happening with late noughties startups.

Oh, and if you also want to work on world changing, original technology, my company is hiring. Love web technologies, think you have what it takes to work for Google, but aren't excited about working for a company of 10,000? Want to work in Sydney, Australia? Beautiful beaches, summer all year long... Send me an email, giles dot alexander at Google's-free-webmail dot com.

Espedair Street

2008-03-23T19:23:00.001+11:00

Espedair Street
Iain Banks

A very good writer, his sci-fi (under the name Iain M. Banks) is consistently original, but his non-genre fiction is also very good. Dead Air is worth reading for the head-butting alone and The Wasp Factory is bizarre, unexpected and simply amazing.

The strength in his fiction is the characterisation. Danny Weir, in Espedair Street, is a great example. A washed-up 70's rock star who has managed to annoy and drive off all his friends. He's now brooding self-pityingly in a stony mansion in Glasgow. But, you're introduced to him, you hang out with him, you drink with him and you get to know him, know him well. Though he spends the book going over everything that's gone wrong in his life and though most of that is down to his amazingly ability to always make the wrong choice and though it may be hard to listen to an hyper-rich rock star complain about his past it doesn't matter because you know him and, ultimately, like him. Enough to hope he finds some way out.

Oh, and the book manages to frequently be damn funny, as well.

The Timeless Way of Building

2008-03-18T21:20:00.001+11:00

The Timeless Way of Building
Christopher Alexander

For the past year or so, this was my bus book. That's a surprisingly long time, and it probably shouldn't have taken me that long to read. Late last year, about 50 pages from the end, I paused in my reading; and then took several months to pick it up again. This seems unfair to the book: it deserved a much more coherent read than that. Though, the ideas are different enough to also benefit from a considered read. I'll pick this up again sometime, and I promise to read much faster that time. Anyway.

One sentence summary: this book will forever change the way you look at and think about buildings, towns and architecture.

Alexander firmly believes that modern planning and building practices are bankrupt and can only result in inhospitable, unwelcoming cities and homes. A belief that seems to be firmly born out by most urban planning since the Second World War: just look at the damage Harry Seidler has wrought on Sydney for an example close to home. This book is a polemic, a grand rant against the current state of his own industry and art. A work in the tradition of many a genius' (and quite a few looney's) Let's Blow Up the Universe screed. So, genius or looney? I've probably already given away my opinion on that matter...

Ultimately, it would not do this book any justice to attempt to briefly summarise what it has to say.

But what the hell, I'll give it a shot anyway. Alexander's central thesis is that there is a shared quality amongst those towns and buildings where people feel most at home; a quality independent of culture, climate and history. He also believes that this quality can be easily achieved, by any person who chooses to build. It is a matter of recognising the forces within the people who will use the building or site and then balancing those forces with the forces intrinsic to the specific location and society. He even outlines a prescription for achieving this balance: a collection of patterns to duplicate in design, planning and construction, with instructions on how to combine these. A language of patterns to construct our built environment.

Unlike many other polemics, this is highly detailed and descriptive: it describes the quality to achieve and then gives instructions on how to achieve it.

If you live in a large city in Australia, it'll be pretty obvious while reading this book that this is not how building is done. First, Australian building practices place the car as king of all. Any building or neighbourhood must be designed for the maximum convenience of the car: people are a distant second. Second, Australian building practices harken back to some long forgotten European past: everyone wants a little brick English cottage, though nothing could be more generally inappropriate for our climate. The Queenslander is not the standard archetype for Australian residential building unfortunately.

The current popular obsession with being 'green' is driving people to a certain superficial realisation about the car. But that is only a symptom of a far deeper problem. Loudly proclaiming that cars are evil and must be disposed of is never really going to achieve anything. And that sort of unbalanced (in the forces sense) thinking will inevitably lead to other problems. As much as I'm a fan of the specific remedies proposed in Jan Gehl's research paper into Sydney's CBD, I do feel some uneasiness.

The pattern approach that Alexander talks about is intended to completely avoid unbalanced forces. He regularly uses cars in his discussion of patterns. They are real, they are valuable and they're not going to just disappear. A central point of these patterns is that they're not something Alexander has devised as a new architectural '-ism' to imprint his vision on the world. These patterns are things that arise naturally, given the way all humans want to live. Growing organically out of a combination of the people and their surrounds. There is a sequel to The Timeless Way of Building, A Pattern Language, that acts as a catalogue of the most important patterns that Alexander and his colleagues have observed in successful towns and buildings.

In the US there is a growing style of design called 'New Urbanism' that attempts to encourage the buildings and towns that Alexander commends so highly. It is interesting to note that in Europe that name is largely unused, people preferring to use 'The Way Towns are Designed' instead. It is also interesting to note that in Australia, we have neither.

Finally, why did I, a software engineer, read this book? To the surprise of many architects, Christopher Alexander is very well known in the field of computer science. In the 1960's his work was discovered and his concept of patterns was co-opted. No serious software engineer can possibly not be familiar with the world of design patterns: named rules for particular structures of code to solve certain problems. It's my opinion that while initially off to a good start the modern Design Patterns movement has completely missed the point of Alexander's original teaching.

His intent was not to catalogue an exhaustive set of patterns that may be thrown at a problem until a solution emerges. His intent was to define an interlocking language from which you can select appropriate terms to grow a solution. In his case a building or town, in my case a software system. Modern design patterns seems to ignore the essential organic growth aspect of a pattern language, and instead seems to focus on cataloguing. An unbalanced approach.

What is this Property?

2008-03-17T21:53:00.001+11:00

I'm not a mathematician, just a computer scientist with an interest in maths, so please excuse the simplifications and inaccurarcies in this. I'm going to describe this with some rigour, but I'm bound to get things slightly wrong, please bear with me.

In maths, a function is defined as a relation between the members of two sets, S and R, that produces members of the third set T. Looking at it another way, the set T is defined by the function. Some functions, taking two arguments from the same set S, always produce members of that set S. Addition across the natural numbers is an example of that: for any two numbers greater than 0, the sum will always be a number greater than 0. There are many functions that behave like this.

Functions have properties. A property describes a rule that a function obeys for given sets of parameters. From a mathematics perspective, these properties are interesting. For example, addition across the natural numbers is associative. This means that no matter what order the parameters to the addition function are arranged, the answer will be the same: 2 + 3 = 3 + 2. Fairly simple and obvious, right? But from the same property we can also say 2 + (5 + (6 + 11)) = (2 + (5 + 6) + 11). This is interesting because once we know that a function has the associative property we can arrange the parameters of the function without changing the meaning: this is useful in proofs.

There are many, many of these properties, and most of the interesting ones have names: associative, commutative, distributive. For the last year I've been trying to find out if another property I've noticed also has a name.

Take the function minimum across the natural numbers. Given the sets {4, 6, 100, 1, 43} and {1} minimum gives the same answer: 1. The result of the function minimum is determined by only a single member of the set, no matter how large the set.

Take the function and across the booleans. Given the set {true, true, true, false, true} the answer is false. It doesn't matter how many true's are in the set, the answer will always be false.

And I'm sure you can imagine other functions that behave like this. My question is: does this property have a name, and if it does, what?

If I was more of a mathematician, I'm sure I could actually describe this property a lot more accurately. In fact, I'm not entirely sure there is a consistent property here, and I have no idea if it's interesting if it does exist. But I notice this often, and it sure feels like it should have a name.

Functions whose result is determined by a single member of the parameter set, irrespective of the size of that set: do these have a common property?

Midsummer Night's Dream

2008-03-14T00:13:00.001+11:00

We went to see Midsummer Night's Dream at the Sydney Theatre tonight. A friend bought the tickets, we were just told it was Midsummer Night's Dream. I really should have found something more out about the performance.

I like fairly challenging books: I believe that the reader should occasionally be made to work for it. I love Pynchon and I enjoy Woolf. I am a huge fan of Shakespeare and I've enjoyed pretty much every production I've seen, even when I didn't know the play, both traditional and modern interpretations.

We left this play at intermission, along with a pretty significant proportion of the audience. I have never done that before. I won't even walk out of a bad movie.

This was plain awful. Absolutely, completely unwatchable. Why? It's about 60% performed in Hindi, with no sub- or sur-titles. If you don't speak fluent Hindi you won't be able to understand what the characters are saying most of the time. I know that's obvious when I say that it's performed in Hindi, but the Sydney Theatre really didn't make this obvious enough. I was also handicapped here as I didn't know the play. I've seen parts of it before, and remember some scenes but I don't know the overall plot and characters. I certainly couldn't imagine what was happening when I couldn't understand the dialogue.

The opening scene to establish the plot was entirely in Hindi, and from then on I had absolutely no idea what was going on. Mana tried to help out by whispering brief explanations as she has previously studied and performed this play. But she couldn't keep this up, and by this stage it was pretty much too late: I already had no idea who any of the characters were.

What do I know of Midsummer Night's Dream? Well, there's one of my favourite Shakespearian lines:

If we spirits have offended, Think but this and all is mended: You have but slumbered here While these visions have appeared. - Puck

That's from memory, so excuse any mistakes. I also remember the sarcasm, wit and lyricism of Puck. And I missed all that in this performance. Surprisingly enough, the play would have been better if it was entirely performed in Hindi: when they were speaking English I could follow what was happening and start to get involved. Then they would switch back to Hindi, kicking me out of any involvement, and leaving me bored and disconnected in my seat. But, then I would try to get involved again in the dance and acting, only to be booted again when they switched back to English.

Shakespeare is entertainment, especially his comedies. These were great works meant to illuminate the human condition, while also highly engaging and entertaining. Anyone should be able to watch a production and enjoy it. The only people who could enjoy this production were those who spoke fluent Hindi, and those who already knew the play intimately. And while I fully support the production of entertainment for specific languages, this should not be promoted to a larger audience as something for everyone. Because is it's not: this is an exclusive production only meant to be enjoyed by those who have already studied the play.

And I don't like this artificial, constructed exclusivity in the arts.

The Three Stigmata of Palmer Eldritch

2008-03-06T21:55:00.001+11:00

The Three Stigmata of Palmer Eldritch
Philip K. Dick

Did you end up finding it, Philip? What it means to be human? Religion didn't seem to provide your answer. Did drugs? A Scanner Darkly is famous for your search, but this appears to be some sort of transition between those two searches.

Like Graham Greene, Dick is one of my favourite authors. Over time I'm steadily trying to read all of his novels. I prefer his later ones, so that's generally what I choose. Unlike Greene, not each of Dick's is better than the last: A Scanner Darkly is still my favourite, and one of my favourite sci-fi novels. Sci-fi is typically a pretty pulpy genre: cheap enjoyment, with very little challenge to the reader. Even the best sci-fi with a great idea at it's heart will present that idea in a pretty straightforward form.

Not Philip K. Dick. He did not shy from challenging the reader with unusual ideas, often in outright confusing forms. This book felt like some sort of mental trap that the reader is drawn into. Only with the hope that all will become clear by the end. The confusion is why I read this book now. How hard can you push the reader? How difficult can you make the story to follow? How many tricks can you pull? And still end up with a populist, enjoyable story.

There's a lot to connect Dick and Pynchon. But, Dick just didn't have Pynchon's talent. Sometimes you are left wondering if this was meant to be confusing, or did he just write it a bit too quick. His later work does show that yes, he was aiming to confuse.

The Ultimate Development Environment

2008-03-05T21:52:00.001+11:00

An enormous claim, I know. But this is not about processes, tools or working conditions. This is about something quite different.

Shrew is progressing, it now sports an s-expr to XML evaluator; I'm reading RESTful Web Services to gain a better idea of how it should expose resources. And I'm also working through The Seasoned Schemer. And therein lies the most interesting aspect to Shrew. I am a reasonably experienced, quite competent polyglot software engineer, but learning Scheme has forever changed how I think about programming. And through example crystallised the ultimate development environment that I have been drifting towards.

When I'm working on Shrew, my editor looks something like this:

In the top-left is the module I am currently working on: writing, expanding or fixing - as I'll try to show there isn't really any difference between those three. In the top-right is a scratch file that contains a bunch of ad-hoc tests for the module I'm working on: nothing structured, just calls of the functions that I'm writing. Across the bottom is the output from a Scheme process running in the background.

Before I go on there's one detail of Scheme I should explain. To write a new function you call a built-in function called define, passing the name of the new function and its body. define is smart enough to simply replace the body if a function of that name already exists.

It sounds like a pretty simple development environment: a plain text editor with three windows on screen. Why so special?

I write a function - not a test, sample or prototype, but the real code I'm planning to commit - I jump to the end of the define and run a command in my editor to evaluate it. That function is then inserted into the running Scheme process and available to be used by anything else that is run in that Scheme process. Or, I'm immediately informed of a syntax error in my code.

I switch over to the scratch file and write some code to call the function I just implemented: typically just one expression, but it can be as many as I need. I evaluate that new code. And immediately see the output in the window at the bottom; the window reflecting the running state of the background Scheme process.

And of course, there's a bug in my function. I switch back to the window containing the module, fix the bug and re-evaluate. I switch back to the test code, re-evaluate that, and see that my change has fixed the bug.

Elapsed time from writing the function through debugging and verifying the fix: 45 seconds.

Instead of having to write a complete library, compile it, write a test harness, compile that and link it to the library and only then run the code to see if it works, I have a Scheme process running in the background that I can just keep adding code to. New code, or code to replace existing code. And at any point I can execute any sub-part of that process and immediately see the output. No delays, no pauses, no backtracking to find which line of code is wrong.

Scheme could be regarded as a fairly direct implementation of a theoretical model of computation: the lambda calculus. Most texts that teach Scheme emphasise this; they encourage you to think of your programs in terms of this theory, in quite some detail. That may sound fairly esoteric, but once you've spent sometime working in this environment you reach this unique state. You are inside your program, reaching around moving code as fast as you can think. There is essentially nothing between your thought and code: no defining boilerplate, no compiling, no creating test harnesses, no waiting for test runs to complete. Your solution simply unfolds before you.

But that's not to say your code is of lower quality. In fact, because you're concentrating more fully, with no distractions and the flexibility to easily push your code in any way you want, the code is of much higher quality. There's no idle thought 'I should test that' which is forgotten in the edit-compile-run-debug cycle: think it, try it. This is flow as Peopleware could only dream.

And once you break out of this magical flow, you're left with complete code; code you lived and breathed for a few hours, code you understand deeply and will have a hard time forgetting. Plus, a comprehensive set of tests to commit alongside.

The Blind Assassin

2008-02-25T21:44:00.001+11:00

The Blind Assassin
Margaret Atwood

Continuing on my plan to expand the types of books I'm reading: this was from Annabel several years ago. A Booker prize winner, and I've enjoyed some of the others I've read, it's also Canadian and a female author. I just haven't been reading enough female authors recently; though Zadie Smith is one of my favourite authors.

It's an interesting story for a number of reasons. It's structured as a story within a story within a story; it's told backwards and forwards, alternatively; it appears to be centred around a mystery, but really isn't; and, perhaps most interestingly, for a large part of the novel the main character is quite unsympathetic. And though unsympathetic, she still manages to maintain your influence and carry the story.

And stopping for a pause... when I first wrote this review immediately after reading the book over a month ago, I enjoyed the novel but wasn't taken by it. However, it's a novel I haven't stopped thinking about. It just keeps cropping up in my mind over and over again. At the time, I put it down as one of those typical, slightly over-wrought Booker prize winners, but now my opinion is going to have to change.

Slow, deliberate, difficult for not liking the main character, but in the end, memorable and worth it. I will be going back to read more Atwood: and I've found myself browsing her shelf in book stores.

Girl Geek Dinner 0

2008-02-24T22:52:00.001+11:00

The first Girl Geek Dinner, Sydney was held on Thursday, at Chinta Ria. For those of you who haven't heard of this: women working in computing/IT get together, have dinner, discuss technical topics and generally network. Damana has been organising this over the last month and a half or so. While the events are intended for women, each woman can escort one man. I was there as Damana's escort, acting in a supporting role. But, I will pause and make very clear here that independently of any personal connections in this particular instance, I fully support this idea. It's a great idea and I'd like to see more of this sort of thing.

So, as a guy at a Girl Geek Dinner, what did I think of it? To be completely honest, it was unusual. There are few enough gatherings of outgoing geeks and when there are the demographics are depressingly predictable. The Girl Geek Dinner was different but not dramatically. The overall mood and atmosphere of the group was what you'd expect from a crowd of people who don't know each other but share an interest. There was a lot of loud, friendly talking. People generally moved around and talked to others they hadn't met. Just like a bunch of strangers socialising.

But there were still differences. Differences that only exhibited on individual scales. Ultimately, this is an issue of gender, oppression and hostile environments. Please bear with me. Try to take this in the spirit that it is intended and above all remember that I am a major proponent of diversity in all its forms in all environments. I don't subscribe to any 'fundamentally different' hypotheses; in my view we're all the largely the same after individual and cultural differences are accounted for.

Onto the differences. Firstly, the women seemed significantly more relaxed in a work-type social setting than I have seen before. There was a lot of very loud talking, joking, laughing; a lot of alcohol was drunk (thanks Google!), the food was enjoyed (thanks ThoughtWorks!) and there were no more moments of self-conscious reflection than you'd expect when there are introverts around. This was particularly noticeable later in the night when some sleazy sales guy from another group tried to attach himself to us. It was a pretty clear reminder of what normally confronts women when they socialise together.

But that's all external observation; me theorising about motivations and feelings in a group to which I do not belong. I can be more certain about my own feelings. Several times, early in the night I mentally gave a start and felt that I had to move seats. I didn't though when I realised where that sense was coming from. I was one guy sitting at a table, laughing and talking with a bunch of women. And in Australia, at least, you just don't do that. I can't be certain where this was coming from. Either, it is not appropriate for a guy to be showing that much attention to a group of women (cf. sleazy sales guy), without also talking to some men occasionally; or, that men would regard other men who spend all night talking only to women poorly. So, was it fear of a negative reaction from men or fear of a negative reaction from women? Because I can say that it was not an inability to relate, not a lack of common interests: once I identified and ignored the feeling, it quickly disappeared, and I had a great night. Either way, this is the sort of thing that needs to be identified and disposed of before the inherent sexism in computing can be fixed.

But, apart form all that serious discussion of gender interaction, I had a great night and it seemed that so did pretty much everyone else. Thank you very much to Damana for organising this, ThoughtWorks for paying for the food and Google for providing the drinks. It was very interesting to get a real chance to talk to women in computing and about computing when I was the minority and I hope these sorts of events can do something to make computing a more balanced, realistic and enjoyable environment. Any geek guys out there, I'd recommend trying to get a date to the next one!

A Burnt-out Case

2008-02-19T22:41:00.001+11:00

A Burnt-out Case
Graham Greene

I read this book many, many years ago: way back in year nine English. It was the first of the Greene novels we read, followed by Our Man in Havanna and then The Quiet American. I remember really enjoying the latter two, but just not getting A Burnt-out Case. Thinking back on it now this might have been the first serious piece of literature I read. Wow. I had to get over fantasy at some point.

Of the three, this is the most adult and serious. At the core this story is about how you derive your meaning for life. What happens when instead of just accepting your life, you can't help question. Why you? Do you deserve your successes? Your failures? Your happiness? Your sadness? At some level, it's very easy to know that we don't deserve the bad things that happen. We move on, ignore those and wait for something good. But if you're going to scratch the surface of your life and ask 'What did I do to deserve this?' of the good things that happen, what answer do you get back? Are we set up to accept and enjoy success? Wuerry has scratched and examined; questioned and dug deep at the heart of who he thinks he is, until there was nothing left to scratch. And then he is on a small boat to a leprosarium in the heart of Belgian colonial Africa.

I'm very glad that I've finally re-read this book. Greene remains one of my favourite authors. Every book I read of his seems to be better than the last. Of course that can't be true - but it sure feels like it. Now, as I look through his list of books, there aren't many more that I've heard about that I am yet to read. This makes me sad. I'll be re-reading Our Man in Havanna at some point, and that gives me hope. I'd have never heard of that one if I hadn't already read it. There's no reason not to randomly try others then!

It is a pity that the Nobel Prize committee could never look past Greene's devout Catholicism. Though with Catholic themes and characters, his books are always much more than that. If there is anyone who deserved the Nobel Prize for Literature, it's Graham Greene.

On Reading and Writing

2008-02-17T23:23:00.001+11:00

You may have noticed a pattern in my posting over the last few months. There was a lengthy quiet period at the end of last year, followed by many posts so far this year. You may also have noticed that I've been clearing a large backlog of book reviews, but with each separated by a non-book review post.

Well, of course the dry spell was caused by complaints. Apparently I was posting too many book reviews, my blog had become too much of a book-blog. It's funny the effect that complaints and criticism can have. When you have to work out if everything you write is something that someone else will want to read, the effort of making that decision on top of the effort of writing can very quickly become pretty expensive.

It's stupid to complain about this though. That kind of complaint is something everyone has to go through as soon as their output is read. If you want readers, then expect that, and if you're not after readers, then don't publish. Pretty simple. At first I kept writing my reviews without publishing, but that wasn't enough, so they're now back. But it doesn't really work to say I want to publish this and readers be damned. So, a compromise. I'm interspersing my book reviews with other posts, like this one. Stories, photos and good old rants. The backlog of reviews will eventually clear. I'm not sure what will happen then...

But I also wanted this post to be some sort of explanation for why I have continued the book reviews. Originally, I said that I just wanted to keep track of what I'd read and any first impressions. But, I've found that planning to write a review changed the way I read and, for me, in a good way.

I found I was reading books a lot deeper. I was predicting plot twists, noticing intentional coincidences and becoming more involved and aware of the atmosphere the author was trying to create. I also started to notice the techniques the author may have used to achieve this. Turns of phrase, pacing with a description at a well-timed juncture to set the mood. The language they use and how it might affect your impressions. I found that this closer reading managed to significantly increase my enjoyment of a book - even a not particularly good one. I can't promise it's for everyone, but I've got a lot out of it and it will continue.

Everything that appears on this blog is written twice. Well, most things, like longer posts such as this and all my book reviews. The first writing is done in long hand using a fountain pen in a journal of some sort (currently, an unruled, leather wrapped Corban + Blair given to me by a couple of friends.) Then I type the entry out using Emacs and post from MarsEdit. A combination of old and new technology that I like. So why do I have this involved writing process? Why not just type directly into Blogger's text edit field?

Well, there's a couple of reasons. Some apply to everyone, and others apply only to me. Firstly, Blogger's (and all web app) text fields suck. You're just far to exposed too exposed to bugs in too many different pieces of software. If I'm going to write a long post, I want some more confidence that it's going to survive to be published. Secondly, I live inside Emacs and I'm officially Emacs-retarded; I want my reflexive editing keystrokes to do what I expect.

But those are just technical reasons and only apply to the MarsEdit/Emacs parts of my process. Why the long-hand? The fountain pen?

In truth, I don't really know. And to be honest, 'affectation' would be the biggest part of the answer. It's hard to justify a fountain pen and a leather-bound journal any other way.

But, but, but... a blank computer screen has never been a very creatively inspirational sight for me. Even when programming. I just can't start thinking when I'm staring at a blank text window. I need to get away from a computer and into a garden and then I can start to think.

That was without planning or thought. While living in Darwin, when doing any programming I'd figure something out - and then find myself in some part of the garden. I'd got up from the computer and walked around the garden on auto-pilot.

Anyway, a journal and a pen get me away from a computer. And once you've written with a fountain pen, you'll never be able to write with any other pen again. There's just something about the way the ink lays down while the nib glides effortlessly across the page...

But there's also something about the act of writing for me. It's just a great way to crystallise and direct my thinking. Of course it largely comes out as a mess on the first pass. Incoherent sentences, the same word repeated over and over again. The typing it all out again is a fantastic editing process. I can't help but fix up all those little problems. In the end, I enjoy the writing process, much as I enjoy reading. These are things that work for me, with how I want to read and write. Always make sure you do what works for you.

Finally, in all my talk about what I write in my blog you may have noticed that there was no mention of the computer science that used to be a staple of this blog. Well, there are a couple of reasons for that. My new work project is intensely interesting and gives my a lot of scope for thinking about comp. sci. and experimenting with ideas: the sort of thing I used to do here. But the damn, freaky secrecy of my work prevents me from talking about that. There is a slight cracking of the paranoia though, and hopefully at some point I'll be able to write about that here.

Also, hang around on reddit enough and you see a veritable flood of badly written tutorials on the latest programming feature to catch the eye of the blogosphere. I choose not to contribute to that until I have something substantially interesting to all. Surprisingly enough, my lament about protoypes fell into that category. I believe Shrew also will; there will be more about that at some point.

First Among Sequels

2008-02-12T19:12:00.001+11:00

First Among Sequels
Jasper Fforde

The latest Thursday Next novel and again a book for book and story lovers. A brief introduction: Thursday Next is a literary detective in an alternative reality where books are the single most popular form of entertainment. And in this alternative reality the line between the real world and a fictional world inside books is blurry. That probably sounds confusing. I'll try to clear it up: In this work of fiction an alternative reality is presented, in that alternative reality there is a world within works of fiction. The line between the fictional real world and the fictional fictional world is blurred. All clear now? No? Better read the books then, starting with The Eyre Affair; you'll love them if you love books.

This, the fifth book, is quite an independent story; as opposed to the preceding three - really a single story. The break between books has also been good for the world of the story. Fforde seems to have spent a lot of time cementing down how the world works. There is none of the drift and none of the continuity problems that the previous books started to show. And of course, for fans of the series there are plenty of cute little jokes. Fforde must remove dozens of these in draft as he always manages to resist the temptation to turn to gimmicks.

As an aside, once you read this book, go read about The Long Now Foundation the 10,000 Year Clock. Don't worry, I haven't given anything away. In fact, go read those now even if you don't plan on reading the book. The review's nearly over.

If you were a little disappointed by Lost in a Good Book or Well of Lost Plots then cast those fears aside. Thursday's back, and she's back on track.

Apoise

2008-02-09T12:24:00.001+11:00

This mega-yacht, Apoise was moored in front of the Park Hyatt at Circular Quay over New Year's; they would have had a pretty good view of the fireworks.

Apoise is 67m long, which just pushes it out of the top 50 yachts in the world; the shortest yacht on that list is 72m. It's impossible to tell who owns Apoise. The owners are listed as some Cayman Island company called Apoise Holding, Pty Ltd. Which tells you nothing.

Stardust

2008-02-02T17:35:00.001+11:00

Stardust
Neil Gaiman

A very cute, very enjoyable fantasy fairy tale. There really isn't a lot more to it than that: it's simple, plain, well-written escapist fun. Like all of Gaiman's work that I've read (which is nearly all of it) it's very well written, and hell of a lot of fun. Highly recommended.

I was inspired to read this by the movie. It's very similar, the movie was generally pretty faithful. Most of the plot changes were pretty minor: just to streamline and shorten things, largely. The movie expanded the minor characters; they became bigger, more memorable roles. This was probably to attract the big names they got, but wasn't actually a bad thing. Gaiman was very clearly attempting to create a rich supporting cast.

The movie did make one very substantial change. And, with the benefit of reading the book, it was very clearly for the worse; and quite forced. I won't say what that was, except to say the book is better without and the movie would have been equally so.

Prototypes and Real Applications

2008-02-01T23:46:00.001+11:00

There is an essay, The Art of Lisp & Writing by Richard Gabriel. It's long, but like all of Gabriel's essays it is worth reading if you're interested in exploring alternative views of the act of writing programs.

I am simply not qualified to do justice to Gabriel's central point. Simply note, it was Brooks of The Mythical Man Month, that bible of solid, empirical, meticulous software engineering planning who first drew the connection between the work of programmers and poets.

Instead, there is a minor point in Gabriel's essay that caused me to think about a brief chat in a recent interview. We'd worked our way through the questions, the candidate had done well, there was enough time left to just chat about his work. His two favourite languages were C++ and Python, so we drilled a little there. How had he used them? What did he think of them? Had he tried to combine them together?

Well, it turned out he had. Using boost::python he had exposed some of his C++ classes into Python. This was interesting. Why? It was so he could quickly produce parts of the application; he'd become a fan of this technique. All very well and good, had he tried that approach elsewhere? Unfortunately, no he hadn't. His other use for Python was as a prototyping language. So close. I drilled a bit on this point. Had he made the obvious step from here? Turns out he hadn't. But that did leave me thinking, and Gabriel's essay crystallised this issue in my mind.

Why did he just throw away his Python prototype and rewrite the whole thing in C++? Why didn't he grab a profiler and start replacing the slow Python parts with fast, small chunks of C++?

Developers seem to see an enormous gulf between a prototype and a 'real' application. Something that is appropriate for a prototype must be discarded for the real version. The basis for this act seems to be that a prototype is a quick and dirty hack and can't be trusted to work.

Well, I don't know about you, but I can't help myself: every piece of code I write, I write maintainably and flexibly. And these are issues in prototypes. A prototype is meant to be an exploration, a journey to see how an idea will play out once it's embodied in a program. It is an idea that has not been seen before - this is when it needs to be most flexible. When everyone who sees is thinks of an improvement, when the sight of running program inspires completely new ideas. This happens during the prototyping phase - and if you don't explore these ideas, then what was the point of the prototype at all?

Your prototype needs to be written quickly and then it needs to change quickly. You'll only be able to do that with a maintainable, flexible code base. In short, a well-written code base. You're a proficient software engineer, you know how to do this. You probably do it without even thinking.

And at some level, everyone knows this. That's why prototypes are created in languages like Python. A language that you can write quickly, but also write well, quickly.

So when it comes time to write the 'real' application, when all the decisions have been made and the exploration has stopped, when we know what the program will do, why do we throw away all that very carefully engineered prototype code? It can't be because the code is no good, because that code is better than you believe the 'real' program needs to be.

Most developers would say that it has to be rewritten for performance. But when was the last time you profiled your code? Because if you haven't profiled you simply can't know what your performance is, and more importantly, where the trouble-spots are.

I want to prototype. I want to explore my ideas in a running program. I want to carefully engineer my program and then change it at a moment's notice. And I want to be able to do that all the time, for the entire life of the program. I don't want to kill my program just when it's about to finally get some users.

I want to express my ideas in a language that gives me all that.

Oh, and that application the candidate had been working on? The one that had Python embedded in the real, final version? It was a massively multiplayer 3D game engine.

Brazil

2008-01-30T19:16:00.001+11:00

Brazil
John Updike

The outline is a re-telling of the classic story of Tristan and Isolde. This is only a frame for a much larger story to be hung from, though. Overall, I was reminded of Gabriel Garcia Marquez's One Hundred Years of Solitude - perhaps because I'm simple and both are set in the wilds of South American jungle. But to be slightly fairer on me, both are apparently non-genre modern classics, but they both contain large elements of the fantastical. In other words, these supposedly very serious, meaningful books turn into fantasy for at least part of their story. Quite amusing: fantasy is typically regared as barely writing at all.

This drift is done unexpectedly, with a sudden jolt. This is something I love. A novel is a depiction of another reality, why does that world have to work exactly the same as our world? Why not answer some of those questions that can't normally be answered?

This is a story of fidelity, change and growth. The story follows the two young lovers as they try to stay together against the weight of society. It sounds very corny, but there is enough to this story such that the apparently corny plot line becomes insignificant; or, it's twisted just enough to become significant again.

My big question while reading, was whether Updike had actually travelled into the back country of Brazil. Beyond the story, fantasy and societal commentary, this is one of the rare books where the author has tried to get his head into another culture. Of course, knowing nothing about the Brazilian back country myself, I can't judge his success. Which makes an interesting point: if the readers can't judge whether this is a real depiction of Brazilian society, does it actually matter if it's accurate or not? Doesn't the society drawn just become part of the fantasy world of the novel?

This book also has 'grand experiment' written in its very pages. The running and movement through the land of Brazil is also echoed as movement through time. Slowly melting your sense of when this book is set. Dates are not mentioned at all until near the end. Your sense of the book's setting slowly comes into focus, emerging from a cloud. There is also a great loop: starting with a world and a story we know and can understand, as the characters run into the wilds of the Amazon rainforest we move deeper into that fog. Until we're so far in you have no idea what you're seeing, what's real, what's imagined. And then the return begins, until you're back with characters, settings and plots that are familiar. Ultimately leaving a sense that nothing untoward has happened; nowhere strange was visited. A genuinely disconcerting sense.

Darwin, December 2007

2008-01-27T23:38:00.001+11:00

Just a random set of photos from Darwin, while there over Christmas 2007

First, three photos looking off the Dripstone cliffs at Casurina beach. This was not a particularly low tide.

I was impressed by the growth in Darwin city; it feels like a very vibrant centre.

Darwin Wharf for a casual dinner of fish and chips with a great sunset.

The Maltese Falcon

2008-01-26T21:52:00.001+11:00

The Maltese Falcon
Dashiell Hammett

I don't normally read detective/mystery novels, but I was given a recommendation to try reading outside of what I normally read. Figuring that if I am to grab a book from another genre, I might as well do that genre justice, I decided to read this.

Ultimately though, I just don't have much to say and I probably won't be going out of my way to read any more detective novels. That's not to say it's not a good book, but the characterisation was just bizarre. Sam Spade is the original hard-as-nails private eye - I'm sure he's intended as a character to be admired; as something or someone to aspire to. But that's just not something I could do. I'm sure at some point in the past Spade's attitude to women was admirable, but I could certainly never do that. Spade's interaction with the police was fantastic though - a real sense of reality. Hammett was clearly writing these scenes straight from memory.

The story and mystery was enjoyable, with plenty of twists, just as you'd expect. Personally though, I just don't get deeply involved in figuring out a mystery. Figuring out the people: for some reason that's much more interesting; much more of a mystery.

This is the original private eye novel: everything that came after is just a refinement of the mould set in this book. As I was reading, when I needed to visualise scenes, I was pulling images from Who Framed Roger Rabbit? of all places - not establishing the exact tough tone required.

Really what this came down to, is this kind of book is now just a cliché and a parody. Even the twists are predictable from this point of view: just expect the worse and you're about on track. And when you're reading a cliché you can't help but do that.

'If this is the kind of thing you like, you'll like this kind of thing,' otherwise... Well, you can read it for the historical value... No matter what you will almost certainly enjoy this story; just try to forget the clichés and parodies. Wind yourself back to the 1930's, if you can, that might help.

Our Old Home

2008-01-25T21:50:00.001+11:00

Our family moved to Darwin in mid-1986, after migrating from Wellington, NZ to Katherine in May of that year. Sometime between my birthday (5th December) and Christmas we moved into 4 Cunjevoi Cres, Nightcliff. I can't remember exactly when; I was young. From then on, my Dad worked on improving the house and garden. Over the next fourteen years the garden evolved into (in my opinion) one of the better examples of a tropical garden you're likely to find in Darwin. A couple of points:

There was a noticeable difference in temperature when you walked through the front gate: it was at least a couple of degrees cooler. And in the middle of the Darwin wet season, you appreciated that.
No grass. The green cancer is woefully unsuitable for all Australian climates, but tropical gardens with large expanses of lawn are virtually uninhabitable.
We had a continuous set of native residents: skinks, possums, blue-tongue lizards, green-tree snakes, honey-eaters nesting off the balcony: we had it all.

I haven't been back to Darwin since sometime in 2003. My Dad finally finished and sold the house at the end of 2004. I was the first of our family back in the city since then, so I made sure to visit the old house. The owners weren't home, but one of their tenants were. She opened the gate; I circled the house and took some photos.

Overall, Darwin was something of an experience after so many years. I always knew I didn't quite fit in, but you need to live somewhere else before you can really understand why. Fun fact: the local Darwin dialect has no equivalent word for Bogan. Make of that what you will; I know what I make of it.

From the street.

A pergoda, with a formal fish pond.

Old fig with a home-made aviary: they're using it now to care for injured possums.

Looking across the pool.

A variegated pandanus.

The natural pond.

The natural pond: closer.

Lilies and water plants run wild.

Front garden with Alexandria and Betel nut palms.

Snow Crash

2008-01-24T22:34:00.001+11:00

Snow Crash
Neal Stephenson

My first cyber-punk novel; I haven't even read any William Gibson. This was recommended to me over his other novels, including Cryptnomicon, which I already had. It's a very good sci-fi novel: unlike much other sci-fi Snow Crash is both written recently and set in the quite near future. This book also has another advantage: Neal Stephenson was a professional programmer and has chosen to write a book about programming.

As another professional programmer however, I do have to object somewhat to the depiction of the profession. Stephenson elevates the cowboy approach to mythical heights. Not something I agree with. But, it's a bit of a ridiculous complaint. He gets the terminology right (even the oft-misused word 'hacker'); he does understand actual software engineering techniques; and he hints at software development being a creative process.

Ultimately, programming is not a spectator activity: there's a lot of quiet thought, some esoteric arguments and a small amount of typing. I can accept jazzing up programming for the sake of the story. Given the reality.

The future presented is very plausible. Including the interaction between programmers, corporations and the vast majority of users. It could very easily be regarded as overly dystopian. The alternative to the future in this story is that improved technology creates more engagement amongst the non-technological priesthood part of the population. There is precedent for taking that view. The novel 1984 is unfortunately implausible as many technologies since have made the techniques described impractical or impossible: the photocopier, for one.

Personally, I have seen blogs improving the literacy level of those writing them. But, ultimately, the Stephenson's dystopian world is all too imaginable. I still have fairly negative attitudes towards the future of technology. It will not save the world.

Aside from the depiction of the future, this book has a fantastic idea at its centre. It's an unusual idea, and very well-executed. There are some distinct echoes of Philip K. Dick's A Scanner Darkly, but only towards the periphery.

The plot is fairly hackneyed and predictable. The whole 'murdered genius leaving behind a trail of clues to pieced together in time to save the world' thing is pretty common in sci-fi/fantasy stories.?But in the end this is not a disappointing novel: a great idea and well-executed, that rarest of combinations.

Two Tales from the Same Day

2008-01-21T23:01:00.001+11:00

The Afternoon Bus Trip Home

It's a rainy day, there's three of us huddling under the balcony of the Town Hall hotel. The rain drizzles down slowly; the ashphalt and brick building glisten damply. We're chatting about garbage collection in Scheme as the bus pulls up and everyone piles on. The three of us take seats at the back and A, R and I keep talking.

A little further down the road (maybe the West End hotel stop, but I wasn't really paying attention) a guy gets on and takes the seat in front of me. The bus continues; droning out of Balmain, onto the Anzac bridge. The sun appears out of the clouds briefly - playing across the bridge pylons and the last vestiges of the working harbour beneath. Our discussion is getting animated - what can I say? Scheme and garbage collection matter to us.

And then suddenly, the guy sitting in front of us interrupts. 'Hey!' You'll have to imagine the nasal, pinched drone for yourself - 'Does any of youse guys has a mobile phone you could lends me?' 'Sorry mate; no.' We respond.

And that was enough. 'Where's the fucking Australian spirit? Ya bunch of fucking wankers. You're all just a bunch of fucking c*&ts. Fuck youse.'

We tried to go back to our conversation, but he wasn't having that.

'Nah! I don't want to fucking hear it. Shut the fuck up, ya poofs.'

Then his phone rang. Yes, he had his own phone. The guy on the other end was told (loudly) about how this poor traveller was surrounded by a bunch of un-Australian c*&ts.

The bus continued along the Western Distributor, over Darling Harbour and into the city. At the next stop, on Sussex St, A, R and I all decide to get off.

We're followed off the bus by more descriptions of our patriotism, anatomy and sexuality. R blames my long hair for the last. Having a look back at the bus we can see him screaming at us and pounding the window. Wow.

The Morning Bus Trip to Work

I run into DC waiting for the bus. She's reading a book: 'Scaling Software Agility.' It's about RUP and agile for big and growing teams. We start talking about it, and continue talking after we're on the bus.

As our bus crosses the Anzac bridge, heading west, there's a lull in our conversation. And into the pause, the old lady in the seat in front of us turns around. 'Would you look at that? Evidence for global warming if ever I saw it.' She gestures towards what appears to be a cloud bank over North Sydney. 'All that smog and all those new cars waiting to add more.'

Below the Anzac bridge is the White Bay car wharves - just about every new car in Australia has passed through there. They sit, gleaming, sparkling cleanly. Waiting to be shipped off around the country to their excitedly waiting owners.

At first DC and I are a little taken aback - as you usually are when a random person on the bus starts talking to you.

'Well, actually, I think that's fog; not smog.' From DC, after a pause.

'Yeah, we've had quite a bit of rain recently, that would be fog, I think.' Me, this time.

'Oh? Are you a climatologist?' Now that surprised me. Such a sweet old lady, so confrontational! I had tried to be polite. There must have been some shock or surprise in my face though, because she quickly continued.

'I'm sorry for interrupting, but I've just been waiting for a chance. Listening in on your conversation has just made my year in Sydney! It's sounded like such an intelligent conversation, especially compared to the usual inanity I overhear on public transport. You know, the usual he said then I said and she said. Just so inane! Anyway, I thought you may have been scientists. Are you climatologists?'

'Close,' DC replies, 'we're both engineers.'

And then, alluding to her experience in public health policy, she regalled us with her very high opinion of engineers. All the way through the narrow street up the side of the Balmain peninsula and to our stop, opposite the Town Hall hotel.

The Songlines

2008-01-13T15:45:00.001+11:00

The Songlines
Bruce Chatwin

On the surface this may seem to be a travel novel, that is what Chatwin's most famous for after all. You could read this as a book about a trip through the red centre of Australia. But this is really a book about travelling: it seems to be Chatwin's final attempt to get at the wanderlust in his heart.

What surprised me though was how well I know and remember this part of Australia. His description of the Katherine Hotel-Motel; Katherine's main street: Main St; the two kinds of roadside pub he stops at on his way south down the Stuart Highway; and above all: the landscape, the desert and the colour. With every one of these I could feel the heat, the dust, see the flys. It was all still there.

Ultimately, Chatwin is trying to get an understanding of the Aboriginal Songlines: the tracks laid down by the ancestors during the dreaming. And for this alone, every Australian has to read the book. I will not try to summarise or explain the Songlines here. I am simply not qualified.

When I was growing up in New Zealand, part of our school education was Maori culture: we learnt basic Maori language, Maori cultural traditions and Maori society. Before our family moved to Australia, our parents as conscientious school teachers tried to make sure we were up to date with the Australian curriculum. English, maths, science were all easy; basically the same things were taught. But we didn't know much about Australian history. We were given books to read, including an Aboriginal dreaming story, told for children, each.

When I arrived at school in Katherine, a town with a significant Aboriginal population, I discovered that I knew more about Aboriginal culture, stories and history than any other white kid in my class. As a seven year-old, I was stunned. To this day most Australians can't name the traditional owners of the land they live on. And if you did want to find out, it's hard: Australian society is not set up to share knowledge of our Aboriginal heritage.

This is a ringing indictment of Australia. Do your bit to counter, read this book. We have this shared history of stories that haven't been told and you haven't heard. Try them, you'll be surprised by how enjoyable they are.

Chatwin's book does wander a little towards the end. I wonder if any of this is to do with some knowledge that this was his last travel book? Did he know and wanted to say his final word on travel? Oh, and by the way: this is it for me and Australian writers. I've long been sick of stories about white farmers, and if an English guy can actually say something about Aboriginal culture then I have no further time for Australian writers. Hear that, Australian literary establishment? Screw you guys, we're through.

Falling Pots

2008-01-11T22:57:00.001+11:00

As you walk around an inner city suburb with high apartment buildings, you may see many balconies with pot plants sitting on the ledges. You may also wander if those pots ever fall onto the street below.

The answer is yes, yes they do.

This is another photo from the large ex-Mark Foy building across the road. This ceramic pot fell from a balcony on the fifth floor, through some trees while people were on their way to work.

I was too slow with the camera to get a shot of the two terrified guys who were walking past when the pot landed. The street was full of fragments of pot for a couple of days.

Y: The Last Man - Unmanned

2008-01-09T19:50:00.001+11:00

Y: The Last Man - Unmanned
Brian K. Vaughan, Pia Guerra, José Marzán, Jr.

There was a long hesitation before I was willing to buy this comic, the premise has immature male fantasy written all over: everything that's bad about comics really. But, I was looking for something else, and so far I'm happy with my experimentation. This first volume has a very sci-fi feel. In particular, the conflict that establishes the characters and series is very much a sci-fi idea. In fact, it now appears that it's a pretty direct derivation of Frank Herbert's The White Plague, which D is currently reading.

In brief, every male mammal suddenly and catastrophically dies. All except for one guy. You can probably pick that up just from the title, though. Unlike the other comics I've been reading, this one is written and set quite recently: 2002. It does give it a more immediate, less abstract feeling.

I do have a criticism however. Yes, the sudden death of 50% (any 50%) of the population would be pretty devastating, but I doubt the social collapse would be quite as bad as depicted. Maybe I'm just disloyal to my gender, and there is the post-Katrina, New Orleans melt-down to consider.

Anyway, I'm starting to understand why heavy comic book readers make such a big deal about publishers. Just about every comic that has looked interesting enough to buy has been published by Vertigo, an imprint of D.C. Comics. I do now have one comic published by Marvel, I'll be sure to let you know how that goes when I read it.

Oh, and the obvious, juvenile male fantasy has not played out. Fortunately. So I will be continuing with the series. At least for now.

Fire

2008-01-08T19:37:00.001+11:00

Across the road from our building is another old warehouse converted into apartments: one of the Mark Foys buildings. This is not the original store, that's now the Downing Centre. This is the brownstone Mark Foy warehouse.

It's a very large building, with a lot of apartments in it. Lots of apartments means lots of 'stuff' going on. A recent trick has been triggering the fire alarm. It's loud, the whole building needs to be evacuated and it requires two fire engines each time.

Two fire engines, just in case...

Some evacuees, and a lot of people just trying to go home.

The League of Extraordinary Gentlemen, Vols. I & II

2008-01-07T20:23:00.001+11:00

The League of Extraordinary Gentlemen, Vols. I & II Alan Moore

I read both of these a while ago, so am combining together two reviews into one blog post.

Volume I

I'm still enjoying comics so I'm reading more and branching out; unforunately this isn't branching our very far. D picked this one out as she enjoyed the movie.

The comic is pretty good - quite different in visual style to The Sandman, with a more straightforward mystery story line. But, good though it was, I didn't like it as much as the other comics I've been reading. The most interesting aspect is that this is actually a re-telling of Sherlock Holmes, from the perspective of a group investigating a minor storyline. This was entirely missing from my understanding of the movie, so either I'm unusually dense, or... I can see why Alan Moore objected so much to the movie.

We have volume two of the series, so I will be going on to read that. And also other comics. Definitely The Sandman, but I'm also going to try branching out a bit more: to works written by authors I haven't already read nor the originals of fairly mainstream movies.

Volume II

This one was read straight after the first volume, which makes comparison easier: unfortunately it just wasn't as good. Volume II is a re-telling of H.G. Wells The War of the Worlds. This re-telling is much more straightforward: it's obvious within the first few pages.

This comic is also much darker than any of the other comics that I've read. V for Vendetta was probably darker in tone, but this is much darker in imagery. There is actually some moderately gruesome stuff in this one. You have been warned. Frank Miller's work (Sin City, The 300) has been been recommended to me: so I'm guessing this is pretty good preparation.

Volume III wasn't available when I went looking, so I'll be trying some other stuff next. And to be honest, I'm no longer in any great hurry to keep reading The League of Extraordinary Gentlemen.

The Observatory

2008-01-06T16:03:00.001+11:00

Back in October last year, we spent a weekend at Sydney's Observatory Hotel for our anniversary. A very nice weekend; their restaurant (Galileo) is highly recommended.

Here are just some random photos from spending time wandering around Miller's Point, Walsh Bay and looking off the balcony of our room.

Funny story: the weekend was kind of a present from me, so I was in charge of the organisation. I think you can see where this is going. We were going to make a whole weekend of it, so we also bought tickets to the play Don's Party at the Sydney Opera House. Come the Friday afternoon as we were about to leave for the hotel and then the play, I went to check the booking for the hotel: and found the booking was for a Wednesday and a Thursday about two weeks later. Oh.

I called the Observatory's reservations centre, and they managed to change the booking to that Friday and Saturday night, in the same kind of room. Fortunately for me. Weekend saved.

The Princess Bride

2008-01-05T17:35:00.001+11:00

The Princess Bride
S. Morgenstern
'The Good Parts' Edition, Abridged by William Goldman

We all know the movie of course. And if you don't, go straight out and see it right now. This is the version of the clasice tale that became popular enough to get that movie made.

It's most surprising to see how close to the abridgement the movie managed to stay. I guess being abridged by the same guy as who wrote the screenplay to 'Butch Cassidy and the Sundance Kid' would help.

And that's something I didn't know before reading this book. Learning the background is the best reason for reading this - did you know: the legend about Goldman reading this to his daughters is not true? He didn't have any daughters, just a son. Did you know: Morgenstern actually wrote 'The Princess Bride' as a satire of royalty? By filling the backstory of the the writing and the history of the original you do begin to understand some of the controversy over the abridgement and why Stephen King got involved in the abridgement of the sequel.

I was particularly inspired to read this by seeing the movie 'Stardust' - also a fairly corny fantasy story, also adapted from a novel. It has a very similar feel: a really traditional storyline, but then told in this slightly mocking, ironic style. It even includes some unusual and unexpected characters.

Neil Gaiman (the author of 'Stardust') was inspired by something, I suspect. Which also brings me to: If you like fantasy, but are maybe a little burned out on the genre, or if you just plain like fantasy, then do yourself a favour and read something by Neil Gaiman.

And of course 'The Princess Bride.' All will then become clear...

New Years Eve 2007

2008-01-04T20:29:00.001+11:00

We moved to Sydney at the beginning of 2007. This city is quite renowned for a reasonably good fireworks show on a pretty nice harbour for New Year's Eve, and as everyone agreed that you need to spend your first Sydney New Year's down on the harbour, that's where we went.

We got to Milson's Point at about 9pm, but still managed to get a pretty good spot right on the water with views of the bridge, the Opera House and the city.

Getting back onto the train to get home was... interesting. Maybe a rave at Bondi, next time?

One Computer

2007-11-27T22:56:00.001+11:00

Thomas Watson from IBM said that he could foresee a need for perhaps five computers worldwide, and we now know that that figure was wrong, because he overestimated by four. - Clay Shirky

There is only one computer in the world now, and that computer is the World Wide Web. Developers must know that, understand that and believe that.

Software does not matter unless it is written to run on that one computer. Know that too.

I have been saying this for awhile now, and some people have had some trouble understanding what I mean. I am not saying that the only thing that is to be written must be web applications. What I am trying to say is that if you are writing software it must be able to be used by applications and software that does run on the Web.

If your software does not, then it will simply be ignored. That's not malicious or intended as a criticism of your code, I'm sure it's very good. Think about it this way: as OS/2 faded away any software that only ran on OS/2 simply stopped mattering. This is just happening again, except all operating systems are fading away, replaced with that one computer.

So, if you have some amazing idea for writing some beautiful video processing code in the current functional-language-of-the-moment, you must have a plan for making that code useful to the web otherwise no one will ever use your code.

Positive Change

2007-11-27T22:27:00.001+11:00

There was a federal election in Australia, just last weekend: the 24th of November. This election went a lot better than the last few years worth of elections. John Howard finally lost. After 11 years in power and four election victories against either ineffective or insane Labor opposition, the mean, evil little troll has finally been ejected from power.

It was very satisfying to also see him so thoroughly ejected from power: only the second sitting prime minister to lose his seat. Ahhh...

And Australia can now try to once again become a better place to live.

There has been much discussion over why Howard deserved to lose. Many have correctly pointed out that for all his claims he was not a leader who championed much reform; that the continuous 11 years of economic growth was really part of a longer 16 years of growth started under the previous Labor government due to actual economic reforms; that he no longer offered any kind of vision for the future of Australia; that his threat to hand over to Peter Costello was seen as just that: a threat; that it was just his 'turn' to go.

For me, while all of those (except the last) are true, none come close to the real reason why I have always been adamantly against Howard, and why I am particularly happy that he has now been voted out.

Howard was a leader who created a government of isolation, selfishness and meanness. And of course this sort of attitude at the level of the federal government can't help but affect the entire shape and direction of Australian society.

Consider a handful of events under Howard's watch:

Pauline Hanson spewed her racist, xenophobic vitriol without any form of condemnation from the government, permanently damaging our image and reputation throughout Asia.
Refugees aboard the MV Tampa were prevented from entering the country, extending our damaged reputation to the entire world.
The GST, a fundamentally inequitable tax hated by both small businesses and the poor alike, was introduced. When there was apparently no need as all the earnings were returned as tax cuts.
Children nearly drowning while on their way to the Australia was used as a wedge to further incite anger against refugees.
A deeply unpopular war was started with the sole justification of supporting George W. Bush.
Tax cuts were the sole mechanism of delivering benefits to the population: a mechanism that always favours the rich over the poor.
University educations became a mechanism to turn a profit.
The enormous earnings from the global resources boom was squandered without any investment in social services.
The Australian republic movement was deeply damaged.
There was absolutely zero progress on Aboriginal reconciliation: capped with the 'intervention.' Itself worryingly reminiscent of the Stolen Generation.

Ultimately, this was a government of the mean to benefit the rich. Under Howard and his cronies Australia became a more selfish country, a more racist country, a country of people who would not do something just because it was the right thing to do.

I was always against Howard because of the damage I thought his ideaology was doing to the fabric of our society. In the end, my theory is that he lost because people finally saw him for what he was. [WorkChoices] showed that he didn't really care about his 'battlers.' And also, I believe that people didn't want to be selfish and scared anymore. The country wanted a more positive view of the future. You can only be terrified of your neighbour for so long before you decide that it's enough.

It is interesting to see that others are also coming to this conclusion. My fervent hope is that his legacy is seen for what it really is. In the end, the country will be a better place without him. Hopefully, it won't take too long for the damage to be repaired.

Good riddance Howard, we're better off without you.

Currying

2007-11-19T23:44:00.000+11:00

As a lazy functional language Haskell has to provide automatic currying of all functions. But once you've tried to use a functional language that doesn't automatically curry you realise this isn't just an implementation detail, it's a useful technique for concisely implementing programs.

but before describing how currying helps, what is it? Currying is a concept from the analysis of mathematical functions. Say, you have a function f(a, b), that takes two parameters and returns a value. Currying reduces this to two functions: one that takes the other parameter and returns the original value.

That is, currying reduces functions parameter-by-parameter into other functions - these new functions each take one less parameter, but return a function, instead of a value, that then takes the remaining parameters.

For example: f(a, b) = a + 2b. If this were curried with respect to a = 5, b = 10, then the function would be reduced as follows:

f(5, b) = 5 + 2b
        = f(b)
f(10) = 5 + 2 x 10
      = 25

And hopefully you noticed that halfway through that evaluation we found a function over b which had already had a value substituted in for a. This, by the way, is also an example of partial evaluation. In fact, it's more an example of partial evaluation than currying, as currying does not require values for function parameters.

Anyway, that's a brief introduction to currying in maths, but I'm not a mathematician so for some more (accurate) information try reading up on combinatory logic.

So, how does currying exhibit in Haskell? Function application is treated as a special operation, distinct from calling a function. Functions are applied from left to right, to the first parameter, producing a new function that is applied to the second parameter and so on. A Haskell function call like:

f 1 10 "Hello"

is evaluated as if it's parenthesised like:

((f 1) 10) "Hello"

If working from left to right each additional expression produces a new valid expression, then does that mean we can leave off an arbitrary number of of expressions from the right and still have a valid expression? Yes, that's precisely true. In fact that would actually be a function which could then be given a name and provided with any number of values.

And this is where currying really comes in use for expressing code. Using the right functions and operations you can typically define your own functions without naming any parameters.

In particular, this dramatically reduces the number of anonymous functions you need to declare. As opposed to Scheme, they become quite rare.

A Scheme implementation of reverse could look something like:

(define (reverse lst)
  (define (r lst)
    (if (null? (cdr lst))
        (lambda (t) (cons (car lst) t))
        (lambda (t) ((r (cdr lst)) (cons (car lst) t)))))
  ((r lst) null))

While an equivalent Haskell implementation could look like:

reverse lst = r lst []
  where
    r (h:[]) = (:) h
    r (h:hs) = r hs . (:) h

Of course, in the languages' standard libraries these functions are defined quite differently (Haskell's is only one line...) But this is useful as those are equivalent implementations - Haskell's is shorter only because it automatically curries.

Finally, currying is required in Haskell because it's a lazy language. The currying produces a train of anonymous functions - each retained as a value. Arriving at a result when required is simply achieved by evaluating those retained anonymous function values. But only the minimum set that are actually required.

Understanding currying and being able to curry your own functions is your head is a fundamental step to 'getting' functional programming.

Bear With Me...

2007-11-19T19:59:00.001+11:00

Now that I'm using some proper tools for my blog editing I'm going back through my old posts making sure the formatting and HTML are correct. In particular, I've turned off the Blogger setting to do magic with line breaks, and instead I'm manually inserting <br/>'s where I want them to be.

This is better for my general OCD nature regarding all forms of code I write, and I hope my blog will look a bit better too. Please bear with me as all my posts slowly get cleaned up...

Oh, and if there are any other Mac Emacs users out there who are interested in my little tool, send me an email and I'll try to arrange to share it.

Local Editing

2007-11-18T19:36:00.001+11:00

I was never entirely happy with using the Blogger editing box in a web browser for an entire, long post. It's in a browser, where the slightest click or accidental action could cause the post to be lost. Plus, all my blog posts are now stored on Google's server. Call me old-fashioned, but I like my work to be on my computer.

Plus, I'm an Emacs user, I want to edit my work in that editor/operating system/religion. Call me very old-fashioned...
Well, I've finally found a solution. The blog editing software from Red Sweater, MarsEdit, will quite happily deal with blogs on Blogger: downloading old posts and uploading new ones, unlilke some other blogging clients I've tried.

Though it has the ability to launch other editors, like TextMate, to edit posts this does not work immediately with Emacs. MarsEdit doesn't ever notice that Emacs eventually closes the file. The MarsEdit developer was very helpful. He pointed me to a page describing what MarsEdit is waiting for, and after a bit of Cocoa hacking I can now edit my posts in Emacs. Woohoo!

Well, that's assuming this post appears OK...

And as an aside, I'm pretty impressed with Xcode 3.0. It's got some pretty nice features: the purtiest were these colourful labels that appear directly in your source marking compiler errors and warnings.

Any way, assuming this appears correctly, I'll be buying MarsEdit, switching to that for blogging, and you should see a lot more posts out of me in the future.

I Don't Get Jeff Atwood

2007-09-22T17:24:00.000+10:00

I read quite a few blogs, on different topics, but there is (obviously) a whole bunch of computing ones in there. I read some of the big name ones, like Joel on Software and The Old New Thing, as well as some that could possibly be regarded as second-tier in the blagosphere like Intertwingly and Steve Yegge.

But one big name blog I don't read is Coding Horror, by Jeff Atwood.

His posts appear pretty often on programming.reddit.com or digg.com, friends (who I respect) read him and recommend his articles, but I just can't figure him out. Sometimes his articles manage to completely miss the point, and other times he reasonably succinctly describes a simple concept.

For example, the post Building a Computer the Google Way contains an historically interesting photo of Google's first server. This server is interesting because it is fairly clearly a cobbled together set of home-built servers with some custom designed hardware. These servers were built by Google from the PCB under the motherboard on up. Sure, they used commodity parts, but I can guarantee there are some unique pieces in there. Probably in the vicinity of the interconnects.

This is an interesting example of how a successful company will control everything that possibly relates to their business. You can't afford to rely on the off-the-shelf components for anything that might be technically critical. For Google, indexing and querying speed were hyper-important. So they built their own hardware. Amazon did something similar when they wrote their own web server. Always remember these example when you're railing against Not-Invented-Here syndrome.

Unfortunately, Coding Horror proceeds to use the Google example as justification for why every good programmer should build their own PC instead of just buying one from Dell. I'm sorry, but I can't see how reading specs on PCI slot counts on a motherboard are going to lead to building the kind of server that could support Google's load, or perform their proprietary indexing any faster.

However, on other occasions, Coding Horror does actually manage to explain concepts accurately and clearly. Though I'm usually left thinking, this only occurred to you now? See Everything is Fast For Small n and You're Probably Storing Passwords Incorrectly.

He does, however, write well. And there lies a real skill. If only he could be brought up to speed on some actual modern computer science and software engineering concepts, we might have something.

An Effective Use of Limited Transport Infrastructure

2007-09-03T17:03:00.000+10:00

I wonder if I'll be arrested as a subversive for posting this?

Method Dispatch and Scheme

2007-09-03T14:45:00.000+10:00

As I work on Shrew and attempt to learn more about Scheme, I've been doing some reading. And in one of my books I had one of those 'Aha!' moments where I really saw the benefit in the Scheme way of doing things.

In object-oriented languages there are two approaches to dispatching methods.

Message Passing: Objects are regarded as actors. Method call is treated as sending messages to those actors. Method dispatch is therefore determined solely based upon the type of the receiver of the method, the types of any arguments is immaterial. SmallTalk typifies this style. C++, Java and most other OO languages have adopted this style (badly.) This is also known as single-dispatch.
Multi-methods: Methods and the objects to which they can be applied are regarded as a cartesian product: objects don't own methods and methods don't own objects. Method dispatch is therefore determined from the type of all the arguments equally. CLOS uses this style. I am not aware of any other widely used languages that also use multi-methods.

The book I am currently reading laid out some Scheme code that showed how to implement multi-methods, and then followed it up with some code that implemented message passing. In both cases the code was extremely simple and easy to understand: the basics for two object-oriented approaches were right there on the page.

And most interestingly, there was no reason why both systems couldn't be used on different objects in the same program.

The book is about general computer science concepts, so it doesn't actually point out what it has just described, and nor does it provide a complete implementation of an object system (yet...) however it is interesting to see such a fundamental part of a object system in so few lines of code. It is the nature of Scheme that makes it possible to implement things that other languages regard as 'part of the language' in your own programs.

It is not possible to do this in Java; it is not possible in C++, not even with template meta-programming; and it isn't even in possible in Haskell*.

Now, that's not to say that having both multi-method and message passing dispatch is itself such a killer feature. But it's going to be kind of hard to beat a language to which both of these things can be added so simply.

* Of course, via differing mechanisms of varying degrees of pain, something like multi-methods could be added to all of these languages. Here I am talking about a simple, transparent at both caller and callee technique for achieving multi-methods and message passing.

Original Policy

2007-09-03T14:05:00.000+10:00

Has a century of compulsory voting and a decade of international neo-conservatism finally killed off the last vestiges of original thought in governmental policy? It certainly feels that way.

As Australia approaches a federal election both major parties are competing based almost exclusively on policies either of cutting taxes in some form or giving money away in some other form. Is there no other mechanism of directing society? Surely there must be.

Look at the first home buyers grant. Instead of just cutting the price of a home, how about some other means of encouraging saving that isn't giving away the beginning of bank account? Where is the inspirational leadership and original thinking that gave us the Industrial Relations Commission; the Truth and Reconciliation Commission; the Marshall Plan?

That's the sort of thing I'd like to vote for.

In my other world of software development, this 'just give money away' approach is known informally as Econ 101 Management; as in it follows an economic model informed by only an introductory understanding of macro-economics.

I'd like to hope that our elected leaders knew a little more. Now please go away and demonstrate that.

Megadata?

2007-08-20T22:54:00.000+10:00

So the big deal is mashups and the massive amounts of data collected by the services being mashed. And this is becoming known as 'Megadata'. Lots and lots you can do when you have all this attention data, the activities of your users, and then when you combine that with what they do on other servers... Wow! The things you can do for your users really is pretty damn impressive.

Well, I was thinking today that with access to data across the activities of individual citizens accurate taxation suddenly becomes really, really easy. I bet you all just shuddered at the idea of the government (pretty much any government) having access to all that, right?

So, why are we so cool with random companies knowing?

Graining and Cities

2007-08-20T20:20:00.000+10:00

In his book The Timeless Way of Building, Christopher Alexander states that people should work where they live. I had some trouble with this at first as it's pretty difficult to achieve. There aren't all that many good jobs, and the chances of already living in the same area as the job are pretty low; what are you supposed to do? Move?

In spite of this, I did keep reading. And Alexander does expand on the idea some more.

The point is not to live in the exact same area as where you work, but to work in an area you could imagine living and, as important, live in an area you could imagine working. When you walk down the street you need to see all the parts of life happening: you need to dodge prams, see people eating and drinking, watch people in their offices as they watch people lounging around in a park. You need to see kids streaming home from school in the afternoon, you need to see the retired out doing their shopping during the day.

And you need all this both where you live, and where you work. You need to feel when you're spending those 40 hours a week away from your home that you are still in a place that is someone's home. Seeing those people and all the disparate things they do forms a connection between you and them. If the place you work and the place you live both have this, then you can easily transplant the connection from home to work and vice versa.

There is a name for all this: community. Community requires all parts of life: babies through to the elderly; workers, shops, services. And if you're hiding away at work for 40 hours a week you tend to forget that not only this is all going on, but also that all these different kinds of people share your world with you.

This all comes down to graining. Think of it in terms of zoning. The zones of use form grains within an area. In older areas of a city, this graining will typically very fine. That is, you can walk a short distance down a single street and pass houses, offices, cafes, libraries, shops. In newer areas, the graining becomes larger, until you reach the 60's ideal of a city composed of satellite towns: rings of suburbs with a shopping/office district at the centre.

And in there, a community dies. Communities need people to spend time with them to grow, if a significant chunk of a community's demographic disappears for a big chunk of the day then it can't really survive. Look for the fine graining, try to live and work there if you can, but always enjoy the graining.

V for Vendetta

2007-08-19T22:25:00.000+10:00

V for Vendetta
Alan Moore & David Lloyd

Another comic, though this one would definitely fit well within the hazy definition of a graphic novel. And this is what I was hoping to get when I started reading comics. On the surface, it's a fairly straightforward tale of a downtrodden society inspired to rise up and throw off their oppressors by an enigmatic leader. But, it's just told so well, and being a comic adds something quite different to the story. There are also enough new angles, and the refusal to go with the obvious pay-off adds interest.

Compared to The Sandman, V for Vendetta is a much more traditionally drawn comic; the panels are all regular shapes, with a regular left-to-right/top-to-bottom flow. And it just ain't anywhere near as purty: the colouring and graphic detail just don't match the amazing art of Sandman. It does have a very nice dark, brooding atmosphere, however.

V was first conceived back in 1981, and then written and published over the remainder of the decade, finally finishing up in 1989. My edition includes a short introduction by Alan Moore, written in 1988. And this is an interesting comment on those times and these. The 80's were a decade of conservative dominance, particularly in the UK and US. Moore writes about the Thatcher government doing everything it could to outlaw homosexuality. Moore was feeling deeply disaffected with his home country: he wanted to take his family and leave.

Now, (hopefully) coming to the end of another decade of worldwide conservative dominance, there is some hope to be had here. Sure, rights have been eroded and the world in general is now a less trusting and friendly place. You may look at leaders and policies and see just a bleak descent into a well of fear and anger.

But, we haven't gone that far backwards. Australia now has an openly gay senator: Bob Brown. Try though the conservatives and neocons might, the world will move forwards, and inevitably open up. The conservative decade is coming to a close, and just maybe the next decade will be a world better fit for everyone to live in.

Feersum Endjinn

2007-08-19T12:32:00.000+10:00

Feersum Endjinn
Iain M. Banks

That's Iain M. Banks the sci-fi writer, not Iain Banks the modern literary fiction writer. It's actually the same guy, but he publishes his sci-fi with the middle initial. I'm working my way randomly through all of his sci-fi and I will probably try to read most of his literary fiction as well. I'm yet to hit one, sci-fi or otherwise, that I haven't enjoyed.

A friend warned me just before starting that this one was a little heavy going, due to the phonetic pidgin. In the end, it wasn't that bad. Not as bad as Cloud Atlas, but personally I don't find pidgin that hard to read, others might so be warned.

Imagine a future where nothing goes particularly wrong for the Earth and humanity. There is no apocalypse to be post-; technology keeps advancing, absorbing more of the little things we need to do, gently wrapping a comforting, protective blanket around everything.

Now, project that world and the people living in it forwards several hundred thousand, or possibly million, years...

And then something does go wrong.

Feersum Endjinn was published in 1994: I'd say that aspects of the Matrix's depiction of the machine world inside the machine are inspired by this book. Apparently, there is also some question about whether this book is actually part of the world of the Culture, from some of Iain M. Banks' other sci-fi novels, even though it isn't identified as one. It seems that this probably isn't the case, due to the back story surrounding Consider Phlebas, but this could be read as a hypothetical explanation of how the Culture originally formed.

Thoughts on Scheme

2007-08-18T19:50:00.000+10:00

There's nothing like writing a complex application in a language for really learning it. I've been able to read and write small pieces of Scheme for years. And I've known enough of the underpinnings of the language to write a (very) small Scheme interpreter in the past. But, part of the point of writing Shrew in Scheme as opposed to, say, Ruby is to really learn the language.

I haven't been at it for long, but here are some random thoughts and comments so far. Much of this will change as I figure out how to use Scheme properly. These comments are also mainly a comparison of the only other functional language I know well: Haskell.

The syntax is not very clear: everything looks the same. This prevents you from scanning the code to get a feel for what is happening. The indentation helps, but fundamentally every operation is just a parenthesized expression. I am certain that this is just cultural, with more exposure I'll get better at reading; don't count this as a negative.
I really miss pattern matching. How can you program without it? I'm going to have to look into destructuring-bind.
Dynamic typing is sweet. As well as all the other benefits, dynamic typing + modules = as much encapsulation as classes give you in C++.
The standard libraries are pretty paltry. The SRFIs add a lot, but it's still nowhere near as rich as pretty much any other language. Maybe I'll explore Common Lisp at some point.
Currying is not just a implementation technicality, it's useful for designing data structures. I miss currying as well.

What is Shrew?

2007-08-14T21:45:00.000+10:00

Shrew: a small insectivorous, mammal; one of the descendents of the first mammals to evolve. Related to voles and hedgehogs. They are not rodents, and are not in related to mice and rats.

It's also an animal name containing the letters S, R and W: I use animal names for my project code names, and I needed this one to contain those three letters. The R and W are for REST and Web Server respectively, the S is for Scheme. I'll get to that later.

The idea of Shrew is to build a web server that makes the creation of RESTful web services easy. Popular web servers are still designed as file servers: a web server exposes a directory tree, and then allows certain files within that tree to be executed on the server, to generate the content to send to the client. This encourages (even forces) web applications and services to be built around files. RESTful services are supposed to built around resources. These resources are meant to be the logical 'things' that exist in your service: not files. You don't want to have ".aspx" or ".jsp" or ".php" appearing in your resource URI. It should look something like /person/visit/730. And one of the rewriting modules should not be how you have to get there.

Shrew will make resources the centre of an application or service. The resources exposed will be listed and mapped onto a URI form, and also a piece of code to handle that resource. Shrew will also make no attempt to pretend that an application is not running over HTTP. All request headers will be available to the application; the application will be able to override the generation of response headers on a response-by-response basis; the application will be able to specify particular response codes.

Most importantly, dispatch of incoming requests will be performed not just on the request URI, but also the request method: GET, POST, PUT or DELETE.

The point of providing this level of control is to allow applications to be properly RESTful. For example, look at the Etags header. When a request for a URI is first served to a client a web server can generate and attach an Etag header. If the client requests that resource again, it can include the Etag header in the request. The server can then use the received Etag to determine if the resource has changed and needs to be re-sent. Sounds fantastic, right? No need to re-generate a complex page on the server, or send down a large amount of HTML. The server reduces its load, and the client can redisplay faster. A popular web framework even automatically implements Etags for you. Everyones' happy.

Not so fast. That popular web framework generates its Etags by taking an MD5 hash of the page before sending it to the client. This requires generating the full page, everytime. Even when it hasn't changed. It saves the bandwidth, but not the server processing time. To do Etags properly the application needs control of headers, it needs to be able to match the response to a particular resource.

The web changes rapidly, there could be other valuable headers coming soon, there could be interesting and unique ways of using existing headers. Instead of trying to forsee all those cases, Shrew will simply provide default implementations for headers, but allow an application to override those.

Of course, there's more to web applications than requests and responses. Part of the reason those rewriting solutions are distasteful is that the generated pages needs to use URIs in the rewritten space, not the developers directory space. Shrew will allow resources to link to other resources, by name and id, generating the correct URI at runtime.

Shrew will also include a library for writing markup. This will not be template based. The markup will be written in a Shrew DSL and then executed to produce the HTML. Inspired by Markaby. This markup will be renderable as HTML for serving to a browser, and also as XML for serving to a web service client. This will probably require some hinting, as large parts of the HTML will not be required in a XML view. It's going to be interesting to get that to work...

Finally, in my introduction to Shrew I mentioned some unusual technology choices: the whole thing will be written in Scheme. This is the other half of Shrew as learning project. As well as really getting a handle on REST, I want to learn Scheme. I can read Scheme and write small projects in Scheme, but there's nothing like writing something large for really getting a feel for a language.

In particular, Scheme's killer feature to me is 'data is code.' Not that code is data, nor the macro system, nor that it's a functional language. I'm really interested in exploring the data is code concept, and a web application/service framework seems like a good place to try that.

The Clown House

2007-08-13T19:50:00.000+10:00

Over in Darlinghurst, on the corner of Bourke and Stanley Sts, there is an old, very run-down, abandoned terrace. Fortunately, though it is essentially a ruin, it has not been knocked down. If you happen to approach this house from the back, by walking up Stanley St, there's a sight you can catch over the back fence.

I've no idea who painted it, or how long it's been there. I like it; it's nice to see these random expressions of art hidden away in the middle of the city.

The New Project: Shrew

2007-08-12T18:50:00.000+10:00

I've been reading and thinking about RESTful web applications and services quite a bit recently. The last large web application I wrote evolved into a RESTful web application: readable, bookmarkable URLs; resources with unique addresses; and alternative renderings of a resource depending on the request.

It would have been a short step to complete this evolution and then expose the programmable interface as RESTful as well. This all happened without being aware of REST: it just seemed like the logical way to design a web application.

REST is starting to make a big splash now. Web services have been around for a few years, and many systems have been built upon them. Beyond the architectural considerations, it seems that the main differences between the two approaches is that it is easier to write clients for a RESTful service, and it is easier to write the service using SOAP.

Up until recently most of the development of web services has been for systems that do not cross organisation boundaries. Therefore, the consumer of the web service has usually been the programmer who wrote it. Or, if not actually the same programmer, then a programmer in the same organisation. SOAP works well in this environment as one development tool can be mandated and this tool can generate all the painful code required to consume a SOAP service, and when the SOAP service changes every single consumer can be informed.

However, now there are many services being exposed on the Internet that provide useful data. Services such as Google Maps. It is becoming more common to write applications that consume a handful of services from different organisations. Suddenly, it is more important for the client to be robust and easy to write.

Unfortunately though, it is still much easier to write the service using SOAP than REST. Most web servers don't deal well with the concept of the URI not referring to an actual file on disk, and there aren't many frameworks that put resources rather pages at the centre. There seems to a hole in the eco-system here.

I'm going to take a shot at filling it. For this project, as I'll be doing this for interest and to learn, I'm going to talk about what I do as I do it. Ruby On Rails will ultimately provide the full RESTful service framework that the world needs; I'm not expecting to make money off this project. No need to hide anything.

I've started writing some code, and planning a simple design. I'm going to make some apparently unusual technology choices, but this is for my learning. I'll talk about the technology and design in some upcoming posts.

The Sandman, Vol 2: The Doll's House

2007-08-10T21:35:00.000+10:00

The Sandman, Vol 2: The Doll's House
Neil Gaiman

Second volume of trade paperback editions of the Sandman comic series from the 90's. Clearly I liked the first one enough to keep going. And this one has kept me interested; I will be reading the third one. Unfortunately I still don't know how to write about comics very well...

Anyway, after this one I'll be reading V for Vendetta by Alan Moore and then probably onto The League of Extraordinary Gentleman, also by Alan Moore. We gave my younger brother Frank Miller's Batman, The Dark Knight books. I'll be hitting him up for some comments, and maybe reading those as well.

One thing I can say, comics take mere minutes to read. Grab a few and just churn through them, no long term commitment, no worry that you'll spend hours, days, weeks reading a book. A good way to just fill in some time with some (very) light escapism.

A Subtle Request

2007-08-09T21:30:00.000+10:00

A few weeks ago, on the 5th of July, we came home to find the following subtle request:

Our London had pushed her empty bowl across the floor, and left it right in front of the cupboard we keep her tinned fish in. I wonder what she could have been after?

On Books and Trains Stations

2007-08-08T21:50:00.000+10:00

I didn't want to say anything about Harry Potter and the Deathly Hallows, but I will say that I deeply approve of the use of the train station. Train stations are magical places: journeys begin and end there. When they were first built, lives ended and began there. Train stations are intersections of everyone's world: your world with all the people around you; your old life with the new life awaiting at the end of your journey. With technology, the train station has largely been replaced in this role with the airport. Which is why it's such a damn shame that, with the exception of the now abandoned TWA terminal at JFK airport, airport architecture is so utterly anonymous and boring.

The builders of train stations through the 19th and 20th century understood the importance of what they were doing. You can't walk under the enormous, soaring arc of the roof of Paris' Gare de Lyon without feeling something. You may say that a train station requires an enormous roof - you need something to fit the trains under - but there's more to it than that. A train station does not require the main concourse of New York's Grand Central station. Humans required that grandeur for a building this significant to their lives.

And this is why I'm so disappointed in airports. Books aren't switching from train stations to airports because airports aren't inspiring or significant. They look like low, bland, corporate office blocks. Full of bland, inoffensive corporate colours, with plenty of practical reusable furniture and rooms. As train travel becomes less and less common, we're in danger of losing a whole raft of ideas and images. In 15 years time will a child reading Harry Potter who's never been inside a train station really understand the significance? Will that chapter grab them? Do children reading now understand this? Architecture is not just about building the most practical, useful building for the cheapest price. Architecture is about shaping our world, and thus our culture and experiences.

Harry Potter and the Deathly Hallows

2007-08-08T18:05:00.000+10:00

Harry Potter and the Deathly Hallows
J.K. Rowling

Don't worry, there won't be any spoilers, or anything even remotely like a spoiler in this post. I'm not doing that to this book.

I liked this book, a lot. It would probably be my favourite of the series. The ending really nicely wraps up the story without leaving any major open holes. Everything that is supposed to happen, happens. Which, according to Rosencrantz & Guildensten Are Dead, is the definition of when a story is supposed to finish.

Two days before reading this book, I was talking to a friend at lunch about books. He was complaining that the ending of Neal Stephenson's Cryptonomicon doesn't live up to the rest of the book. I then made the extraordinary claim that no book I've ever read had an ending that could live up to whatever expectations had been built during the previous few hundred pages of the story.

And then this book comes along and makes a liar of me. Maybe my expectations were lowered by reading a kid's fantasy adventure; maybe it was exhaustion: finishing at 4.30am after reading continuously all day. I'm a slow reader, I know. Whatever it was, I thought the ending actually worked.

Many people choose to regard Harry Potter with some funny mix of amusement and contempt. You're probably missing the point. It's a fun, easy to read fantasy epic that doesn't take itself too seriously. There is a special skill in writing something that is both easy and enjoyable to read.

Harry Potter 1-6

2007-08-07T18:30:00.000+10:00

Harry Potter 1-6
J.K. Rowling

In preparation for the upcoming release of Harry Potter and the Deathly Hallows, I followed the fantasy fan grand tradition and re-read the entire series from the beginning. It took me about 10 days, including staying up til 4.00am on the Friday before the release of book seven, to finish them all. It was worth it in the end: I cared deeply about the characters and their fates; perfect for finding out how it all ended.

Some random thoughts from reading them a second time.

The Philosopher's Stone will always be a sentimental favourite. It has an innocence and magic that the other's can't match. The surprise of this enjoyable world can't be repeated.
The Prisoner of Azkaban is the strongest; The Chamber of Secrets is the weakest.
Ron is a lot less irritating on paper than he is in the movies.
The big action sequence at the end of The Order of the Phoenix is disappointing.
The Philosopher's Stone was a test book: will this sell? The Chamber of Secrets was 'Oh no, we're on to something get this out quick!' The Prisoner of Azkaban was introducing some new characters and casting around for the real story. The Goblet of Fire is where the story really started. The Order of the Phoenix and The Half-Blood Prince were middle books.

But no matter how I read them, and what different spin I read into them, I loved the story. And to be honest, it fits together a little too well for that final (cynical) analysis to be entirely accurate. All those of you planning on re-reading The Deathly Hallows but slower, I can recommend starting right from the beginning again, if you haven't already. You will get something out of it.

Perl

2007-08-06T10:59:00.000+10:00

I don't usually post entries that just refer to someone else's post and then comment, but I thought this one was worth it.

For the sake of comparison, people have been saying that Cobol has been dead for more than 20 years, but there are over 1000 Cobol jobs per month posted on Monster.com, and more than 5000 per month for Perl. - JT Smith, posted by chromatic on O'ReillyNet.

Everyone got that? Perl is doing absolutely fine as a language; there are still heaps of jobs going for COBOL programmers, and Perl has even more! Clearly this is the only measure of a language's health and therefore we should all kindly stop pointing and laughing.

Personally, I think this couldn't come sooner. A technology isn't really dead until people feel the need to loudly proclaim how not dead that technology is. People have been declaring the end of C++ since Java first arrived, but no one is defending C++ yet... more's the pity.

Oh, and I particularly love his analogy to COBOL programmers: smooth.

Against the Day

2007-08-05T22:50:00.000+10:00

Against the Day
Thomas Pynchon

This was an absolutely massive brick of a book. 1,085 very dense pages. The longest sentence I remember encountering was a page in length. He's maintained his pattern of hard to read, but very rewarding, books. Though this is only the second Pynchon I've read, I will be going back for more, call me a masochist if you will.

Mathematicians are major characters; reasonably common in sci-fi, but most surprisingly the maths is depicted accurately.

Except... ideas of science are never considered in a rational vacuum. These ideas are always filtered through unbalanced, biased, imperfect humans. We attach extra value to these ideas; we invest in them, growing them beyond a purely rational consideration into something larger. Something that a person could start to believe in. Don't be distracted by false claims of inaccurate ascriptions of religiosity. It's not there. These both emerge from the same base. The linguists who analysed the unusual language of the Pirahã ended up spending sometime literally not speaking over a disagreement in interpretation of language constructs. This is more than science and rational discourse over ideas: this is belief.

No scientific proof is ever accepted from being derived from first principles. Russell showed that isn't actually possible. Well, Russell gave it a shot; his failure showed it wasn't possible then Gödel showed why.

This book is an historical fiction set around the turn of the 20th century: but if it was set now then it would be a near future sci-fi. Something in the line of Stephenson or Gibson where the possible effects of recent discoveries are explored. That's not what the book is really trying to explore however. It feels like a daydream in the end. After exploring many different forms of literatures, sciences, ideas and political battles through the pivotal period of the 20th century the story lazily surfaces into what feels like our world.

You're never quite sure where you've actually been.

For London

2007-07-30T19:15:00.000+10:00

Funeral Blues
W.H. Auden

Stop all the clocks, cut off the telephone,
Prevent the dog from barking with a juicy bone,
Silence the pianos and with muffled drum
Bring out the coffin, let the mourners come.

Let aeroplanes circle moaning overhead
Scribbling on the sky the message He Is Dead,
Put crêpe bows round the white necks of the public doves,
Let the traffic policemen wear black cotton gloves.

He was my North, my South, my East and West,
My working week and my Sunday rest,
My noon, my midnight, my talk, my song;
I thought that love would last for ever: I was wrong.

The stars are not wanted now: put out every one;
Pack up the moon and dismantle the sun;
Pour away the ocean and sweep up the wood.
For nothing now can ever come to any good.

We'll always love you Pundy.

The Box Doesn't Exist

2007-06-20T21:43:00.001+10:00

One word that always causes me to shudder: 'Content.' This is now apparently the standard word that people in the software industry use to refer to that which is created and owned by their users.

To me it's just the latest horrible, bland, meaningless MBA word. Except it's not. Like all words, 'content' implies something about that which it refers to: that it's irrelevant and doesn't matter, that anything would do in its place. Content is what you put in a box so that the box is no longer empty, it doesn't matter what it is, it's just there to fill the box.

Sure, as software people you're all very proud of your box. You've put a lot of effort into making your box very pretty, and emergent, and Web 2.0, and now you just need something to fill it up. When all you do is build boxes it's very easy to only see boxes.

Well, guess what? Your users don't even notice the box. That content that you regard as interchangeable filling is all they care about. That's what they've spent time, effort and emotion in creating. They don't want to just dump it in some box somewhere, they want it treated specially and with respect. Referring to it with some bland management-speak term is not showing respect.

Words matter. The words you use to refer to something shape your perception of it. If you're in the business of building something where users can display their creations you need to remember that those creative works are the entire purpose of your web site. So don't refer to it as 'content' and then focus on the box. Remember the box don't exist, and treat your user's works with respect.

Don't call it 'content.'

Ticket'd

2007-06-20T18:13:00.000+10:00

Spotted out our window one cold winter night:

A drug bust? A shooting? International crime syndicate? Nah, just a P-plater getting a speeding ticket.

Is Something Burning?

2007-06-13T14:45:00.000+10:00

We suspect this singe:

On this cat:

Was caused by one of these lights:

Trendy though they may be, it seems that if you have a brain the size of a peanut and a love for heat, heat, glorious heat, halogen lights set at floor level are a little too attractive on a cold winter's morning.

If they hadn't attached themselves to humans cats would long since be extinct. Probably through some accidental fire.

Where-house?

2007-05-29T20:14:00.000+10:00

These were taken in January, after we picked up the keys but before our stuff had been moved in. By now I figure that those of you who can make it here to see our new home have, so these photos are for the rest of you.

The building is a converted warehouse from the late 1890's. Until (I think) 1994 it was used as a warehouse. For most of that time as the headquarters, offices and printing rooms of the Sunday Times, Sydney's third Sunday paper. From the 1840's to the 1890's there were smaller buildings on the site, and before that, the area was market gardens.

The envelope of the building (the external brick walls) is original, and a lot of the original materials have been reused in the reconstruction. There's an old wooden pillar by the front door, for example, and some of the floor joists are original, though the floors themselves have been moved. Moving the floors has given us the 6m ceiling I've mentioned before, and the bedrooms have been tucked into a mezzanine that hangs over and opens out into the living area. The black rectangle in the photo is one of those openings.

They don't mention the joys of living in a converted 1890's warehouse in the trendy New York novels: our wall leaks. That's right: wall. Ahh, well. I call it character.

Embrace Your Inner Geek

2007-05-28T11:24:00.000+10:00

Why bother reading long posts about esoteric language features that your language doesn't provide, or even if it does, it's not a good idea to use too often?

Two and a half reasons really.

Understanding your language is important. This is not magic we're doing here. It's engineering, one of the purest and most mathematically oriented branches of engineering you're likely to earn money engaged in. When it's magic you're not supposed to know what's going on, that's why it's magic. You are supposed to understand engineering though. Languages are the most complex pieces of computer software. You understand complex things by exploring the boundaries. What do they do well? Why don't some things work at all? How could they be better? All professional programmers should be able to name several critical failings of the language they are currently working in; if they don't hate it all together.

Related to that, when programming it's not enough to memorise the class and function names from the whopping great big framework of the week. You'll generally have inferred something about how the framework works underneath. You'll know that before calling function A, you had to instantiate class B first, unless you called function C first, because that happens to create a B for you. Blah. Languages are the same. Why does a variable declared inside a while loop disappear outside? This isn't voodoo. The goblins don't eat it. There's a reason. The language is implemented in a certain way by the compiler and runtime combination that causes these things to happen. You should understand that relationship. Every thing we write is built on some similar layer of abstraction. You need to understand (at least theoretically) how to implement the layer immediately beneath you or you'll end up wandering off into some abyss you didn't even know was there.

And finally, we're professionals, dammit. Where's the intellectual curiosity? Where's the 'Hey, that's cool. Now, how do I break it?' Or, 'I wonder what will happen if I do this...' Or, 'How does that work?' And most basically of all, 'Why?' Geeks are supposed to be into that sort of thing. We got beaten up in high school because we were enthusiastic about things it wasn't cool to be enthusiastic about. Now there's a lot of money to be had out of that enthusiasm. Let it out, embrace your inner geek. Enjoy this amazing, complex, unique, ever-changing world around us. Explore those things you've always felt a connection to. If you're in computing, there's a good chance that's something mathematical. Most things in computing are... For me it was languages. What's it for you?

Oh, and last week at the pub we had a big discussion about category theory, higher-order functions, contination passing web servers and finishing up with the difference between a program that generates a program and writing a custom compiler and interpreter. Geeky you may say. Well, today I noticed a colleague reading up on continuations on Wikipedia. That's what I'm talking about.

The Sandman, Vol. 1: Preludes & Nocturnes

2007-05-27T16:09:00.000+10:00

The Sandman, Vol. 1: Preludes & Nocturnes
Neil Gaiman

Is that how you credit authorship of comics? I don't quite know. There are artists involved, and I'll leave it at that for now. But yes, that means I'm reading comics now.

I've never been a comics reader. Sure I read some comics as a kid. Asterix and Obelix and Footrot Flats were both pretty important to me. I'm pretty sure I've read them all. I also read the occasional Tintin, and there was one dark single issue comic Where the Wind Blows that I discovered in high school that was pretty damn good. But I never read seriously the Marvel and DC comics involving Spiderman, Batman, etc, etc.

Sure I'd get hold of a comic here and there. But it would usually be issue 23 out of 148, then several months later I would read 92, then 5, then the grand finale. Overall, the story didn't make a whole lot of sense...

I have always been a big fantasy reader, and for the last five years of so I'd been hearing a lot about Neil Gaiman. Very popular, very well regarded. But when I looked into him I found he'd only written a couple of novels and a collection of short stories. Strange. It usually takes a long line of novels and an epic series or two to build this kind of cult following. Looking closer he was always credited as 'gaining his following through the cult 1990's comic series The Sandman.' Interesting...

After talking to friends, reading blogs and then reading Pynchon's Gravity's Rainbow, my interest in comics, especially the more modern darker ones was lifted. For example, I thought the V for Vendetta movie was pretty damn good, and apparently the comic was significantly better.

So, on finding Kings Comics on Pitt St, here in Sydney I decided to start with The Sandman. This is the first volume and so far I'm impressed enough to go back for the next. It's a dark, kind of gothic horror fantasy story. I can't review this as any sort of expert comic reader, but I enjoyed the stories and the artwork. Most importantly for me, the character of the Sandman was one that interested me. He is not clear cut. Not a hero, not a villain and not always someone you appear able to trust.

For me so far, comics are looking good. I want to read more Sandman, V for Vendetta, and as Damana also bought volume one of The League of Extraordinary Gentleman, I'll be checking that out as well. Any other recommendations?

Stupid Software Tricks

2007-05-27T15:07:00.000+10:00

Or, My Computer Was Possessed!

Last night I was following some blogs, and I found a link to a kind of rubbish photo sharing site. I won't mention the name here, for reasons that shall become clear. It's been around for a few years and has gone through some major rounds of publicity and then failed. It's now into it's third beta in a row. In short, going nowhere real fast.

Anyway, I was curious about their site, so I went to read their news page. When I'm using a computer I tend to leave what I was last doing open on screen. On coming back I can then pick up where I left off. There's always a browser open, usually with four or five tabs I want to read.

I open the photo sharing news page in a tab and then wander off. Our apartment is a loft-style space in a converted 1890's warehouse. The entire apartment is basically open-plan with the upstairs bedrooms opening out near the six metre ceiling of the living room. My computer is in a study that opens into the living room, downstairs.

At 4am this morning the entire apartment was awakened by the these bizarre screams, crazy distortion and static. What the hell is going on? Is this something from the pub next door? Have our neighbours turned some movie on? Has one of our cats stood on the TV remote again?

I stuck my head out into the living room to try to work out where the noise was coming from. The weirdest thing was that I couldn't understand the noise. I could hear voices that seemed to speaking English, and I could pick out words, but I could not understand what was actually being said.

Coming downstairs to investigate closer, the noise was coming from my computer. And it really was strange screams, distortion and static. Being closer made it no better. I started closing tabs in Firefox, as that was all that was open. Eventually I closed the photo sharing news page and it shut up.

I've never seen anything like it. The news page had a bunch of embedded videos from all over the web, and it was like Firefox and Flash had buffered the videos, and then after several hours of boredom started to explore what interesting sounds could be made by sampling and looping the soundtracks.

Spooky.

And with apologies to Yasmin, who was visiting us this weekend and was very rudely awakened.

And I haven't linked to the site because they really don't deserve even the feeble publicity a link here would grant. I found them through the Blagosphere, and I don't post vias.

A Conspiracy

2007-05-23T19:00:00.001+10:00

I don't believe in the Internet. It's a conspiracy of network engineers... with a lot of pigeons.

Continuing on Closures

2007-05-18T23:19:00.000+10:00

This is the (long awaited) second part of my tutorial on closures. This part describes how your language's runtime might provide closures. I've assumed that you understand what a closure is, if not, read the first part of this tutorial first.

So by now we know that we can think of a closure as an object: an instance of an automatically generated class. This class has just one method: apply() that executes the body of the closure. The class has one member variable for each variable in the active scope at the time the closure object was constructed.

But local variables are only available to the function that defines them and even then only for the lifetime of that particular function call. How then is it possible for a closure to access these values? To explain that I'm going to briefly go over how functions work, bear with me if this all seems a bit basic.

Sometime back in the 1950's, during the design of ALGOL the concept of the call stack and stack frames were discovered. The call stack is just a list of called functions, implemented as a stack so that as functions return they can be popped off the call stack, leaving the calling function on top. A stack frame is the object that is pushed onto the stack to describe a function. A stack frame contains the line of code to return to when the function returns, the parameters to the functions and the local variables of the function.

Every invocation of a function causes a new stack frame to be created and pushed onto the call stack. Among other things, this makes recursion possible. Early versions of FORTRAN don't support recursion: no call stack.

Your program executes; functions are called; stack frames are constructed; values are calculated; frames are updated; and return values are returned. But every time a function returns its stack frame is destroyed, and the very next function called will completely overwrite the memory where the last stack frame was. So how can a closure always be guaranteed to still be able to access a local variable?

Earlier I referred to a stack frame as an object. In languages like C and Java a stack frame is not an object at all; it's just some conventions for how certain pieces of data are stored. All of which are calculated and laid out ahead of time by the compiler.

But what if the stack frame was an actual object stored with other objects on the heap? What if, instead of a stack, a linked list was used to connect the frames? Hmmm... interesting... The linked list would represent the call stack. Every new stack frame would be constructed with a pointer to the frame of the current function. Like all other objects, these frame objects would be subject to the garbage collector. This prevents a frame from being destroyed until it, and all functions it called, have returned.

Once the runtime provides this implementation of frames and the 'call stack,' a closure can include a pointer to the current frame. When the closure needs access to anything in its containing scope it can now just use this pointer. And as the frame contains a pointer to its calling function, the closure can now work its way back through any scopes it's interested in.

Extra Credit: The ability to work backwards through the scopes of all the functions in the call stack is known as dynamic scope. It's pretty uncommon now, as it's virtually impossible for a human to understand in any kind of complex program. As a result languages that support closures generally provide lexical scoping. In lexical scope the structuring of the program text defines the scopes available to a function; and therefore a closure can only use the local variables of the frame immediately referred to by the closure. But enough on scoping; for more compare early versions of Lisp to Scheme.

Frames as a linked list would not make any difference to your program as the act of reading the data stored in a frame is still hidden by the compiler and runtime.

And that's really all there is to it. Closures can now access local variables that seemed to only exist at the time the function was defined. The reference by a closure to a frame behaves like any other reference: the frame will not be destroyed until the closure is destroyed.

Hmmm... didn't I mention earlier that a frame also contains a line of code to return to? Doesn't that mean that any frame can be safely returned from? Yes, it does. Now, if we move our current line variable into the frame itself we have what's called a continuation. That is, a single frame object describes the current values of all local variables for a function, the next line of code to execute in the function and which function to return to. And that function in turn also has all the same information preserved.

If the language then allowed it, you could pick up that underlying frame object, store it away somewhere and then use it to later on continue execution where it was previously left off. Hence the name: continuation. To pull this off the language needs to provide two features.

Firstly, the ability to get a reference to the frame object for the currently executing function, at the same time halting execution. Secondly, the ability to call that frame object at some later point and have execution pick up exactly where it left off, as if there was no pause.

Continuations as a concept are regarded as 'difficult.' As a result, even though most languages that provide closures could also provide continuations the programmer is not given direct control over them. Some languages do provide this control: Scheme (noticing a patten here?), Ruby and Smalltalk, amongst others.

So what are these good for? Before getting into an example of how continuations can be used, I want to say one thing: powerful language features are always useful. If you're not used to having a feature in a language, the first time someone shows it to you the immediate reaction will be: 'Sounds cool; but what use is something like that?' A fairly normal reaction and simply the result of not having the feature available. Once you've used a language with an 'advanced' feature you'll find all sorts of uses, until eventually you won't be able to imagine coding without it.

Anyway, onto an example. When writing web applications a big problem is the enormous gap between the page displayed in the user's browser and where the server thinks the user is up to. Continuations can make this almost completely disappear. Once the application on the server has prepared a page, instead of winding up the call stack and then returning the page to the user a continuation could be captured. This continuation is stored on the server and a key is sent to the browser to be returned with the other data on the page. Using the key, the continuation is invoked, and execution of the function picks up where it left off.

The code running on the server to handle screens in the application could then look something like the below. The statement yield causes the continuation capturing to occur. And remember, all of this code executes on the server, there is no AJAX or anything like that.

function checkout()
{
  // The following three functions add UI elements to the
  // current page. These are rendered as HTML. 
  displayCartContents();
  addButton(new NextButton());
  addButton(new PrevButton());

  yield; // Continuation is captured, and the page so far
         // is sent down to the user

  // The user has sent a response back to the server
  // The functions payment() and home are similar to
  // this function: they create specific pages.
  if (Page.SelectedButton == Button::NextId)
    payment(); // Display a page to collect payment
  else
    home(); // The user wants to continue shopping
}

As well as this simple server side logic, the user can then explore multiple paths through the application concurrently, with multiple windows and using the 'Back' button with abandon. You, as a programmer and a program, will never become confused. Pretty nice, huh? There is a web server already out there that gives you this: seaside.st

And any language can make these powerful features available simply by implementing the call stack as a garbage-collected, linked list of simple frame objects, instead of a one-dimensional stack of data. Pretty powerful, and remarkably simple. Makes you wonder what other abstractions are lurking behind small implementation decisions.

Worst Example...Evar!

2007-05-14T19:01:00.000+10:00

Way back in my first year of university, as part of a first year discrete maths unit we were taught introductory formal logic. After introducing propositional logic and covering ANDs, ORs and truth tables, we moved on to implications. For this particular lecture we had a fill-in lecturer. I can't remember why.

In explaining implication to us this lecturer used an example that has stuck with me ever since. This isn't because it was one of those glorious examples that are like a light being turned on; suddenly it is all clear and new vistas of understanding open up. Oh no, this example was the other kind.

One of those examples that distracts with its own internal errors and inconsistencies; that steers you in completely the wrong direction; and, best of all, pollutes your mind enough that you may never understand the original point.

The bad example:

An implication would be when your mother says to you: 'If you clean your room, then you can go to the movies.' So after you clean your room, your mother lets you go to the movies and the statement is true. But if you don't clean your room and go to the movies anyway, the statement is also true.

Huh?

Logically, yes, that's correct. Single implication works like that. But the world doesn't, and it's pretty hard to ignore that little detail.

Bonus points if you can work out who the lecturer was.

Peopleware

2007-04-19T21:48:00.000+10:00

Peopleware; Productive Projects and Teams
Tom DeMarco & Timothy Lister

Note: This review is of a software development book, not a work of fiction. The book that is; the book's not a work of fiction. Neither is the review, I hope.

I've heard about this book very often from Joel, of JoelOnSoftware.com. He recommends it very highly, in fact one of his recommendations is on the back. This is also one of the first 'software development process' books that I've ever managed to read cover-to-cover, all the way through, and in only a day and a half. Usually these sorts of books are very dry, kind of suck and end up just annoying me.

This book is different, however.

Essentially, Peopleware is saying that any attempt to project manage software development is doomed to (painful) failure if it focuses on issues like technology or process and policy. The point is that like most things humans do, the problems are usually sociological. It's all a matter of getting different people to work together - understanding how they think and what is important to them at a very personal level is how you achieve this.

Joel makes a big deal of how important the office environment, particularly a quiet space with a door that shuts, is to any software developer. This book really makes that very clear. All 'knowledge' workers need some level of quiet and privacy, or there really is no point in having them around. The book took the discussion further and points the reader off towards Christopher Alexander, as the entire environment from the building down to the angle of the desk matters.

As an aside, if you haven't heard of Christopher Alexander, then go read about him. He has nothing to do with software development. He's a real architect. Not one of those pretentious software ones. Anyone who ever lives or works in a building should have some idea of what Alexander is on about.

I can't overstate this, if you currently manage software development; would like to some day manage software development; or are a software developer being managed, you really should read this book.

And the thing is, if you aren't in software development but can ignore the references while reading this book, you'll find it useful. The principles apply to any job where the workers are required to think independently to get anything useful done. If you think while working, this book is for you.

The Power and the Glory

2007-04-10T15:04:00.000+10:00

The Power and the Glory
Graham Greene

Reviewing this book really seems a bit ridiculous. Who the hell am I to review this? If you haven't read it, go, go now. It's only 220 pages, won't take you a minute... Don't worry, I'll wait for you.

It also didn't occur to me that I was reading a 'Catholic book' over Easter weekend until I was nearly finished.

This isn't a book about the Catholic sacraments, or about religion or faith. It isn't even about persecution and martyrdom. The value and effect of all those is well understood. This is about something deeper: the moral ambivalence and ambiguities at the heart of everyone. All these characters fit together and draw so much from those around them - misinterpreting each others actions, intentions and clinging to the the symbolism of another person. Their individual actions are small. The overall result may be large, but often it isn't. In the end though, it doesn't matter. Labels are attached, conclusions are drawn, meaning is derived, and the world moves on.

Mandarin

2007-04-08T23:21:00.001+10:00

Some places and people really ain't worth remembering, but hey! At least I got some cool time lapse photos out of it...

Kind of a symbolic progression, actually.

A Fire Upon the Deep

2007-04-07T15:33:00.001+10:00

A Fire Upon the Deep
Vernor Vinge

This book was recommended to me about four years ago by a colleague at SoftLaw. It took me about that long to find it. If you're in Sydney, I can highly recommend the Galaxy bookstore on York St - thanks for putting me onto them, Jen!

The underlying premise interested me enough to keep an eye out for this book for four years. The galaxy is divided into zones. As you move up through the zones more advanced thought and science becomes available. From the Unthinking Depths, where rational thought is barely possible, through the Slow Zone, where there is no faster-than-light travel, to the Beyond, with faster-than-light travel and advanced artificial intelligence. To me this is still an interesting and novel view of the Universe. It provides both an extrinsic motivation common across individuals and species - to move up to a more glorious life that can't even be expressed at your current level - and also a danger and a fear. There is nothing worse than falling into a lower zone. Sounds religious doesn't it? And that is alluded to, although it would have been good to see that developed some more.

The rich potential of this premise left the book as a whole slightly disappointing.

Overall, the story is a quest. Which is another point in favour of my 'Sci-Fi is fantasy set in the future' hypothesis... A quest story is really a journey to the depths of the soul for all the major protagonists. For some of the characters in the story, this was handled very well. For others though you just didn't get the feeling of plumbing depths from which they could never return.

And so, just like Thomas Pynchon's 'Gravity's Rainbow' ruined David Mitchell's 'Cloud Atlas' for me, so was 'A Fire Upon the Deep' ruined by Iain M. Banks' 'The Algebraist.'

To first world citizens, the world is no longer a huge and almost infinite place. Stories where a significant part of the dramatic weight to the quest is the enormous distance that must be travelled to cross a country no longer ring true. We can now fly to the other side of the world in less than 24 hours and for less than $3,000. The other side of the world no longer feels like an vast distance from where we started. But we are starting to stare up into the night sky and imagine just how far those other points of light are. Distances such that after 30 years a satellite has only just left our solar system.

Treks across this almost infinite emptyness are the things that amaze and scare us now. And unfortunately that was exactly the feeling 'A Fire Upon the Deep' failed to capture.

Don't get me wrong, it is a great Sci-Fi story, and the premise is very interesting. It just could have been so much more. Hence, the feeling of disappointment. If you're into Sci-Fi though, I would recommend it.

Shalimar the Clown

2007-04-07T15:08:00.000+10:00

Shalimar the Clown
Salman Rushdie

This book appears to be about Kashmir and so of course runs into that eternal Rushdie question, 'How autobiographical is it?' And as usual, the answer is 'Not at all.' As someone else has put it (gotta love that attribution) this is as much about Kashmir as 'The Satanic Verses' is about Islam.

Instead, 'Shalimar the Clown' is about, well, Shalimar the Clown. This is a book in four parts all revolving around one central character, and his doomed love for a beautiful woman. And even though one part of this book is nominally told from his perspective, this character remains an enigma right to the last line of the last page. He is never truly revealed to you, and you never truly know him. Why does he cause so much pain? Does he feel pain himself? This is certainly not sloppy writing. How well can you ever know another person? How well can you know yourself?

On a mechanical note, the first part of this book was the hardest to get through, at least for me. It starts at the end, presenting you with disconnected characters and no explanations for their distance. Fortunately, this part is only forty pages. The book then dives back into the past and builds the motivation, making you believe. While all the time concealing the true motivation apparently at the heart of everything. But then after all, aren't we just a function of our past experiences? Are their truly any motivations beyond what has already happened to us?

Well! Overall, a very good book. Some of my friends have described Rushdie as a bit pretentious, and yes, I can see that. But, my criteria for a good book (or movie) is simply 'Does it keep me thinking once I've finished?' And on that standard this book passes. Although, 'The Moor's Last Sigh' is still my favourite of his.

Oh, and the Kashmir thing? My opinion is he writes in a setting he knows, and that is always subservient to what the book is 'about.' In this case a very significant percentage of his readers could never know the Kashmir he writes of. It is probably lost forever, just like the mind of Shalimar the Clown.

Book Reviews

2007-04-07T14:37:00.000+10:00

I read a bit, nothing impressive, or unusual, but a few books. And I find it hard to keep track of which books I've read, when, and what I thought of them immediately after finishing them.

So, I'm going to start posting short reviews of books I read as I read them here, starting with the last two books I've read. With any review it can be hard to avoid giving away some crucial detail of the plot. And of course, everyone has a different view of what constitutes a crucial detail of the the plot. In my view, it shouldn't be possible to spoil a good book. It's not about what happens at the end, but what happens to the characters (and you) as you read...

Anyway, I will try to restrict what I reveal about the plot to only that which is revealed on the cover or in the first couple of pages. But if you're really sensitive to 'spoilers' watch out.

Achieving Closure

2007-03-30T07:54:00.000+10:00

I was having a chat over lunch a few days ago about lambda abstractions. The guy I was talking to didn't fully understand what they did and how they worked, but he's a smart guy so he understood it pretty quickly. I explained them using the below example, and I thought I'd have a shot at writing this up for other people to read.

Say you have the following functions in your code somewhere. The code is intentionally simple. If that's a problem, imagine that everywhere you see double, it actually reads encrypt.

int double(int val);

void doubleList(int values[], int valCount)
{
  for (int i = 0; i < valCount; ++i)
    values[i] = double(values[i]);
}

int double(int val)
{
  return val * 2;
}

Now, it's pretty inconvenient to have to declare and define a whole other function for something as short as double. Excuse me while I invent some syntax, but wouldn't it be convenient to be able to do something like:

void doubleList(int values[], int valCount)
{
  for (int i = 0; i < valCount; ++i)
    values[i] = function(int val) { return val * 2; }(values[i]);
}

And that bit with function... is a lambda abstraction or closure. What's happened here is that I've defined and called a function all on the one line. This function doesn't have a name, so it can't be called from anywhere else. But it is otherwise just like a normal function; it appears on the call stack, it can return values and it can even be passed parameters. Which is how values[i] ends up in the val local variable.

In fact the only unusual thing about this function is that it was defined 'inline.' And perhaps the syntax, but I invented that on the spot, so don't hold that against it. There are nicer ways to express this.

This sort of thing is known in some programming languages as a lambda abstraction. That name comes from something called the 'lambda calculus,' which doesn't really matter, but briefly it's a way of writing programs as a series of anonymous functions, each referred to as a lambda.

But why is it also called a closure? That's much more interesting because that name actually says something about what you can do with it.

Technically, it's called a closure because the anonymous function 'closes over' objects within the containing scope.

Which means almost nothing. Looking back at that inline doubling example above it did appear slightly unwieldy. Something like this is more readable:

void doubleList(int values[], int valCount)
{
  for (int i = 0; i < valCount; ++i)
    values[i] = function() { return values[i] * 2; }();
}

Notice now how the anonymous function no longer takes any arguments. How can that still work? The anonymous function is a separate function on the callstack, with its own local variables. How can it access the values array, or the counter i, for that matter?

Imagine the following code:

class DoubleFunction
{
  public:
    DoubleFunction(int values[], int valCount, int i)
    {
      mValues = values;
      mValCount = valCount;
      mI = i;
    }

    int apply()
    {
      return mValCount[mI] * 2;
    }

  private:
    int mValues[];
    int mValCount;
    int mI;
};

void doubleList(int values[], int valCount)
{
  for (int i = 0; i < valCount; ++i)
  {
    DoubleFunction func(values, valCount, i);

    values[i] = func.apply();
  }
}

So, what's happening here? We've defined a class with a constructor and private members that happen to match exactly all the local variables and arguments in scope where we want to construct our closure, this class has only one member function called apply that matches the body of the closure.

This code is precisely equivalent to the version with the closure. And in fact, it goes further than that. This version is how a compiler could implement closures. A closure is an object with exactly one method: apply. So, anytime you see a closure in code just imagine it as an object with a constructor that captures the current scope and you're there.

A few details:

Some may be jumping up and down to point out how inefficient the code with the class is. And they're right. But no compiler would actually do that, it's just a way to understand what's happening under the covers. Real closure implementations can be quite efficient.
That class implementation takes copies of the values in scope. A real implementation would actually take a reference to the exact same value. This is important because it allows the value to change between when the closure is created and when it's evaluated, and have the new value be used at execution time.

I said above that you can imagine the closure as an object. There is a consequence to that. You can take references to objects, pass those around and call methods on them at some point in the future. The exact same is true of a closure.

What do I mean and what use is this anyway? Well, in the following code I'm using an invented syntax for closures. The part in the parentheses to the left of the -> is the parameter list, and the type to the right of the -> is the return type.

void applyFunctionToEveryElement(int values[],
                                 int valCount,
                                 (int) -> int fn)
{
  for (int i = 0; i < valCount; ++i)
    values[i] = fn(values[i]);
}

void doubleList(int values[], int valCount)
{
  applyFunctionToEveryElement(values,
                              valCount,
                              function(int val)
                                { return val * 2; }
                             );
}

Here instead of evaluating the closure as soon as it is created, it's being passed into another function, which will evaluate it. Once for every element in the list. Which is exactly what you could do with an object. Big deal? Well, with that exact same applyFunctionToEveryElement function, you can also write the following functions. Oh, and applyFunctionToEveryElement is traditionally known as map, so I've used that name below - same function though, just a shorter name.

void printList(int values[], int valCount)
{
  map(values,
      valCount,
      function(int v) { printf("%d\n", v); return v; });
}

void scaleList(int values[], int valCount, int factor)
{
  map(values,
      valCount,
      function(int val) { return val * factor; });
}

Both of these functions do completely different things to a list of integers, but use the exact same function to perform the mechanical walking the list part. Congratulations, you've abstracted out iteration. The first function is pretty straight forward, it prints every value and leaves the list unchanged.

The second function is a little more tricky. The closure created there is using a local variable, which the map function has no knowledge of or access to. Remember that a closure captures all objects in scope at the time it is created, and those objects can be used anytime the object is evaluated. That's exactly what's happening here.

How that works requires a few more tricks, so I'll be leaving it for another post. It's related to why continuations and closures commonly go hand in hand. Look out for my crash course in continuations and how to implement closures coming soon.

For now, just know that this works. Closures are good, if your current language has them, use them. You won't regret it. If you, like me, find your language doesn't have closures, then...

What Would You Do With It?

2007-03-28T23:05:00.000+10:00

This is a question I've been thinking about a bit lately:

If you had a computer that was as smart as a human, what would you do with it?

It almost seems obvious; everyone wants a computer that has human level intelligence, people have been working towards that goal for literally 40 or 50 years, but what would they do with it if it was suddenly dropped on them?

OK, some clarification. It's a currently modern computer that is good at doing all the things computers are usually good at (arithmetic, finding files, storing lots of information) but it also happens to be really good at doing the sorts of things humans are usually good at. You know, making an 'educated' guess, pattern matching, seeing links and generally being partially intuitive.

It would not have any special hardware, it wouldn't be able to speak for example. It would look just like a computer, except for this extra bit.

So, what would you ask it? What would you do with it?

I think I can't actually answer this question as I'm not really a computer consumer anymore. Because of my job a computer is no longer just a tool, it's an end in itself. So for my purposes, I guess I'd want it to be able to write code really well. But seriously, what would you as a computer consumer want it to do?

Throwing this one open. Please comment or email me with your ideas, because I really do wonder...

shb75

2007-03-18T21:40:00.000+11:00

Today was the 75th anniversary of the opening of the Sydney Harbour Bridge. To celebrate, the bridge was closed to traffic and people were allowed to walk across. North to South. We were there for the walk, along with a lot of other people. No rabid, right-wing monarchists though...

An icon and engineering marvel like this is really all about the photos though, so here you go.