Dreaming in Code

Dreaming in Code Scott Rosenberg

Rosenberg is one of the co-founders of the online magazine Salon.com, a magazine I've been reading on and off since 2000. After having a bad experience with internal software development, he became interested in how, after more than 50 years experience, we still find, in the words of Donald Knuth, that 'software is hard.'

In the interests of disclosure, I will point out that I got a copy of this book for free from the author after he offered copies to bloggers if they agreed to mention the book. So, this is me (gratefully, happily) fulfilling my end of the deal.

This book is a story, a story of a collection of great developers attempting to write an amazing piece of software. It is not a retrospective story either. It begins shortly after the project has started, but unfortunately couldn't follow the project to completion: the project still isn't done yet.

The Open Source Applications Foundation (OSAF) was formed by Mitch Kapor (of Lotus 1-2-3 fame) to build a new form of personal information management software: Chandler. Dreaming in Code tells the story of how they tried and (so far) failed. With many informative diversions into the theory of software engineering in an effort to discover why software seems to so consistently be delivered late (if at all) and buggy (if it even gets close to meeting the original vision.)

Rosenberg does an impressinve amount of research into the theory. No appropriately read professional software engineer will find any revelations here: this is all stuff you should already know. But Rosenberg is not claiming to invent or discover anything new. In fact, he goes out of his way to disclaim that there is any original research contained in this book. He is a journalist, and as a journalist he has produced a detailed literature survey. The summary in two well-written chapters is useful even for experienced software engineers. I'm sure non-software engineers will find this all very interesting; assuming they are interested in how software gets written.

For software engineers, there is something very interesting here. The internal mechanics of a a team building a piece of software is a very secret thing. Companies are secretive and companies and open source projects want to protect their reputations. Most software engineers only ever see how a project they are working on proceeds, and then they're too close, plus there's no nice summary of what happened. Dreaming in Code is something very valuable to our field: an accurate story of the human side of a software development project. With both the clarity of distance and the accuracy of events recorded at the time they happened.

It was simply astounding how familiar this story was. OSAF and Chandler get some things spectacularly wrong, but then in other cases they do things very right. It's easy to point at the things they stuffed up and claim that you would never make those mistakes, but it's a little too easy to ignore the things that were done right. The good decisions end up forgotten and never noticed.

So what did I think that got wrong? Firstly, and by far the biggest: Analysis Paralysis. This is a mistake that I've seen projects make over and over again. In fact, I'd go as far as to say that it's more common to see this affliction than not. Projects just can't seem to make a decision, stick to it and then start building. The fear of the future locks them solid: what if we make the wrong decision? What if someone blames me for the wrong decision? In the end, ikf the decision wsas so egregiously wrong that it can be traced back to just one person, then everyone else around at the same time is just as culpable for allowing that decision to happen. Everyone, please just get in the habit of making decisions. Rely on those around you to spot a bad direction: that's what they're there for.

Second big mistake for Chandler? People. No surprises there, it's the big and obvious disaster. If you believe that software is hard and you care about your software then clearly you should only work with the best. That's easy to say, but pretty hard to achieve. And in Chandler's case the people problem exhibited in a couple of interesting ways. Before I talk about the people problems that I saw Chandler as having I need to say that any attempt to judge is based solely on what is described in this book. I (obviously) didn't work on this project, I don't know the people, there's only so much I can say. Having said that, I'd also like to say that people problems kill most projects and if we hope to advance we need to get over this fear of talking about the problems with people. Maybe then we can find some solutions. Or maybe that's just an overly analytical geek talking. Why can't everyone just be a nice reducable puzzle, dammit?

There was a permanent employee of the OSAF who was hired quite early and ended up in quite a senior technical position, and this employee was unfortunately the very soul of Analysis Paralysis. Any story about some interminable technical discussion has this particular employee at its heart. He was extraordinarily conservative and wary of making decisions, but ultimately many of the technical decisions came down to him. After a few of these stories, I was left shouting 'Do something about him!' at the book. And I've seen precisely this problem face-to-face too many times to ignore. Once again people, make decisions! You probably won't get it too far wrong. In fact, in this case, the inability to make a decision led to one of their few definite cases of over-engineering.

The second people problem: OSAF planned to run themselves as an open source project that encouraged volunteers. Predictably enough, the only people who could volunteer for any extended period tended to be ex-Apple and -Netscape employees who had already made their fortune and no longer needed to work for an income. From the outside these volunteers presented an interesting problem. They were all brilliant and had done amazing work in the past, but they didn't see this project as any kind of meal ticket. There was no drive for them to get this project finished and out the door. In the end, they come across as people partially on the side-lines, commenting endlessly but never really pushing the project forward. Instead, they were offering endless advice on how things could be done better. Software projects are always cursed with people like this; encouraging volunteers just seems to guarantee it.

But back to the book, I think I've already made clear that I really enjoyed it. I also think the opportunity to see inside another project is infinitely valuable for software engineers and software engineering. Aside from all that, I will say that sometimes the brief newspaper style of the book was a little irritating. On occasion I felt a topic or anecdote could have done with some more depth before moving on.

Software engineers will get a lot out of this; non-software engineers who care about how software (upon which, our civilisation is built) is written will get even more out of this. Oh, and as a professional programmer and wannabe-amateur writer-slash-blogger, software is definitely written - then organically edited into shape.

To finish on a positive note, something Chandler got right? Python. Choosing to write their software in an expressive high level language was clearly a win. There is no question now that Python is fast enough for desktop software, and that's really the only doubt. Hopefully Chandler can be used as an example of choosing better languages.

If you choose to read this, or not, in the end people! Make a decision!

Atonement

Atonement
Ian McEwan

This is simply an excellent book. It doesn't go in for any special literary tricks, there's no special effort to make some obvious point: it's a really good story, told very well. There's some intimations of other layers, but feel free to ignore those. One thing that this book does pull off is an unsympathetic main character who I actually managed to not hate in the end. I didn't want to hurl the book across the room; always a worthwhile achievement.

A couple of points: I always find it vaguely amusing to see novelist characters in books written by professional novelists; even the best write what they know. The characterisation and the imagery are what really grabbed me. I was there on that hot, summer day in 1935. I knew Robbie Turner, and I knew Cecilia?Tallis.

There is much that could be said about the effect of fantasy and the blurry line between a clearly seen artificial vision and reality, especially in relation to the powerful imagery that provides this story.

But instead, I'll just say this is a great novel, read.

Computers Hate Me

It's true, they do. Possibly something of a disadvantage in my chosen career, but I get by, carefully. Don't believe me? Hear my tales of woe, come cry with poor, poor me.

April, 1997 - Still at University, just started my second year of a computer science degree. I save up the cash and buy the first computer of my very own: a Performa 6360. I get it home; I set it up, including copying all my work off the family computer: there must have been 80 megs of data! I'm about to go downstairs and delete everything off that shared computer, whe I stop. Nah, I'll do that tomorrow. I shut my brand new Mac down, I go to sleep. I wake up in the morning and one of the first things I do is turn my new Mac on. To be confronted with the dreaded flashing disk icon. My Mac couldn't find a disk to start from. Uh oh... Even booting from a system CD showed nothing. In the end this wasn't even a disk crash, there was a bug in the disk driver. It completely lost everything meaningful off the disk. Nice. Good thing I hadn't deleted my backup. Words to live by.

July, 1998 - For our third year project we decided to write a TCP peer-to-peer IM system for Apple's up coming new OS: Rhapsody. The beta didn't run on my 6360, so I sold that and bought a Power Mac G3, one of the original (or so I thought) beige ones. Turns out it wasn't quite 'original' enough: I scored a motherboard rev that wouldn't boot the developer seed of Rhapsody that I had access to. Argh! We still got our IM system working: we wrote it using the cross-platform environment Apple released for Windows NT. Everyone else in the class wrote Access databases.

March, 2004 - Time to finally upgrade the now ancient G3, so I order an iLamp G4. It arrives, complete with a nice line of purple pixels all the way down the screen. Fortunately, it was declared DOA and a complete replacement was sent.

Hmmm... all Macs so far. Why do I keep buying these?

July, 2005 - We've now started a startup. We know .NET, so that's what we're writing it in. I buy my first Windows PC - a Compaq Presario. It came with XP Home, so I buy an upgrade to XP Pro at the same time. At home that night, running the upgrade - and it just stops. No upgrade for me. And even better, it deleted the old XP Home installation, leaving me with an unbootable PC. Sound familiar? I got my computer back with a clean installation of XP Pro. Except that didn't include any hardware drivers at all. Instead of just using VGA 640x480 resolution on my 19in LCD monitor, I spent the evening finding, downloading and installing all the right drivers. That is also my only experience of trying to convince someone I had willingly bought software from that I was not a criminal. Thanks, Microsoft Software Activation. This computer lasted barely two years before a very fatal disk crashed, ended that incarnation.

So, it's not the computers, it really seems to be me. Those are the only computers that I've bought. Seriously, no other computers were hidden away in there. I've also had zip drives inexplicably and suddenly give the click of death, lamps leave scorch marks on my desk, monitors catch fire (really! there was smoke), printers refuse to power on and USB devices make my machine reboot right now. Maybe I just have a special relationship with hardware? I've definitely got a reputation for it... But, I'm a software guy, and there are uncountable software disasters tucked away in there.

Now, it's that time again: I need to replace my four year old iMac. I'm planning on getting a laptop, hopefully a MacBook Pro. Doesn't sound dangerous to me, what could possibly go wrong?

The Player of Games

The Player of Games Iain M. Banks

That 'M' is important and very distinctive. This is a completely different author to Espedair Street; even though both books list all Iain Banks and Iain M. Banks books. No? Don't believe me? Well, yeah. His fiction is published as Iain Banks and his sci-fi as Iain M. Banks. Strange, but that's the way he does it.

His sci-fi is some of the best I've read since Philip K. Dick. And as he doesn't produce anywhere near as much as Dick, it averages a lot better. Though without some of the crazed inventiveness. But that sounds like damning Banks with faint praise: his sci-fi really is that good. There are fantastic ideas and a very plausible feel to everything. He doesn't shoot himself in the foot by trying to explain how everything works: the technology is just there and it works.

But his strongest points are actually his characterisations and story. You get involved, you believe, and most importantly, you care. And on top of that, the story is usually about the growth and life of a character - sometimes a descending spiral with no apparent way out; sometimes a broadening and opening of a character you initially dislike.

This book is fascinating for the first real peek inside the Culture, instead of the view of a mercenary looking from the outside, in.

Orlando

Orlando
Virginia Woolf

Another literary fantasy novel. AFter the disappointments of Jordan, Martin and, most of all, Feist, I'm happy to be looking to Marquez, Updike and now Woolf for my fantasy fix. As I've said before, any story is by definition a fantasy, so why restrict your scope to only the events that can take place in this prosaic world we are trapped in? Sure, there's a place for the great everyday; but fantasy can be so much fun!

And given how dry Woolf is, it's surprising to see how fun Orlando can be. There are two key elements of fantasy here: Orlando (the character) lives for a very long time, and there's a second question of gender... The age question is handled interestingly. There's never a discussion of this, Orlando just keeps on living, aging at a different rate to everyone else around.

This disconnect from reality creates a dreamy, flowing world: the story reads like a lyric poem: drifting from image to image guided by your narrator, Orlando. And then towards the end it starts to coalesce on but two points. But slowly, like a willow emerging from the mist. Left wondering if those were always there, you float past.