Dreaming in Code
Scott Rosenberg
Rosenberg is one of the co-founders of the online
magazine Salon.com, a magazine I've been reading on and off
since 2000. After having a bad experience with internal software
development, he became interested in how, after more than 50 years
experience, we still find, in the words of Donald Knuth, that
'software is hard.'
In the interests of disclosure, I will point out that I got a copy of
this book for free from the author after he offered copies to
bloggers if they agreed to mention the book. So, this is me
(gratefully, happily) fulfilling my end of the deal.
This book is a story, a story of a collection of great developers
attempting to write an amazing piece of software. It is not a
retrospective story either. It begins shortly after the project has
started, but unfortunately couldn't follow the project to completion:
the project still isn't done yet.
The Open Source Applications Foundation (OSAF) was formed by
Mitch Kapor (of Lotus 1-2-3 fame) to build a new form of
personal information management software: Chandler. Dreaming in
Code tells the story of how they tried and (so far) failed. With many
informative diversions into the theory of software engineering in an
effort to discover why software seems to so consistently be delivered
late (if at all) and buggy (if it even gets close to meeting the
original vision.)
Rosenberg does an impressinve amount of research into the theory. No
appropriately read professional software engineer will find any
revelations here: this is all stuff you should already know. But
Rosenberg is not claiming to invent or discover anything new. In fact,
he goes out of his way to disclaim that there is any original research
contained in this book. He is a journalist, and as a journalist he has
produced a detailed literature survey. The summary in two well-written
chapters is useful even for experienced software engineers. I'm sure
non-software engineers will find this all very interesting; assuming
they are interested in how software gets written.
For software engineers, there is something very interesting here. The
internal mechanics of a a team building a piece of software is a very
secret thing. Companies are secretive and companies and open source
projects want to protect their reputations. Most software engineers
only ever see how a project they are working on proceeds, and then
they're too close, plus there's no nice summary of what
happened. Dreaming in Code is something very valuable to our field:
an accurate story of the human side of a software development
project. With both the clarity of distance and the accuracy of events
recorded at the time they happened.
It was simply astounding how familiar this story was. OSAF and
Chandler get some things spectacularly wrong, but then in other cases
they do things very right. It's easy to point at the things they
stuffed up and claim that you would never make those mistakes, but
it's a little too easy to ignore the things that were done right. The
good decisions end up forgotten and never noticed.
So what did I think that got wrong? Firstly, and by far the biggest:
Analysis Paralysis. This is a mistake that I've seen projects
make over and over again. In fact, I'd go as far as to say that it's
more common to see this affliction than not. Projects just can't seem
to make a decision, stick to it and then start building. The fear of
the future locks them solid: what if we make the wrong decision? What
if someone blames me for the wrong decision? In the end, ikf the
decision wsas so egregiously wrong that it can be traced back to just
one person, then everyone else around at the same time is just as
culpable for allowing that decision to happen. Everyone, please just
get in the habit of making decisions. Rely on those around you to spot
a bad direction: that's what they're there for.
Second big mistake for Chandler? People. No surprises there, it's the
big and obvious disaster. If you believe that software is hard and you
care about your software then clearly you should only work with the
best. That's easy to say, but pretty hard to achieve. And in
Chandler's case the people problem exhibited in a couple of
interesting ways. Before I talk about the people problems that I saw
Chandler as having I need to say that any attempt to judge is based
solely on what is described in this book. I (obviously) didn't work on
this project, I don't know the people, there's only so much I can
say. Having said that, I'd also like to say that people problems kill
most projects and if we hope to advance we need to get over this fear
of talking about the problems with people. Maybe then we can find some
solutions. Or maybe that's just an overly analytical geek talking. Why
can't everyone just be a nice reducable puzzle, dammit?
There was a permanent employee of the OSAF who was hired quite early
and ended up in quite a senior technical position, and this employee
was unfortunately the very soul of Analysis Paralysis. Any story about
some interminable technical discussion has this particular employee at
its heart. He was extraordinarily conservative and wary of making
decisions, but ultimately many of the technical decisions came down to
him. After a few of these stories, I was left shouting 'Do something
about him!' at the book. And I've seen precisely this problem
face-to-face too many times to ignore. Once again people, make
decisions! You probably won't get it too far wrong. In fact, in this
case, the inability to make a decision led to one of their few
definite cases of over-engineering.
The second people problem: OSAF planned to run themselves as an open
source project that encouraged volunteers. Predictably enough, the
only people who could volunteer for any extended period tended to be
ex-Apple and -Netscape employees who had already made their fortune
and no longer needed to work for an income. From the outside these
volunteers presented an interesting problem. They were all brilliant
and had done amazing work in the past, but they didn't see this
project as any kind of meal ticket. There was no drive for them to get
this project finished and out the door. In the end, they come across
as people partially on the side-lines, commenting endlessly but never
really pushing the project forward. Instead, they were offering
endless advice on how things could be done better. Software projects
are always cursed with people like this; encouraging volunteers just
seems to guarantee it.
But back to the book, I think I've already made clear that I really
enjoyed it. I also think the opportunity to see inside another project
is infinitely valuable for software engineers and software
engineering. Aside from all that, I will say that sometimes the brief
newspaper style of the book was a little irritating. On occasion I
felt a topic or anecdote could have done with some more depth before
moving on.
Software engineers will get a lot out of this; non-software engineers
who care about how software (upon which, our civilisation is built) is
written will get even more out of this. Oh, and as a professional
programmer and wannabe-amateur writer-slash-blogger, software is
definitely written - then organically edited into shape.
To finish on a positive note, something Chandler got right?
Python. Choosing to write their software in an expressive high level
language was clearly a win. There is no question now that Python is
fast enough for desktop software, and that's really the only
doubt. Hopefully Chandler can be used as an example of choosing better
languages.
If you choose to read this, or not, in the end people! Make a decision!