Startups as the Future of Technology

It's very fashionable in geek circles to attack Paul Graham at the moment, particularly after his essay You Weren't Meant to Have a Boss. I've been reading his essays for a few years now and I've wavered between agreement and an undefinable sense of unease. Now I believe I can finally pin this down.

The central point of Graham's Boss essay seems to be that over a certain size organisations become rigidly hierarchical and once the hierarchy sets in the creativity of programmers is significantly and fatally constrained: over a certain size an organisation will be incapable of producing original software. This is a continuation of a theme through much of Graham's writing. Startups do interesting work and software development will migrate exclusively to startups.

Well, I think that's something I can disagree with. Of course, it's pretty obvious that I have a vested interest. An iconoclast like Paul Graham will always get the most vicious response from those he seeks to help. Allow me to give my personal background, in the interests of disclosure.

My entire career has been in software development. In my first job out of Uni I fell into the 'hero' programmer archetype. In every job after that I've been in some form of senior position: tech lead, architect, team lead. I've also been an independent consultant, co-founded my own startup (we failed; be very careful about selecting your co-founders) and now I'm working for a startup. Yep, as an employee with a boss and all.

I spent sometime this morning going through all the startups listed on the Y Combinator website. For each startup I've tried to classify them according to the current big themes in web sites.

Social Networking: Reddit, Loopt, Flagr, LikeBetter, JamGlue, Scribd, I'm In Like With You, SocialMoth, Anywhere.FM, Disqus, Reble, AddHer, Inkling, Draftmix

Advertising & Sales: ClickFacts, Adpinion, Bountii, Octopart, Auctomatic, TextPayMe, TipJoy

Apps on the Web: Snipshot, Wufoo, YouOS, Thinkature, Weebly, Buxfer, Heysan, Versionate, Fuzzwich, RescueTime, 8AWeek

Other: Virtualmin, Justin.TV, Xobni, Webmynd, Heroku,

Dead: Shoutfit

The 'other' category is the interesting one: into that bucket falls a server admin dashboard, a web-based TV channel and a plugin for searching Outlook email. But, there are a lot less of those than the social networking and web-based desktop application startups.

I'm sure many, particularly the founders, will disagree with my classifications. But, these are mainly right, especially if you read 'Social Networking' as 'Social Networking around Common Interest X.' And in the end you'll find the exact classification is not important.

All of these startups share a few things in common. They were all launched quickly and they're all pure software development, often running on someone else's eco-system. There is a place for development like this, but if this is the future for computer science I believe the field will be significantly poorer for it.

I am working for a company that by pretty much every definition is a startup. By the time we left stealth mode in March last year the company had been around for 13 years and had grown from a core group of computer scientists to a company of over 300, including chemists and physicists. Oh, and we invented a new type of printer. A startup like ours simply doesn't fit into the Y Combinator model. We also don't fit into the small company with no bosses model: building hardware takes time and a lot of people, you simply can't avoid either.

This is my concern. Is all future computer science productisation and development really going to be latest and cool ad-funded mobile social networking site for parrots? Because, excuse me if I'm not excited by that future. I am still excited by the potential for computing and the Internet in particular, but that potential is better served by longer-term thinking and grander plans than refinements of what everyone else is doing.

This criticism may actually run deeper. A continuing trend in computing is to make programming easier for all. This has had the effect of pushing some tasks out of the realm of the programmer and back to expert users. This has been a good thing. Users have more control and programmers are free to work on interesting problems. The web has also been fantastic at improving human to human communication. Recently, the potential of the web as a mechanism for computer to computer communication is becoming more apparent. Many of the Y Combinator startups very effectively exploit this: improved experiences and convenience by combining the information on eBay with the blogosphere. It also appears that these startups are surfing a wave. The gap between technology becoming easy to use and becoming easy to program. In other words, I suspect this style of startup is not long for this industry. An historical aberration, automatic arbitrage for company startup, as it were.

This is not the end of startups of course. There will always be startups, however there will have to be some interesting, risky and difficult technology behind the curtain. Originality will once more be prized. In this world, You Weren't Meant to Have a Boss will make a lot less sense.

Personally, I fully expect to do the startup ride at least once more. And I'm looking forward to doing that in a world that once more demands innovation rather than just another social network. After all, I really do want to add something to the world and I just don't see that happening with late noughties startups.

Oh, and if you also want to work on world changing, original technology, my company is hiring. Love web technologies, think you have what it takes to work for Google, but aren't excited about working for a company of 10,000? Want to work in Sydney, Australia? Beautiful beaches, summer all year long... Send me an email, giles dot alexander at Google's-free-webmail dot com.

Espedair Street

Espedair Street
Iain Banks

A very good writer, his sci-fi (under the name Iain M. Banks) is consistently original, but his non-genre fiction is also very good. Dead Air is worth reading for the head-butting alone and The Wasp Factory is bizarre, unexpected and simply amazing.

The strength in his fiction is the characterisation. Danny Weir, in Espedair Street, is a great example. A washed-up 70's rock star who has managed to annoy and drive off all his friends. He's now brooding self-pityingly in a stony mansion in Glasgow. But, you're introduced to him, you hang out with him, you drink with him and you get to know him, know him well. Though he spends the book going over everything that's gone wrong in his life and though most of that is down to his amazingly ability to always make the wrong choice and though it may be hard to listen to an hyper-rich rock star complain about his past it doesn't matter because you know him and, ultimately, like him. Enough to hope he finds some way out.

Oh, and the book manages to frequently be damn funny, as well.

The Timeless Way of Building

The Timeless Way of Building
Christopher Alexander

For the past year or so, this was my bus book. That's a surprisingly long time, and it probably shouldn't have taken me that long to read. Late last year, about 50 pages from the end, I paused in my reading; and then took several months to pick it up again. This seems unfair to the book: it deserved a much more coherent read than that. Though, the ideas are different enough to also benefit from a considered read. I'll pick this up again sometime, and I promise to read much faster that time. Anyway.

One sentence summary: this book will forever change the way you look at and think about buildings, towns and architecture.

Alexander firmly believes that modern planning and building practices are bankrupt and can only result in inhospitable, unwelcoming cities and homes. A belief that seems to be firmly born out by most urban planning since the Second World War: just look at the damage Harry Seidler has wrought on Sydney for an example close to home. This book is a polemic, a grand rant against the current state of his own industry and art. A work in the tradition of many a genius' (and quite a few looney's) Let's Blow Up the Universe screed. So, genius or looney? I've probably already given away my opinion on that matter...

Ultimately, it would not do this book any justice to attempt to briefly summarise what it has to say.

But what the hell, I'll give it a shot anyway. Alexander's central thesis is that there is a shared quality amongst those towns and buildings where people feel most at home; a quality independent of culture, climate and history. He also believes that this quality can be easily achieved, by any person who chooses to build. It is a matter of recognising the forces within the people who will use the building or site and then balancing those forces with the forces intrinsic to the specific location and society. He even outlines a prescription for achieving this balance: a collection of patterns to duplicate in design, planning and construction, with instructions on how to combine these. A language of patterns to construct our built environment.

Unlike many other polemics, this is highly detailed and descriptive: it describes the quality to achieve and then gives instructions on how to achieve it.

If you live in a large city in Australia, it'll be pretty obvious while reading this book that this is not how building is done. First, Australian building practices place the car as king of all. Any building or neighbourhood must be designed for the maximum convenience of the car: people are a distant second. Second, Australian building practices harken back to some long forgotten European past: everyone wants a little brick English cottage, though nothing could be more generally inappropriate for our climate. The Queenslander is not the standard archetype for Australian residential building unfortunately.

The current popular obsession with being 'green' is driving people to a certain superficial realisation about the car. But that is only a symptom of a far deeper problem. Loudly proclaiming that cars are evil and must be disposed of is never really going to achieve anything. And that sort of unbalanced (in the forces sense) thinking will inevitably lead to other problems. As much as I'm a fan of the specific remedies proposed in Jan Gehl's research paper into Sydney's CBD, I do feel some uneasiness.

The pattern approach that Alexander talks about is intended to completely avoid unbalanced forces. He regularly uses cars in his discussion of patterns. They are real, they are valuable and they're not going to just disappear. A central point of these patterns is that they're not something Alexander has devised as a new architectural '-ism' to imprint his vision on the world. These patterns are things that arise naturally, given the way all humans want to live. Growing organically out of a combination of the people and their surrounds. There is a sequel to The Timeless Way of Building, A Pattern Language, that acts as a catalogue of the most important patterns that Alexander and his colleagues have observed in successful towns and buildings.

In the US there is a growing style of design called 'New Urbanism' that attempts to encourage the buildings and towns that Alexander commends so highly. It is interesting to note that in Europe that name is largely unused, people preferring to use 'The Way Towns are Designed' instead. It is also interesting to note that in Australia, we have neither.

Finally, why did I, a software engineer, read this book? To the surprise of many architects, Christopher Alexander is very well known in the field of computer science. In the 1960's his work was discovered and his concept of patterns was co-opted. No serious software engineer can possibly not be familiar with the world of design patterns: named rules for particular structures of code to solve certain problems. It's my opinion that while initially off to a good start the modern Design Patterns movement has completely missed the point of Alexander's original teaching.

His intent was not to catalogue an exhaustive set of patterns that may be thrown at a problem until a solution emerges. His intent was to define an interlocking language from which you can select appropriate terms to grow a solution. In his case a building or town, in my case a software system. Modern design patterns seems to ignore the essential organic growth aspect of a pattern language, and instead seems to focus on cataloguing. An unbalanced approach.

What is this Property?

I'm not a mathematician, just a computer scientist with an interest in maths, so please excuse the simplifications and inaccurarcies in this. I'm going to describe this with some rigour, but I'm bound to get things slightly wrong, please bear with me.

In maths, a function is defined as a relation between the members of two sets, S and R, that produces members of the third set T. Looking at it another way, the set T is defined by the function. Some functions, taking two arguments from the same set S, always produce members of that set S. Addition across the natural numbers is an example of that: for any two numbers greater than 0, the sum will always be a number greater than 0. There are many functions that behave like this.

Functions have properties. A property describes a rule that a function obeys for given sets of parameters. From a mathematics perspective, these properties are interesting. For example, addition across the natural numbers is associative. This means that no matter what order the parameters to the addition function are arranged, the answer will be the same: 2 + 3 = 3 + 2. Fairly simple and obvious, right? But from the same property we can also say 2 + (5 + (6 + 11)) = (2 + (5 + 6) + 11). This is interesting because once we know that a function has the associative property we can arrange the parameters of the function without changing the meaning: this is useful in proofs.

There are many, many of these properties, and most of the interesting ones have names: associative, commutative, distributive. For the last year I've been trying to find out if another property I've noticed also has a name.

Take the function minimum across the natural numbers. Given the sets {4, 6, 100, 1, 43} and {1} minimum gives the same answer: 1. The result of the function minimum is determined by only a single member of the set, no matter how large the set.

Take the function and across the booleans. Given the set {true, true, true, false, true} the answer is false. It doesn't matter how many true's are in the set, the answer will always be false.

And I'm sure you can imagine other functions that behave like this. My question is: does this property have a name, and if it does, what?

If I was more of a mathematician, I'm sure I could actually describe this property a lot more accurately. In fact, I'm not entirely sure there is a consistent property here, and I have no idea if it's interesting if it does exist. But I notice this often, and it sure feels like it should have a name.

Functions whose result is determined by a single member of the parameter set, irrespective of the size of that set: do these have a common property?

Midsummer Night's Dream

We went to see Midsummer Night's Dream at the Sydney Theatre tonight. A friend bought the tickets, we were just told it was Midsummer Night's Dream. I really should have found something more out about the performance.

I like fairly challenging books: I believe that the reader should occasionally be made to work for it. I love Pynchon and I enjoy Woolf. I am a huge fan of Shakespeare and I've enjoyed pretty much every production I've seen, even when I didn't know the play, both traditional and modern interpretations.

We left this play at intermission, along with a pretty significant proportion of the audience. I have never done that before. I won't even walk out of a bad movie.

This was plain awful. Absolutely, completely unwatchable. Why? It's about 60% performed in Hindi, with no sub- or sur-titles. If you don't speak fluent Hindi you won't be able to understand what the characters are saying most of the time. I know that's obvious when I say that it's performed in Hindi, but the Sydney Theatre really didn't make this obvious enough. I was also handicapped here as I didn't know the play. I've seen parts of it before, and remember some scenes but I don't know the overall plot and characters. I certainly couldn't imagine what was happening when I couldn't understand the dialogue.

The opening scene to establish the plot was entirely in Hindi, and from then on I had absolutely no idea what was going on. Mana tried to help out by whispering brief explanations as she has previously studied and performed this play. But she couldn't keep this up, and by this stage it was pretty much too late: I already had no idea who any of the characters were.

What do I know of Midsummer Night's Dream? Well, there's one of my favourite Shakespearian lines:

If we spirits have offended, Think but this and all is mended: You have but slumbered here While these visions have appeared. - Puck

That's from memory, so excuse any mistakes. I also remember the sarcasm, wit and lyricism of Puck. And I missed all that in this performance. Surprisingly enough, the play would have been better if it was entirely performed in Hindi: when they were speaking English I could follow what was happening and start to get involved. Then they would switch back to Hindi, kicking me out of any involvement, and leaving me bored and disconnected in my seat. But, then I would try to get involved again in the dance and acting, only to be booted again when they switched back to English.

Shakespeare is entertainment, especially his comedies. These were great works meant to illuminate the human condition, while also highly engaging and entertaining. Anyone should be able to watch a production and enjoy it. The only people who could enjoy this production were those who spoke fluent Hindi, and those who already knew the play intimately. And while I fully support the production of entertainment for specific languages, this should not be promoted to a larger audience as something for everyone. Because is it's not: this is an exclusive production only meant to be enjoyed by those who have already studied the play.

And I don't like this artificial, constructed exclusivity in the arts.

The Three Stigmata of Palmer Eldritch

The Three Stigmata of Palmer Eldritch
Philip K. Dick

Did you end up finding it, Philip? What it means to be human? Religion didn't seem to provide your answer. Did drugs? A Scanner Darkly is famous for your search, but this appears to be some sort of transition between those two searches.

Like Graham Greene, Dick is one of my favourite authors. Over time I'm steadily trying to read all of his novels. I prefer his later ones, so that's generally what I choose. Unlike Greene, not each of Dick's is better than the last: A Scanner Darkly is still my favourite, and one of my favourite sci-fi novels. Sci-fi is typically a pretty pulpy genre: cheap enjoyment, with very little challenge to the reader. Even the best sci-fi with a great idea at it's heart will present that idea in a pretty straightforward form.

Not Philip K. Dick. He did not shy from challenging the reader with unusual ideas, often in outright confusing forms. This book felt like some sort of mental trap that the reader is drawn into. Only with the hope that all will become clear by the end. The confusion is why I read this book now. How hard can you push the reader? How difficult can you make the story to follow? How many tricks can you pull? And still end up with a populist, enjoyable story.

There's a lot to connect Dick and Pynchon. But, Dick just didn't have Pynchon's talent. Sometimes you are left wondering if this was meant to be confusing, or did he just write it a bit too quick. His later work does show that yes, he was aiming to confuse.

The Ultimate Development Environment

An enormous claim, I know. But this is not about processes, tools or working conditions. This is about something quite different.

Shrew is progressing, it now sports an s-expr to XML evaluator; I'm reading RESTful Web Services to gain a better idea of how it should expose resources. And I'm also working through The Seasoned Schemer. And therein lies the most interesting aspect to Shrew. I am a reasonably experienced, quite competent polyglot software engineer, but learning Scheme has forever changed how I think about programming. And through example crystallised the ultimate development environment that I have been drifting towards.

When I'm working on Shrew, my editor looks something like this:

In the top-left is the module I am currently working on: writing, expanding or fixing - as I'll try to show there isn't really any difference between those three. In the top-right is a scratch file that contains a bunch of ad-hoc tests for the module I'm working on: nothing structured, just calls of the functions that I'm writing. Across the bottom is the output from a Scheme process running in the background.

Before I go on there's one detail of Scheme I should explain. To write a new function you call a built-in function called define, passing the name of the new function and its body. define is smart enough to simply replace the body if a function of that name already exists.

It sounds like a pretty simple development environment: a plain text editor with three windows on screen. Why so special?

I write a function - not a test, sample or prototype, but the real code I'm planning to commit - I jump to the end of the define and run a command in my editor to evaluate it. That function is then inserted into the running Scheme process and available to be used by anything else that is run in that Scheme process. Or, I'm immediately informed of a syntax error in my code.

I switch over to the scratch file and write some code to call the function I just implemented: typically just one expression, but it can be as many as I need. I evaluate that new code. And immediately see the output in the window at the bottom; the window reflecting the running state of the background Scheme process.

And of course, there's a bug in my function. I switch back to the window containing the module, fix the bug and re-evaluate. I switch back to the test code, re-evaluate that, and see that my change has fixed the bug.

Elapsed time from writing the function through debugging and verifying the fix: 45 seconds.

Instead of having to write a complete library, compile it, write a test harness, compile that and link it to the library and only then run the code to see if it works, I have a Scheme process running in the background that I can just keep adding code to. New code, or code to replace existing code. And at any point I can execute any sub-part of that process and immediately see the output. No delays, no pauses, no backtracking to find which line of code is wrong.

Scheme could be regarded as a fairly direct implementation of a theoretical model of computation: the lambda calculus. Most texts that teach Scheme emphasise this; they encourage you to think of your programs in terms of this theory, in quite some detail. That may sound fairly esoteric, but once you've spent sometime working in this environment you reach this unique state. You are inside your program, reaching around moving code as fast as you can think. There is essentially nothing between your thought and code: no defining boilerplate, no compiling, no creating test harnesses, no waiting for test runs to complete. Your solution simply unfolds before you.

But that's not to say your code is of lower quality. In fact, because you're concentrating more fully, with no distractions and the flexibility to easily push your code in any way you want, the code is of much higher quality. There's no idle thought 'I should test that' which is forgotten in the edit-compile-run-debug cycle: think it, try it. This is flow as Peopleware could only dream.

And once you break out of this magical flow, you're left with complete code; code you lived and breathed for a few hours, code you understand deeply and will have a hard time forgetting. Plus, a comprehensive set of tests to commit alongside.