Wednesday 26 November 2008

Dealing with Bot Nets

Currently at work I'm designing a large-scale system that will be susceptible to a certain kind of denial-of-service attack. By way of analogy, imagine that Gmail didn't bother to prevent robots from creating accounts. By the time the first human went to create an account all the reasonable combinations of the top 10,000 human names would have had already been taken, by robots. This would be very irritating to all actual human users.

Our problem is much more serious than simply losing human-preferred free email addresses. But, it is a case of preventing robots from soaking up a finite resource and depriving real humans of using resource.

My approach to large system design is to always get security right first: you can never effectively retrofit it later. And the central question we keep coming back to on security is how to defend ourselves against robots. Our thinking has typically followed certain lines:

  1. To acquire a resource, a user must prove they are human.

  2. All users must have a registered account, so we can identify who is consuming the resource and only have to verify their humanity once: on registration.

  3. The user's account must be protected with a password to avoid a bot misusing a real human's account.

  4. Each account has a threshold of resource acquisition. If the threshold is exceeded than that account is temporarily blocked in some way.

At this point in our thinking we're pretty confident that we've dealt with the risk of a robot creating an account and using that single account to soak up all our resources. We're also pretty certain we've dealt with the issue of a robot creating many, many accounts, using those accounts to soak up resources while staying under the threshold for each.

But. What about bot nets? And by restricting single accounts like this, haven't we just forced attackers to use a bot net? Attackers would want to distribute a bot across the Internet. Each bot would not use its own account, instead it would use the account of the human owning the computer the bot had infected. Once the bot is on the human's computer it can easily grab the credentials, as a key logger or by sniffing around in the browser cookies. In this situation our threshold control hasn't really stopped the attacker, but it has hurt the human. The effective threshold for the human is now much lower.

And it is on this point that our discussions tend to go around and around. How can we prevent bots (who may have acquired a human's account) without negatively affecting the human's experience and without placing prohibitive barriers to use in place?

Thinking about this issue tonight, I wonder if we're not completely wrong in this argument? If a user's computer has been compromised and is now part of a bot net, should we be trying to give that user a smooth experience at all? They've been compromised, shouldn't we identify that, inform the user and then attempt to lock them out completely? There's a question there about when we can let them back in, but I'll leave that now.

My central question is, should web applications actually aggressively make the experience worse for user's who have been compromised? In the case of a bank the answer seems obvious. I suspect we're actually similar.

Wrath of a Mad God

Wrath of a Mad God
Raymond E. Feist

Pure crack for fantasy geeks and about as high quality. I've been reading Feist since a friend recommended Magician to me when I was nine years old; in grade four, back in 1988. My friend's name was Paul Reid and that was 20 years ago now. It's also long since I realised that I'm pretty much only reading Feist because reading Feist is what I do.

As his books get steadily worse that becomes a weaker and weaker reason. He does have some redeeming features: he doesn't forget where he put the plot; his sagas actually finish; he manages to avoid appearing a total right-wing fascist. After the disappointment of Martin and the betrayal of Jordan those are very good things to a recovering fantasy geek. He is still one of the reasons that I haven't completely given up on fantasy. And of course, Gaiman.

Why am I now so disappointed? His first three books (Magician, Silverthorn and A Darkness at Sethanon) were really great fantasy epics. Magician even managed that rarest of fantasy firsts: a self-contained, single, enjoyable novel. What was so enjoyable? A rich, consistent, well-thought through world, with a deep and fascinating history. The sort of thing that makes Tolkein so popular. Those books sold well, Feist proceeded to mine that world and his characters in countless sequels. And like the fools we are, us fantasy fans lapped those sequels up.

You may think you want the blank spots in the story filled in, you may think that those tantalising glimpses are only a fraction of the glory that is fully formed, but hidden, in the author's mind. But. You are wrong. The back story you build, the worlds you imagine around the glimpses? Those are the real joy in fantasy. Do not burn those worlds to the ground by demanding ad reading endless prequels and sequels. Let the great stories stand alone.

Feist is a great example of this. It turns out that he didn't really have anything to surround those brief histories and as he writes more and more he's starting to change things. Sometimes for the better, but many times the things I've loved have died.

I see two things here: the world is not meant to change, even if it does make things easier for someone; and, you don't want to know your heroes too well. Even if they are only characters in a book.

Sunday 23 November 2008

The Worst Desktop Operating System. Evar.

I complain a lot about FreeBSD here and on Twitter and, thankfully, I am now about to stop using that horror on my desktop. But why horror?

  • In the world of desktop computers, anything that is not Windows, is niche.

  • In that niche, anything that is not Mac OS X is niche.

  • In that niche, anything that is not Ubuntu Linux is niche.

  • In that niche, anything that is not Red Hat or SUSE Linux is niche.

  • In that niche, anything that is not one of the commercial workstation UNIX operating systems, like Solaris, or AIX, or HP/UX is niche.

  • And down there, in that niche, in that fraction of a fraction of a fraction of a fraction of a percent of the world of desktop computers, FreeBSD is niche.

From a technical point of view it actually has quite a lot to recommend it. The kernel is very well tested and reliable. For a UNIX, it has generally made decisions for correctness over performance. Something Linux certainly can't match. The user land is a consistent space, harking back through over 20 years of tradition. The ports system is a pretty good way to install and manage software.

But. In the whole world there are perhaps 15 people using it (no, not really). Anytime you Google for any problems or issues, you'll find Linux, and just have to hope that you can figure out to translate the instructions.

And this is to say nothing of the complete dearth of available software. To use FreeBSD is to always be several versions behind in Firefox. To have to compile Emacs from CVS source. To have to tweak the source code to your video driver.

FreeBSD may once have had the One True Filesystem layout, but not anymore. Linux is now nearly the king of that hill. Don't use FreeBSD as your desktop. You really don't care about how good the kernel is. You really do care about not having to compile video drivers. Worst Desktop Operating System Evar.

Thursday 20 November 2008

Still Alive

Yes, this is one of those irritating posts. Where a blog that you thought had quietly retired suddenly reappears with a post. A post that says basically nothing. A very self-indulgent post just promising that there will actually be real work worth reading reappearing soon.

Why couldn't the blogger just leave us all in peace? Why this attempt to appear that he hasn't just gotten bored or too lazy to update? Why this empty post tantalising and teasing with a promise; only to disappoint with more deathly silence.

Yep, this is one of those posts.

But! I actually do promise to post something real soon. No! Really!

And, in a desperate attempt to appear trustworthy, here's a short overview of what's been going on.

  • Switched from the horror of FreeBSD I now have a brand new MacBook Pro as my primary computer. After nine years I'm finally being paid to use the platform I stayed loyal to throughout the dark years. Hopefully the new computer, well set up, will actually help me write more here. It got me writing this.

  • New project. Can't talk about it. Cool though. Has inspired some general problem solving that I can talk about though. There will be some technical recipes on here for the first time.

  • Briefly had a fish tank on my desk at work. It was very nice. The tank did well, but then I had to move desks. Probably worth doing, but you'd want to be more sure of where you were sitting.

  • Joined a book club. Read quite a few books. And yep, that means reviews. There will be some of those coming soon.

  • Still annoyed at various parts of my industry, enough to rant.

Hopefully, all that and more to be posted.