Bad Software Is Good For You

In which I make a number of unsupportable assertions about the design of community software and attempt to bolster my position with seemingly unrelated advice from four wise friends.

Boris Anthony, web community designer extraordinaire, left the friendly confines of Montreal to come visit with the Global Voices team at Harvard this past Tuesday. As we detailed the remaining bugs on the Global Voices blog and the new features we’d like to add, Boris and I found ourselves skirting the edges of the debate we’ve had since day one of our collaboration on the site: WordPress versus Drupal.

(A quick translation for those lucky people who don’t spend their free time obsessing about content management systems. WordPress is an open source weblogging platform. It’s the platform I use to manage this blog and the platform – with some modifications – that Global Voices runs on. It has a reputation for being very user friendly, but for having some underlying architectural problems that make it hard to scale. Drupal is an open source multi-purpose content management system designed for the support of complex websites with multiple authors. It has a reputation for being ludicriously flexible, ungodly powerful and far too complex for mere mortals to use.)

The details of this specific debate are so boring that they’d drive six of my seven readers to Slashdot to follow flamewars instead of listening to it. (The seventh reader, who knows more about the subject than I ever will, would write a polite email convincing me that I’m an idiot while making me feel good about having her as a reader.) But this debate is emblematic of a larger debate that I think is (possibly) interesting to nongeeks.

I found myself making the argument to Boris that WordPress was a better tool for Global Voices right now precisely because it wasn’t as good a tool as Drupal. As I thought about it a bit more, the success Global Voices has had so far has had a great deal to do with using mediocre tools rather than working to build better ones. And I think there’s an argument to be made for stretching tools to see what they can be made to do rather than building new tools from scratch.

Thoughts from a pair of smart friends as I try to make this point:

My friend Doone Mackay studied “studio art” (i.e., the one where you make paintings instead of talking about them) at Williams College a few years after I graduated. When she began her career, the art students worked in random corners of campus that had been reclaimed from other departments. (The sculpture studio in which I spent some of my happiest college hours was in a disused garage next to the gravel pile and fuel oil storage tanks.)

Halfway through her career, the College built a beautiful new studio art building. Many alums, myself included, saw this as a sign that Williams was taking studio art seriously and giving the department the respect it deserved. Doone felt differently, and wrote a long editorial about what the art department lost by moving from an old space. One of the primary loses, she posited, was the freedom to use the space in outlandish, sometimes destructive, ways. (I’m recalling driving nails into the wooden floor of an old studio space to steam-bend planks for a boat a friend was building as part of a studio art building…) With new, pretty, perfect buildings, would students be afraid of abusing the new spaces, and how would this effect the art they made in those spaces?

I’m convinced that Doone is right. While the sculture space I worked in was far from pretty, I was never afraid of bending it to my needs. The best installation I built in college was installed in a disused, dusty, cobwebby corner of the basement of the space, and most of the character of the piece came from the environment. I suspect there’s a shortage of dark, spooky corners in the beautiful new space students work in today.

Second thought: my friend Daniel Beck and I were talking about software design the other day as we moved furniture. Daniel (whose most recent coding exercise may well bring about the Rapture as soon as anyone dares to run it) has spent the last five years developing e-learning software for a variety of funct and defunct companies, and was preparing to write his fourth e-learning community system from the ground up. In response to my unanswerable question – “Why, despite the fact that smart programmers have been writing bulletin board/discussion group/community software systems for the last two decades is there no software in this genre that’s actually any good?” – Daniel offered the following observations:

– Every community needs – or at least, thinks it needs – its own, custom-made community discussion system.

– One can either design a tool sufficiently flexible that it can be customized to those users needs or handroll a tool from scratch.

– Once one gets into the business of handrolling, it’s a great advantage to handroll similar types of communities – elearning environments, for instance – because they’re all going to have some common traits that you’ve coded before.

While I think Daniel’s right on the first point (and I’m sure he’s right about the third point), I think there’s an alternative solution that most people choose other than building their own tools (or hiring someone to build them) or customizing a super-flexible tool like Drupal: they use a hodgepodge of tools that don’t work very well, and either figure out how to use the tools in new ways, or shape their behavior to match the behavior the tools expect and reward. Most communities fail… but I would argue that most fail for reasons other than inadequate tools. Those that succeed sometimes build their own tools (like Slashdot’s karma mechanism), but others succeed by using existing tools (the hundreds of blogs that have complex discussions in a simple, linear comments mechanism.)

When Rebecca and I started to put together Global Voices early this year, my instinct was to build a new tool that handled blog aggregation in a new way. It would be open to public input – like a wiki; it would be combine multiple input sources into a single website, like an aggregator; it would enable a strong editorial voice, like a blog or content management system; it would support tagging and folksonomy, like del.icio.us. So I started reading APIs for existing tools and source code for del.irio.us and making creat plans to duct tape together a few snippets of original code with huge cunks of existing code into a brand-spanking new original system. And then I got lazy, and busy, and distracted and put the project on hold.

Which is probably a good thing. (Advice from a third wise friend ahead.) When Rachel and I found the house we currently live in, she wanted to buy it and I didn’t. There were lots of things wrong with it and I thought we’d be better off buying land, designing and building our own house. I ran this idea past my mother, who helpfully pointed out that I was an idiot. “You’ve never had your own house. You don’t know what you want yet, and you don’t understand what’s easy and hard to change. Buy the house, live in it for a decade, change it a few dozen times, and then you’ll know enough about houses to design your own house.” We did, and she was right. Six years in, I’m still making changes. It’s not perfect – or even close to it – but we love it. It’s shaped us, and we’ve shaped it, in a long, ongoing, organic process.

The tools we’re running Global Voices on are far from perfect. The blog runs on a version of WordPress that’s been tweaked within an inch of its life. But despite the extensive changes Boris has made to plug-ins and templates, there are no changes to the underlying code. After two months of complaining that it would be impossible to get WordPress to do what we wanted it to do (specifically, support multiple authors making hundreds of tiny, heavily tagged posts, in a part of the blog independent of the main entries), Boris made it do exactly what we want it to do. (Well, almost. We’re still tweaking how the RSS feeds work. And getting Search to make more sense. And figuring out how email digests should work.) But basically the tool has let us do what we’ve wanted to do, and our success or failure has been based on human factors (recruiting smart, multilingual bloggers who can write articulately about their corners of the world) than technical ones.

The biggest unexpected success of Global Voices, in my opinion, has been the Bridge Blog Index. Basically nothing more than a set of premade, editable pages for different nations, the Index invites contributors around the world to create lists of bloggers and bridgebloggers for each country. In a surprising number of cases – Cambodia, Tanzania, Bangladesh, Jordan, Argentina, to name a few – there’s rich, detailed lists of vibrant blog communities. In many other cases, there’s nothing. But what’s amazing is that we often have no idea who’s creating this content.

The downside of this – a problem of all wikis – is that we’re highly susceptible to spam. Several times a day, some jackass decides to replace one or more pages of the wiki with a set of links promoting phenteramine or particularly foul porn sites in the hope that links to these sites will improve their Google rank and help them snare more customers. Fortunately, this stuff is easy to remove – because wikis keep history files, we can revert to previous editions of articles in a couple of clicks. But this only works if someone’s watching the wiki. (More on that in a moment.)

Recently, a contributor to our wiki has been using the example of wiki spam to make the argument that “wikis are weak” and will be destroyed by dedicated spammers. (He is, not coincidently, promoting an alternative solution.)

The obvious counterexample to this seems to be Wikipedia, where spam occurs, but is rapidly eliminated by a community of dedicated volunteer editors. Spam seems to become less of a problem with wikis once they’ve got a vibrant, involved community. (I notice it less on our wiki because I spend more time editing actual content these days, and less time removing spam, much of which has been eliminated by other editors.)

It reminds me of a talk (wise friend #4) organic farmer Sam Smith gave a few years back, where he argued that using insecticides or herbicides to protect one’s crops missed a fundamental point – insects and weeds were a symptom of underlying problems, usually with the condition of one’s soil. Rather than trying to eliminate potato beetles, one should contemplate what’s wrong with one’s fields that one would be experiencing potato beetles. Eventually, one becomes grateful for the potato beetles as their presence communicates valuable information about the conditions of one’s fields.

Potato beetles are a good indicator that you’re not rotating your fields often enough (and therefore allowing the beetle population to breed year after year) and that you’ve got insufficient diversity of flowering plants, which means you don’t have enough predatory insects, especially 10-spotted lady beetles, which eat Colorado potato beetle larvae. Spam on my wiki is a good indicator that I’m not spending enough time gardening the wiki and that I haven’t recruited a sufficiently large and dedicated team to help cull the spam and expand the wiki. While I have no doubt that there are other tools that would have helped me prevent spam from entering the wiki in the first place (Socialtext, for one), I’m not sure I would have gotten the same lessons about what makes a wiki work.

MediaWiki and WordPress have worked relatively well for us, and we’ve had some luck using del.icio.us tags GV and globalvoices to recommend stories to one another. We conduct most of our editorial meetings in the #globalvoices channel on freenode.net and through a Google Groups mailing list. Zoneedit takes care of email redirection and domain name forwarding. While I’ve got gripes with all these tools, none are serious enough to make me want to abandon them – though, in more than one case, I’m tempted to add new features to an existing featureset.

Online aggregation is another story. We’ve tried to use a Bloglines aggregator to create a feed of blogs that editors could use to create daily roundups, and which dedicated users might monitor. Three months into the experiment, it’s becoming clear it’s not the right tool for the job – most of our editors, myself included, don’t use it. It was never intended to accomodate the 800 feeds we’ve loaded into it, it doesn’t allow us to provide RSS feeds aggregated per country or per region, it doesn’t let us let users add feeds and allow us to moderate which are included in an aggregator… and so on.

After three months of trying to solve the aggregation problem with a wiki and bloglines, Boris and I have more or less agreed on a structure for how Global Voices should do aggregation. We’ll have directory pages for each nation and group, which anyone can add to. Some sort of script – perhaps interfacing with a future MediaWiki API? – will harvest URLs from those directories and enqueue them for inclusion in a regional aggregator. (People will be able to put URLs directly into the queue as well.) A regional editor will decide which of these URLs make it into this aggregator, which will likely run on Eyebeam’s ReBlog.

Users with a deep interest in a region would be able to read the regional ReBlog directly; most readers will choose to read just the stories the regional editor chooses for inclusion in the main Global Voices blog. Truly hardcore readers can go directly to a country’s page in the directory and select from all the feeds for a region. From directory to regional aggregator to main blog, there’s increasing editorial control and decreasing volume.

This solution bears very little resemblence to the solution I was sketching to the same problem six months back. Having lived in the house for a few months now and banged my share of nails into the floor, I now have a pretty good sense of which wall I now want to move. Had I built a hybrid wiki/aggregator six months ago, I’d probably be kicking myself and wondering whether I should scrap my own code and rebuilt around reBlog – now I find myself hoping I can convince the folks at reBlog to add a very few features to their code which would mean I don’t have to write anything at all.

I realize I may have just written an apologia for bad software. That wasn’t my intent (title aside). There’s some software that’s just plain bad, and you get no net benefits from struggling with it rather than abandoning it for something better. (Why yes, I am writing this on a Macintosh – why do you ask?)

What I’m attempting to do, instead, is write in praise of the cardinal virtue of programmers – laziness – and its sister virtue, patience. Being patient with imperfect tools allows you to learn what you really need a tool to do… which is likely quite different from what you think you want a tool to do. And being lazy increases the chances that someone else will solve the problem you need solved, allowing you to use her code rather than writing and maintaining your own. (And if you’re as inept as I am, there’s a damned good chance her code will be better than yours.)

All of which frees up more time to write essay-length blogposts on topics orthagonal to the stated aim of your blog, alienating your seven readers…