Tools for open government

I’m in Sebastapol, CA today and tomorrow, participating in a conversation hosted by Tim O’Reilly and Carl Malamud of public.resource.org. The topic is “Open Government” and the folks in the room are some of the leading developers working on tools to make government information – especially US and UK information – more accessible to a wider audience, including journalists, policy wonks and perhaps interested users. Thanks to air traffic in Newark and the ever-flexible United Airlines, I’m here a few hours late, trying to catch up with the state of the art by looking at people’s demos of new tools.

Adrian Holovaty is one of the superstars in this field, known for creating digital journalism tools like the Chicago Crime mashup and the Django web development framework. He shows off a tool created to help him co-author a book on Django. Rather than putting the text of the book into a wiki and allowing anyone to edit it, the system allows fine-grained commenting on a fixed text. While the book isn’t currently open for commenting, you can see the comments placed on each paragraph of text, often suggesting very specific refinements to the book.

There’s the interesting potential for this model for document annotation to start discussions around political documents. It probably doesn’t make sense to put the text of a political speech in a wiki – the speech was delivered and the discussion is around interpretation of the words of that speech. There’s the exciting possibility that document annotation could become a new form of community interaction. Tom Steinberg of MySociety pointed out that the Free Software Foundation is trying an annotation method to allow group discussion of the new GNU Public Licenses which shows lines that are uncontroversial or more controversial based on the number of comments they’ve received. There’s a sense in which tools for allowing group development of software – versioning systems, repositories – might be applied to group authorship of text as well.

Michael Dale from Metavid has created a remarkable tool for annotating video through a wiki model. It’s a bit like Democracy Player/Miro, DotSub and MediaWiki colliding at high speed. The current MediaWiki site hints at what the future will look like – it currently provides video from CSPAN correlated with transcripts, with the transcript and video embeddable within other publishign platforms. The forthcoming version allows users to improve these captions in wiki form, to search video via captioning, and to edit and package video for export. It looks like it’s going to be an amazing and powerful system when it’s released.

Greg Elin with Sunlight Labs is a master of meshing sets of political data. He talks about Sunlight’s holy grail – one click disclosure – integrating data from GovTrack.us, Open Congress, Center for Responsive Politics, GovernmentDocs.org and others. Sunlight has taken steps to ensure that these sites are cross-referenced and integrated, so you can view portraits of US politicians that include information on fundraising, contributions from lobbyists, voting on earmarks, etc. In the long term, Sunlight is looking into doing real-time analysis of newsfeeds from sources like AP, feeding the data through “data chewers” that monitor the articles for information on politicians and link the references to detailed profiles on the individuals in question. Elin points out that most newspapers don’t have the technical capacity to integrate this sort of data into online stories – his goal is to create a “journalists’ desktop” that puts this information at the hands of every reporter, and makes it as easy as possible for a paper to integrate this information into their coverage.

Tom Steinberg of MySociety is responsible for some of the most innovative projects in UK politics and online organizing. (Tom was very careful to correct me, reminding me that he’s not a programmer and that MySociety projects are put together by a team of paid and volunteer programmers and designers who work with him – he gives that team the credit for these remarkable projects.) He explains that They Work For You, a site he’s largely responsible for, began as a project to make the Hansard (the record of parliamentary proceedings) accessible, annotatable, and linkable. In the process, TWFY created profile pages for each UK parliamentarian, which includes information characterizing how they’ve voted, how many questions they ask in session and how well they respond to constituent questions.

These pages are often the best linked pages for UK parliamentarians, and they’re generated automatically, based on information reported by the UK government… and from Tom’s scripts as well. One of his sites invites constituents to ask questions of their parliamentarians, and surveys them two weeks later to see whether questions have been answered. This information is included in the profiles of MPs, which gives them a strong incentive to be responsive to constituent questions. (Steinberg has seen evidence that TWFY is so effective that some politicians have resorted to “spam speeches”, attempting to goose their numbers on TWFY to improve their electability.)

(The coolest thing about TWFY, in my opinion – rather than simply presenting stats, the site contextualizes them. Instead of telling you that an MP was present for 74% of votes, it tells you she was there for 74% of votes which is well below average. It’s a hugely useful technique nearly every political site could learn from.)

Other projects from MySociety focus on more personal aspects of politics. Fix My Street invites citizens to document problems in their local areas, including photos and geolocation information, so that local officials can see problems under their jurisdiction. The site has a comprehensive set of rules for routing email reporting problems to the proper authorities and has registered over 10,000 reported issues thus far. The Travel Time Maps project appeals directly to the heart of many Britons, showing the average commute time per neighborhood for areas across the nation. The isochrome maps make very clear what neighborhoods are and are not well served by public transit.

It’s clear that Steinberg’s projects are largely oriented towards citizens. It’s a bit harder to know the audience for some of these other projects. Journalists? Established political activists? Lobbyists? Congressional staffers? Some projects fall into the “build it and they will come” trap. Others are compelling enough that it’s possible that people will flock to these tools as they’re made available.