CHI keynote: Desperately Seeking Serendipity

I’m giving the closing keynote at CHI 2011 this afternoon. I’m thrilled to have the chance to share some thoughts with some of the smartest researchers and practitioners working on questions of human/computer interaction, and perhaps to poke some to help me think about a topic I’m increasingly obsessed with: creating structures, online and offline, to increase the chances of serendipity. I’m particularly honored to share the stage, virtually, with Howard Rheingold, who gave the opening keynote earlier this week, focused on his key work in digital learning and teaching.

I know from past experience that there’s no way I can say everything about a topic in a 40 minute keynote, even talking like a New Yorker on speed. This blog post serves as an “extended dance mix” of my talk, including some digressions I probably can’t make on stage and references to the research and ideas I’m referencing throughout the talk. If you’d like visual accompaniment, my slides from the talk are posted at SlideShare… but this post provides a narrative that they probably lack.

---

As of 2008, the majority of the world’s population lives in cities. In highly developed countries (the membership of the OECD), the figure is 77%, while in the least developed countries (as classified by the UN), 29% of people live in cities. It’s an oversimplification, but one way to think about economic development is a shift from a rural population, supported by subsistence agriculture, to an urban population engaged in manufacturing and service industries, fed by a small percentage of the population that remains focused on farming.

This graph from the World Bank may even understate the apparent inexorability of the rural/urban shift. In 1800, 3% of the world’s population lived in cities, many in European cities like London and Amsterdam. Even so, those societies had rural majorities – roughly 80% in England, 75% in the Netherlands. A century later, 14% had moved to cities. And since 1950, we’ve seen a rise in urban populations at a much faster rate than rural populations, and the United Nations Department of Economic and Social Affairs World Urbanization Prospects report predicts that we’re about to see this continued growth complemented by a decline in rural populations.

It may not sound intuitively obvious to people living in the developed world, but a city like Lagos – with a population of 8 million, over 4% growth a year, living in a dense, crowded, traffic-choken sprawl – is an extremely appealing destination for Nigerians living in rural areas. In a developing world city, the schools and hospitals tend to be far better than what’s available in rural areas. Even with high rates of unemployment, the economic opportunities in cities vastly outpace what’s available in rural areas. But there’s a more basic reason – cities are exciting. They offer options: where to go, what to do, what to see. It’s easy to dismiss this idea – that people would move to cities to avoid rural boredom – as trivial. It’s not. As Amartya Sen argued in his seminal book, “Development as Freedom“, people don’t just want to be less poor, they want more opportunities, more freedoms. Cities promise options and opportunities, and they often deliver.

What’s harder to understand, for me, at least, is why anyone would have moved to London in the years from 1500 – 1800, the years in which it experienced rapid, continuous growth and became the greatest metropolis of the 19th century. First, the city had an unfortunate tendency to burn down. The Great Fire of 1666, which left as many as 200,000 in the city homeless, was merely the largest of a series of “named fires” severe enough to distinguish themselves from the routine, everyday fires that imperiled wood and thatch houses, packed closely together and heated with open coal or wood fires. It’s likely that more Londoners would have been affected but for the fact that 100,000 – a fifth of the city’s population – had died the previous year from an outbreak of the bubonic plague, which spread quickly through the rat-infested city. (It didn’t help that mayor of London had ordered all cats and dogs killed for fear they were spreading the plague – instead, they were likely keeping the plague rats in check.)

By the time of Dickens’s London, the threat was less from fires than from the water system. Open sewers filled with household waste, as well as the manure of the thousands of horses used to pull buses and cabs, emptied directly into the Thames, which was the source of most of the city’s drinking water. We remember a particularly severe cholera epidemic in 1854 because it led to John Snow’s investigation of the Broad Street pump and the eventual vindication of the germ theory of disease. But cholera was common from the 1840s through the 1860s, due to a combination of open sewers and cesspits dug behind private residences, which often overflowed as London residents upgraded from chamber pots to the more modern flush toilet, which greatly increased the volume of human waste that needed disposal. The smell of London during the hot summer of 1858 was so bad that it led to a series of Parliamentary investigations – “The Great Stink”, as historians know the event, finally led to the construction of London’s sewer system in the 1860s.

People flocked to cities in the 18th and 19th centuries, but not for their health. In the 1850s, the life expectancy for a man born in Liverpool was 26 years, as compared to 57 years for a man in a rural market town. But cities like London had a pull not unlike that of Lagos now. There were more economic opportunities in cities, especially for the landless poor, and an array of jobs made possible from the international trade that flowed through the ports. For some, the increased intellectual opportunities provided by universities and coffee houses was an attraction, while for others, the opportunity to court and marry outside of closed rural communities was the reason to relocate. Amsterdam built itself to prominence in the 1600s in part by allowing French Huguenots, Spanish and Portuguese Jews and Dutch Catholics to worship relatively freely – such religious tolerance would have been much harder to find in rural areas.

To a large extent, the reason to come to the city was to encounter the people you couldn’t encounter in your rural, disconnected lifestyle: to trade with, to marry, to learn from, to worship with. You came to the city to become a cosmopolitan, a citizen of the world.

“Diogenes, Jean-Léon Gérôme, 1860

The term “cosmopolitan” comes from the Greek words for world (Cosmos – Κόσμος) and for city (Polis – Πόλις). It was coined by the Cynic philosopher Diogenes, who’d fled his native Sinope (possibly one step ahead of the authorities, as some accounts have him leaving home to escape charges of counterfeiting) for Athens, where he lived in a barrel in the agora, picked fights with prominent philosophers, and did his level best to violate every conceivable societal norm. (The dogs in the portrait above are a reference to his nickname, Diogenes the dog. Like his namesakes, historians reason, Diogenes ate, slept, bathed, urinated and defecated in public.) It’s probably worth reading his declaration that he was not a citizen of Athens or of Sinope, but of the world, as much as social transgression than as a lived identity.

Philosopher Kwame Appiah points out that living as a cosmopolitan, a citizen of the world, has really only been possible in the past few hundred years. If you were one of the 97% of people living in rural areas in 1800, it’s likely you would have had little or no contact with people who didn’t share your language, culture or belief system. One of the reasons we have such difficulty living in a genuinely cosmopolitan way, Appiah suspects, is that we have vastly more experience as a species with parochialism than with cosmopolitanism.

If you wanted to encounter a set of ideas that were radically different than your own – say those of a confrontational homeless guy who sleeps in a tub – your best bet in an era before telecommunications was to move to a city. Cities are technologies for trade, for learning, for worship, but they’re also a powerful communication technologies. Cities enables realtime communication between different individuals and groups and the rapid diffusion of new ideas and practices to multiple communities. Even in an age of instantaneous digital communications, cities retain their function as a communications technology that enables constant contact with the unfamiliar, strange and different.

To the extent that a city is a communications technology, it may not be a surprise that early literally portrayals of the internet seized on the city as a metaphor. Early cyberpunk authors, like William Gibson and Neal Stephenson, were fascinated by the ways in which the internet could bring the weird, dangerous and unexpected (as well as the trivial, mundane and safe) into a constant fight for your attention. Both seized on cities as a way the future internet would present itself to participants, which is slightly odd, given that Gibson was utterly naïve about computing technologies, writing Neuromancer on a typewriter, while Stephenson was a seasoned programmer, developing Macintosh software in the hopes of rendering Snow Crash as an animated film. And, after all, there’s no reason data can’t be presented as a forest of trees or a sea of bits.

But both Gibson and Stephenson were interested in virtual spaces as ones in which people were forced to interact because lots of people wanted to be in the same spaces at the same time, bumping into each other as they headed towards the same destinations. On the one hand, it’s an insane way to visualize data – why would we force people into close contact when we’re building “spaces” that can be infinite in scale? Both believed that we’d want to interact in cyberspace in some of the ways we do in cities, experiencing an overload of sensation, a compression in scale, a challenge of picking out signal and noise from information competing for our attention.

We hope that cities are serendipity engines. By putting a diverse set of people and things together in a confined place, we increase the chances that we’re going to stumble onto the unexpected. It’s worth asking the question: do cities actually work this way?

In 1952, French sociologist Paul-Henry Chombart de Lauwe asked a young political science student to keep a journal of her daily movements as part of his city study “Paris et l’agglomération parisienne”. He mapped her paths onto a map of Paris and saw the emergence of a triangle, with vertices at her apartment, her university and the home of her piano teacher. Her movements illustrate, “the narrowness of the real Paris in which each individual lives.”

That pattern of home, work and hobby – whether it’s a comparatively solitary activity like piano studies or the “great good place” of public socialization celebrated by Ray Oldenburg – is a familiar one to social scientists. Most of us are fairly predictable. Nathan Eagle, who has worked with Sandy Pentland at MIT’s Media Lab on the idea of “reality mining”, digesting huge sets of data like mobile phone records, estimates that he can predict the location of “low-entropy individuals” with 90-95% accuracy based on this type of data. (Those of us with less predictable schedules and movements might be only 60% predictable.)

We might choose to see our predictability as evidence of contentment and lives well lived. Or we can react as situationist cultural critic Guy Debord did and decry the “outrage at the fact that anyone’s life can be so pathetically limited.” One way or another, the likelihood we will be confronted with one of these maps is increasing.

Zach Seward, outreach editor for the Wall Street Journal, is a heavy Foursquare user. As he checks in at venues in and around New York City, he generates a “heat map” of his wanderings. It’s easy to see a heavy concentration around Manhattanville, where he lives, and midtown, where he works. With a bit more work, we can see that he enjoys hanging out in the East Village, rarely strays into the “outer boroughs” except to fly from LaGuardia and to watch baseball games – the one venue he’s checked into in the Bronx is Yankee Stadium.

If you’re using Foursquare, you’re broadcasting the data that can be used to make a map like this one. Yiannis Kakavas has developed a software package called “Creepy” designed to allow users – or people watching users – to build maps like this one from information posted on Twitter, Flickr and other geolocated services. Creepier, perhaps, is the discovering that you’re leaking this data simply by using a mobile phone. German Green Party politician Malte Spitz sued his phone company, Deutsche Telecom, to gain access to whatever data they’d retained on his phone usage. He ultimately obtained an Excel file with over 35,000 lines of data, each recording his location and activities. Working with German newspaper Die Zeit, he turned the data into a map of his movements over six months and published it online. While you may not be interested in suing your mobile phone provider, it’s likely they have similar data on your movements, which could be released to law enforcement on request… or perhaps used to build a behavioral profile to target ads to you.

Seward took a close look at his Foursquare check ins and discovered they provide a piece of profile information he hadn’t realized he was providing: his race. He overlaid his check-ins in Harlem over a map that showed the racial composition of each block and discovered that “his” Harlem is almost exclusively blocks that are majority-white. As he observes, “Census data can describe the segregation of my block, but how about telling me how segregated my life is? Location data points in that direction.”

It’s worth pointing out that Seward is neither a racist, nor is he “pathetically limited”, as Debord suggests. We all filter the places we live into the places where we’re regulars and the ones we avoid, the parts of town where we feel familiar and where we feel foreign. We do this based on where we live, where we work, and who we like to spend time with. If we had enough data from enough New Yorkers, we could build maps of Dominican New York, Pakistani New York, Chinese New York, as well as black and white New Yorks.

The patterns we trace throughout our cities tend to reflect a basic sociological truth: birds of a feather flock together. Lazarsfeld and Merton saw the effects of homophily in patterns of friendship in Hilltown, Pennsylvania and Craftown, NJ, where neighbors were more likely to establish close friendships if they shared common demographic (racial, religious, economic) characteristics, and a wealth of sociological research has confirmed the effects of homophily in social networks.

When we talk about cities, we recognize that they’re not always the cosmopolitan melting pots we dream they are. We acknowledge the ethnic character of neighborhoods, and we’re conscious of ghettos that get separated, through a combination of physical structure and cumulative behavior, from the rest of the city. (Bill Rankin’s Chicago Boundaries map, which shows racial self-identification atop a map of Chicago makes these structures uncomfortably apparent.) We hope for random encounter with a diverse citizenry to build a web of weak ties that increases our sense of involvement in the community, as Bob Putnam suggested in Bowling Alone. And we worry that we may instead isolate and cocoon ourselves when faced with a situation where we feel like outsiders, as Putnam’s recent research suggests.

---

I’m less interested in the ways in which we limit our paths through cities than in how we constrain what we do and don’t encounter online. As with cities, where urban planning and design interact with individual behavior, I don’t want to make the case that our constraints are solely by choice. But through the design of the systems we use and our behavior with those systems, I see reasons to worry that our use of the internet may be less cosmopolitan and more isolated that we would hope.

In 1993, MIT Media Lab researcher Pascal Chesnais designed a piece of software called Freshman Fishwrap. Using a range of online news sources available at the time, Fishwrap allowed individuals to produce a customized digital newspaper, including mentions of their hometown or favorite sports team and filtering out less interesting news. Nicholas Negroponte celebrated the project in his book “Being Digital”, pointing to it as part of the personalized future possible in a digital age.

Legal scholar Cass Sunstein saw the Daily Me as a threat, rather than a promise. In his book Republic.com, he articulated a fear of internet echo chambers, where individuals could encounter only views they agreed with. In such an environment, Sunstein worried, we would see increased political polarization and a shift of moderate views to the extremes.

Much of the scholarly response to Sunstein’s critique has focused on not on countering the argument that isolation leads to polarization, but on demonstrating that the internet is not as polarizing as he thinks. Matthew Gentzkow and Jesse Shapiro studied the online reading habits of thousands of US internet users and concluded that while some internet sites have a great deal of partisan separation, the news sites most visited by internet users (Yahoo! News, CNN, AOL News, MSNBC) were visited both by left and right-leaning users. They conclude that the internet may be more polarized – in terms of who reads what content – than most forms of broadcast media, but suggest the polarization is less than we fear, and less than we likely experience in our physical communities.

I’m less concerned about left-right polarization in the US, and more concerned about us/them polarization around the world. Above is a visualization of the data Gentzkow and Shapiro collected, put together by the team at Slate to show the polarization of small sources versus the broad appeal of the larger sites. I’ve annotated it with a couple of labels, and with yellow boxes, which show which news sources are non-US. You’ll note that there’s not a lot of yellow in the image – the largest international news source, in terms of page views, is the BBC, which is probably the most visited “pure news” site on the entire Web. (You may also note that it’s got a much larger share of liberal than conservative readers – 22% conservative versus 78% liberal.)

It’s not that Americans are particularly bad about favoring local news sources over international news. I analyzed the media preferences of 33 nations using data from Doubleclick Ad Planner and discovered that the US preference for domestic news sources (roughly 93% to 7% when I ran the analysis in May 2010) is actually pretty low in comparison to the 9 other nations with the most internet users. Countries that have more than 40 million or more internet users generally have a very strong bias towards local sources – the mean is roughly 95%/5%, which makes Americans look (slightly) cosmopolitan in comparison.

This data set doesn’t tell us about our appetite for international news so much as it comments on our preference for content pitched to ourselves and our countrymen. It’s possible that we’re getting tons of international news from Yahoo or CNN, though there’s good reasons to think otherwise. (Media Standards Trust in the UK saw a sharp drop in the percent of UK newspapers focused on international stories over the past 30 years, and research conducted by Alisa Miller of Public Radio International suggests that US broadcast media focuses much more on entertainment stories than on international news.) What’s striking to me about this preference data is that there’s so little effort required to access international news sources like BBC, the Times of India or the Mail and Guardian – they’re one click away and don’t require crossing a language barrier – and how strong the “local” bias for national news sources appears to be.

Here’s the danger of this sort of isolation – we miss important stories. Through my work on Global Voices, I’m blessed with a set of close friends from around the world, and I often catch glimpses of important breaking stories, either through the work we do on the site, of from my friends’ preoccupations on their social media feeds. In late December 2010, it became clear that something very unusual was happening in Tunisia – friends like Sami Ben Gharbia were both covering the protests unfolding in Sidi Bouzid and spreading across the country, and asking loudly why no media outside the region was covering the revolution underway. I got into the act with one of my better-timed blogposts – on January 12th, I published “What if Tunisia had a revolution, but nobody watched?“… and I got a lot of phone calls when Ben Ali fled the country two days later.

The revolution in Tunisia caught intelligence and diplomatic services around the world flat-footed. It didn’t have to – there was a wealth of information being published on Tunisian Facebook pages, aggregated by groups like Nawaat.org and distributed on Al Jazeera (primarily through their Arabic service.) But this shift from a world where news is dominated by superpowers to a multipolar world is a hard one for diplomats, the military, the press and individuals to get used to. And if I’m honest about my view of the world, I’m forced to admit that there’s no way I would have known about the revolution brewing if I didn’t have close Tunisian friends.

Like everyone else, I’m experiencing a shift in how I get news about the world. In the pre-web world and early web days, news of the world came primarily through curated media – broadcast television, newspapers, magazines. There were – and are – reasons to distrust curators, but there’s a critical aspect of their work I believe we need to preserve as we move towards new models for organizing news. Curators implicitly tell us what they believe we need to know about the world. High quality curators often have a broader view of the world than individuals have, and well-curated media often demands we pay attention to people, places and issues we might have otherwise ignored.

On the other hand, curators invariably have biases, and the ability to seek information that appeals to our own interests and preferences is one of the most powerful capacities the modern web has put in our hands. Search lets me learn a great deal about things I care about – sumo, African politics, Vietnamese cooking – but it’s quite possible that I miss topics that I needed to know about because I was paying more attention to my interests and less to curators. We need mechanisms to ensure that search gets complemented with serendipity.

There’s a trend in the design of web tools that seeks to guide us to novel content by examining what our friends care about. Community-based tools like Reddit, Digg and Slashdot have formed communities around shared interests and direct us to stories the community agreed (through voting and karma mechanisms) is interesting and worth sharing. Twitter, and especially Facebook, work on a much more personal level. They show us what our friends know and believe is important. Or as Brad DeLong puts it, Facebook offers a different answer to the question, “What do I need to know?” – “You need to know what your friends and your friends of friends already know that you do not.”

The problem, of course, is that if your friends don’t know about a revolution in Tunisia or a great new Vietnamese restaurant, you may not know either. Knowing what your friends know is important. But unless you’ve got a remarkably diverse and well-informed set of friends, there’s a decent chance that their collective intelligence has some blind spots. Guardian columnist Paul Carr tells a funny story about returning to a San Francisco hotel room and being baffled that it, and the rest of the hotel, hadn’t been cleaned that day. The hotel workers were protesting the Arizona immigration bill, SB1070, and while there was extensive conversation about the protests and the legislation on Twitter, they weren’t taking place on feeds Carr followed on Twitter. By missing the protests (until they manifested as an unmade bed in his room), Carr realized that he was living in “my own little Twitter bubble of People Like Me: racially, politically, linguistically and socially.” It’s worth asking whether that bubble is able to provide us with the serendipity we hope for from the web.

---

A brief look at the word “serendipity”: Robert K. Merton devoted a book, written with collaborator Elinor Barber and published posthumously, to the topic. This may seem an odd exploration for a celebrated sociologist, but it’s worth remembering that one of his many contributions to the field was an examination of “unintended consequences”. Serendipity, at first glance, looks like the positive side of unintended consequences, the happy accident. But that’s not what the term meant, at least originally. The word was coined by Horace Walpole, an 18th century British aristocrat, 4th Earl of Oxford, novelist, architect and gossip. He’s remembered primarily for his letters, 48 volumes worth, which offer a perspective on what the world looked like through an aristocrat’s eyes.

In a letter written in 1754, Walpole tells his correspondent, Horace Mann, about a unexpected and helpful discovery he made, due to his deep knowledge of heraldry. To explain the experience, he refers to a Persian fairy tale, The Three Princes of Serendip, in which the titular characters were “always making discoveries, by accidents and sagacity, of things they were not in quest of.” Walpole’s neologism is a pat on the back – he’s congratulating himself both for a clever discovery and for his sagacity, which permitted the discovery.

Useful as the concept is, the word “serendipity” didn’t come into wide use until the past couple of decades. By 1958, Merton tells us, it had appeared in print only 135 times. In the next four decades, it appeared in book titles 57 times, and graced newspapers 13,000 times in the 1990s alone. A Google search turns up 11 million pages with the term, including restaurants, movies and gift shops named “serendipity”, but very few on unexpected discovery through sagacity.

Merton was one of the major promoters of the word, writing about “the serendipity pattern” in 1946 as a way of understanding unexpected scientific discoveries. Fleming’s discovery of penicillin in 1928 was triggered by a spore of Penicillium fungus that contaminated a petri dish where he was growing Staphylococcus bacteria. While the mold spore landing in the dish was an accident, the discovery was serendipity – had Fleming not been cultivating bacteria, he wouldn’t have noticed a stray mold spore. And had Fleming not had a deep understanding of bacterial development – sagacity – it’s unlikely he would have noticed the antibiotic properties of Penicillium and developed the most important advance in health technology of the first half of the 20th century.

Louis Pasteur observed, “In the fields of observation chance favors only the prepared mind.” Merton believed that serendipity emerged both from a prepared mind and from circumstances and structures conducive to discovery. In “The Travels and Adventures of Serendipity”, he and Barber explore discovery in a General Electric laboratory under the leadership of Willis Whitney, who encouraged a work environment that focused as much on fun as it did on discovery. A healthy blend of anarchy and structure was necessary for discovery, and over-planning was anathema as “the policy of leaving nothing to chance is inherently doomed by failure.” (Riccardo Campa’s review of Merton and Barber’s book is very useful for anyone interested in questions of serendipity and structure.)

The idea that serendipity is a product both of an open and prepared mind and of circumstances conducive to discovery can be traced back to the story quoted by Walpole in 1754. The three princes were deeply learned in “Morality, Politicks and all polite Lerning in general”, but they did not make their unexpected discoveries until their father, the Emperor Jafer, sent them out from his kingdom to “travel through all the World, to the end that they might learn the Manners and Customs of every nation.” Once the well-prepared Princes met circumstances conducive to discovery, unexpected and sagacious discoveries occurred. (For more on the 1722 translation of the Three Princes of Serendip, you may be interested in this blog post.)

When we use the word “serendipity” now, it’s usually to mean “a happy accident”. The parts of the definition that focus on sagacity, preparation and structure have slipped, at least in part, into obscurity. Our loss, I believe, is that we’ve lost sight of the idea that we could prepare ourselves for serendipity, both personally and structurally. I suspect that we – and even Merton – understand those preparations poorly. And, as my friend Wendy Seltzer pointed out to me, if we don’t understand the structures of serendipity, it appears no more likely than random chance.

If we want to create online spaces to encourage serendipity, we might start by learning from cities.

In the early 1960s, a fierce public battle erupted over the future of New York City. The proximate cause of the battle was the Lower Manhattan Expressway, a proposed ten-lane elevated highway that would have connected the Holland Tunnel (which links Manhattan and New Jersey under the Hudson River) to the Manhattan and Williamsburg Bridges (which cross the East River and connect Manhattan to Brooklyn and Queens.) Plans for the highway required the demolition of 14 blocks along Broome Street in Little Italy and Soho, and would have displaced roughly two thousand families and eight hundred businesses.

The proponent of the plan was Robert Moses, the legendarily influential urban planner responsible for much of New York’s park and highway systems. His fiercest opponent was Jane Jacobs, activist, author and chairperson in 1962 of the “Joint Committee to Stop the Lower Manhattan Expressway”. The lasting legacy of Jacobs’s opposition to Moses is both the survival of Broome Street and her masterwork, “The Death and Life of Great American Cities”, which is both a critique of “rationalist” urban planning and a manifesto for preserving and designing vibrant urban communities.

Jacobs framed many of her battles over urban planning by asking whether cities were for the benefit of cars or of people, suggesting Moses’s indifference to the people he proposed to displace. A slightly less biased frame might be to observe that Moses took a bird’s-eye, city-wide view of urban planning while Jacobs offered a pedestrian-eye, street level view of the city. From Moses’s point of view, one of the major challenges of a city is allowing people to move rapidly from their homes in the suburbs to business districts in the center of cities, and back out to the “necklace” of parks he’d painstakingly constructed in the outer boroughs.

This principle of separation of uses – residential neighborhoods separate from business districts, separated from recreation areas – was one of the main foci of Jacobs’s critique. What makes cities livable, creative, vital, and ultimately, safe is the street-level random encounter that Jacobs documented in her corner of Greenwich Village. In neighborhoods where blocks are small, pedestrians are welcome and there’s a mixture of residential, commercial and recreational destinations, there’s a vibrancy that’s thoroughly absent from planned residential-only communities or from city centers that empty out when offices close. That vibrancy comes from the ongoing chance encounter between people using a neighborhood for different purposes, encountering one another as their paths intersect and cross.

Jacobs’s vision of a livable city has been a major influence on urban design since the early 1980s, with the rise of “New Urbanism” and the walkable cities movement. These cities – and Vancouver, where we’re having this conference is a prominent example of one – tend to favor public transit over private automobiles, and create spaces that encourage people to interact and mix, in mixed-use neighborhoods and pedestrian-friendly shopping streets. As urban planner David Walters observes, they’re designed to help individuals linger and mix: “Casual encounters in shared spaces are the heart of community life, and if urban spaces are poorly designed, people will hurry through them as quickly as possible.”

If there’s an overarching principle to street-level design, it’s a pattern of designing to minimize isolation. Walkable cities make it harder for you to isolate yourself in your home or your car, and easier to interact in public spaces. In the process, they present residents with a tradeoff – it’s convenient to be able to park your car outside your home, but walkable cities ask you to be suspicious of too much convenience. The neighborhoods Jacobs celebrates are certainly not the most efficient in terms of an individual’s ability to move quickly and independently. Vibrancy and efficiency may not be diametrically opposed, but it’s likely that the forces are in tension.

---

Cities embody political decisions make by their designers. So do online spaces. But urban planners tend to be more transparent about their agendas. Urban planners will declare an intention to create a walkable city with the logic that they believe increased use of public space will improve civic life. And, in the best of cases, planners test to see what works and report failures when they occur – the persistence of private car use in walking cities, for instance. It’s much harder to get the architects behind Facebook or Foursquare articulate the behaviors they’re trying to enable and the political assumptions that underly those decisions.

I think many people who are designing online spaces are trying to increase exposure to diverse range of information and to cultivate serendipity. But I also worry it’s difficult to accomplish, in part because it’s too easy to start from scratch. An urban planner who wants to make changes to a city’s structure is held in check by a matrix of forces: a desire to preserve history, the needs and interests of businesses and residents in existing communities, the costs associated with executing new projects. Progress is slow, and as a result, we’ve got a rich history of cities we can study to see how earlier citizens, architects and planners have solved these problems.

It’s possible to gain inspirations about the future of Lagos by walking the streets of Boston or Rome. For those planning the future of Facebook, it’s hard to study what’s succeeded and failed for MySpace, in part because an exodus of users to Facebook is gradually turning MySpace into a ghost town. It’s harder yet to study earlier communities, like LamdaMOO or Usenet of the early 1980s. I often find myself nostalgic for Tripod, the proto-social network I helped build in the late 1990s. The admirable Internet Archive includes several dozen snapshots of pages on the site from 1997 – 2000, which gives a sense for the changing look and feel, but doesn’t give much insight into the content created by the 18 million users of the site in 1998. Tripod’s more successful competitor, Geocities, disappeared from the web entirely in 2010 – it’s legacy is less than 23,000 pages stored accessible through the Wayback Machine, which threw up its hands at the impossible task of archiving the vast site in mid-2001.

If we learn from real-world cities instead of abandoned digital ones, what lessons might we take?

The Jacobs/Moses debate suggests we need to be cautious of architectures that offer convenience and charge isolation as a price of admission. This is the concern Eli Pariser articulates in his (excellent) new book, “The Filter Bubble“. He worries that between Google’s personalized search and the algorithmic decisions Facebook makes in displaying news from our friends, our online experience is an increasingly isolated one, which threatens to deprive us from serendipitous encounter. Filter bubbles are comfortable, comforting and convenient – they give us a great deal of control and insulate us from surprise. They’re cars, rather than public transit or busy sidewalks.

With the rise of Facebook’s “like” button on sites across the web, we’re starting to see personalization come into play even on heavily curated sites like the New York Times. I can access whatever stories I want, but I also get signals of which of my friends have “liked” the story I’m reading, and what other stories they’ve liked as well. It’s not hard to imagine a future where “like” informs even more information spaces. In the near future, I expect to be able to pull up an online map of Vancouver and see my friend’s favorite restaurants overlaid on top of it. (I can already, using Dopplr, but I expect to see this functionality creeping into Mapquest, if not Google Maps, at some point soon.)

Whether that scenario is exciting or troubling has a lot to do with whether I see only my friends’ recommendations, and whether I can see the favorites of other communities too. As Eli observes, the filters we really have to worry about are those that are opaque about their operations and on by default. A map of Vancouver overlaid with my friends’ recommendations is one thing; one that recommends restaurants based on paid advertisements and doesn’t reveal this practice is another entirely. The map I want is the one that lets me shuffle not just through my friends’ preferences but through annotations from different groups: first time visitors to the city; long-time Vancouverites; foodies; visitors from Japan, Korea or China.

When we wander a city, we encounter thousands of signals about ways other people use the space. The crowd waiting to get into one bar and the empty stools in another; a lively basketball court in one playground, mothers with toddlers in another, unused benches in a third. People’s actions inscribe their intentions onto a city. The newly planted grass in a park becomes crisscrossed with paths, worn to dirt by people’s footsteps. Frustrating as these “desire lines” are for landscapers, they send invaluable signals to urban planners about where people are coming from and going to, and how they want to use a space.

Online spaces are often so anxious to show me how my friends are using a space that they obscure how other audiences are using it. In the run up to revolutions in Tunisia and Egypt, an enormous amount of reporting (and a not-insignificant amount of organizing) took place on Facebook. If you didn’t have friends in those countries, and specifically in those movements, that activity was entirely invisible. It’s possible to find out what’s popular on Facebook to an audience broader than that of your friends. The Pages directory shows stars, bands and brands with audiences in the hundreds of thousands and millions – strolling through it is a pretty fascinating tour of what’s popular in the Philippines, Colombia and Nigeria, as well as in the US or Canada. Facebook has the data on the desire lines, but they bury it deep within a site rather than bringing it front and center. Twitter’s Trending Topics in an example of making these desire lines visible – we may not know what “Cala Boca Galvao” means when it shows up as a trending topic, or care that #welovebieber, but at least we get indications of what matters to those outside of our list of friends.

Whether we click on an unfamiliar Twitter tag or explore someone else’s annotations of a city map, we’re choosing to stray from our ordinary path. Cities offer multiple ways to wander, as well as a philosophical stance – the flâneur – that prizes wandering as strategy for encountering the city. I think two particular forms of structured wandering have strong potential to be useful in wandering through online spaces.

A few weeks ago, I met an old friend for lunch in New York City. In the twenty years since we’d last met, he’d become a leading figure in the US Communist Party (an organization that, I confess, I thought had disappeared sometime in the late 1960s). As we walked from a restaurant to his office, across from the legendary Chelsea Hotel, he pointed to otherwise unremarkable office and apartment buildings and told me stories about the unions that had built them, the tenants’ rights struggles that had unfolded, the famous Communists, Socialists and labor activists who’d slept, worked and partied under each roof. Our twenty block walk became a curated tour of the city, an idiosyncratic map that caused me to look closely at buildings that would otherwise have been background noise. I begged him to turn his tour of the city into an annotated map, a podcast walking tour, anything that would allow a broader audience to look at the city through his lens, and I hope he will.

One of the reasons curation is such a helpful strategy for wandering is that it reveals community maxima. It can be helpful to know that Times Square is the most popular tourist destination in New York if only so we can avoid it. But knowing where Haitian taxi cab drivers go for goat soup is often useful data on where the best Haitian food is to be found. Don’t know if you like Haitian food? Try a couple of the local maxima – the most important places to the Haitian community – and you’ll be able to discern the answer to that question pretty quickly. It’s unlikely you dislike the food because it’s badly made, as it’s the favorite destination for that community – it’s more likely that you simply don’t like goat soup. (Oh well, more for me.) If you want to explore beyond the places your friends think are the most enjoyable, or those the general public thinks are enjoyable, you need to seek out curators who are sufficiently far from you in cultural terms and who’ve annotated their cities in their own ways.

Another way to wander in a city is to treat it as a game board. I’m less likely to explore Vancouver by following a curated map than I am by searching for geocaches. Within five kilometers of this conference center, there are 140 packages hidden somewhere in plain sight, each containing a logbook to sign and, possibly, mementos to trade with fellow players. As a geocacher, it’s something of a moral imperative to find as many of those caches as time allows during your visit to an unfamiliar city. In the process, you’re likely to stray far from the established tourist sites of the city, if only because it’s hard to hide caches in such busy places. Instead, you’ll end up in forgotten corners, and often in places where the person who placed the cache wanted you to see something unexpected, historic or beautiful. Geocaching is its own peculiar form of community annotation, where the immediate goal is leaving your signature on someone else’s logbook, but the deeper goal is encouraging you to explore in a way you otherwise wouldn’t.

Other games make explicit the connection of exploring to expanding civic capital. SF0, founded by a trio of Chicagoans transplanted to San Francisco, was designed to encourage players to discover things they’d never seen or done in the city, in a way that encouraged independence and exploration. Their game, SF0, invites you to score points by carrying out tasks, many of which are surreal, silly or surprising. You score by documenting your “praxis” and posting photos, videos and other evidence of the intervention. What’s so exciting to me about the game is how many tasks are specifically designed to encourage encounters with unfamiliar people or locations – one task requires you to convince total strangers to invite you into their house for dinner. The players who’ve completed the task report that it was surprisingly easy and that their hosts seemed to appreciate the random, unexpected contact as much as the players did. (More musings on SF0 in this blog post.)

Not all games are played by groups. Many years ago, Jonathan Gold set up a game mechanic for himself the year he decided to eat at every restaurant along Pico Boulevard in Los Angeles. The resulting article, “The Year I Ate Pico“, is an amazing exploration of the diversity of ethnic food available in that city, and his path down Pico helped launch his Counter Intelligence column for the LA Weekly. That work that eventually won him a Pulitzer, the first one awarded to a food critic. I see some of the same mechanics in the wonderfully strange project, “International Death Metal Month“, where the curators are mining YouTube to find death metal bands in each of 195 UN-recognized nations. Perhaps Botswanan death metal is unlikely to become your personal cup of tea, but using your set of interests as a lens through which to view the world is a time-honored xenophilic tactic used by the likes of Anthony Bourdain or Dhani Jones.

There’s a danger in taking these geographic metaphors too far. As attractive as we can make the game mechanics, as compelling as we can make the curation, it still takes a long time to get from the Bronx to Staten Island. One ability we have in digital spaces is to change proximities – we can sort bits any way we want to, to reshuffle our cities any way we can imagine. We can create neighborhoods that are all waterfront, all park, all brick buildings, all eight story buildings built in 1920 and discover who and what we encounter in these new spaces.

My friends at the Harvard Library Innovation Lab have been experimenting with reshuffling the library shelve, one of the most powerful structures we have to encourage constructive stumbling through an information landscape. Sorted by subject, we start with what we think we want to know and expand our search visually, broadening the topics we consider as our eyes move away from our initial search. As we scan the stacks, there’s information available about a book from its appearance – its age, its size. Width tells us whether the volume is brief or long, height is often a hint at whether a book contains pictures, as tall books tend to feature colored photos.

ShelfLife, a new tool developed at the Lab, offers the ability to reshelve books using these physical factors – size, width, height, age – as well as by data like subject, author, popularity with a group of professors or a group of students. The goal is to take what’s useful about physical ways of organizing and the implicit information conveyed in those schemes and combine it with the flexibility or organizing digital information. Combining the insights we may find from studying the organization of cities with the ability to reshuffle and sort digitally may let us think about designing online spaces for serendipity in different and powerful ways.

This isn’t a talk that ends with conclusions – it ends with questions. I don’t have a good sense of exactly which insights we might take from studying cities are best applied in virtual spaces – for me, these questions are best answered with experiments:

– How do we design physical spaces to encourage serendipity?
– What lessons about serendipity in physical spaces can we bring into the virtual realm?
– How can we annotate the physical world, digitally, in ways that expand our encounters with the world, rather than limiting them?