Social software, serendipity and salad bars. (Mmm. Sybillance…)

My friend Evgeny Morozov‘s bookmarks on del.icio.us are one of the places I look for inspiration when I’m feeling burned out on blogging, writing or thinking… which is more or less how I’m feeling near the tail end of a very long year. Evgeny linked to a story from Nat Torkington on O’Reilly Radar about the ways in which social software can reinforce homophily – the tendency of individuals to associate with people who are alike in age, gender, class, value terms – and how users or designers of tools might fight these effects.

Nat points out, “Designers first need to decide whether homophily is a a feature or a bug. Life is easy when you’re unchallenged: this is why people read the New York Times or watch Fox News…” This is, in essence, what Cass Sunstein worries about in Republic.com – in a world where one can choose media that matches one’s preconceptions and prejudices, what prevents us from choosing to live in an echo chamber of supportive voices?

There’s an odd paradox at work in the world of the pervasive web. On the one hand, it’s easier than ever before for an individual to share her ideas with the entire world. On the other hand, the mechanisms we use to discover ideas may make it harder for us to discover different ideas from different people. Assume that our social networks contain a lot of people who’ve got similar interests and backgrounds to our own, as homophily implies. There’s a good chance those folks are going to recommend similar stories for us to read. If our social networks become a major source of new ideas for us, there’s a real danger that homophily traps us in a conceptual echo chamber.

(At this point, if you’re like most readers, you’re ready to fight the premise of homophily with a list of all the people you spend time with/link to/have friended on Facebook, etc. who don’t share your nationality, language, socioeconomic status or race. That’s okay. danah’s here for you: “Sociological fact: most white people hang out with mostly other white people. Individually, everyone immediately screams not me! and starts listing off all of the people of color that they know. Individuals never want to see themselves as non-diverse, but the desire to be seen in a positive light does not make someone diverse.”

You may also be asking the “what’s so bad about homophily?” question. Here Sunstein can offer some help, suggesting that people who hear only similar voices end up polarized, less likely to compromise with people of different political viewpoints, less likely to find the sort of common ground and experience neccesary for a democratic society. I’d go further and argue that too much homophily can make you a) dumb and b) boring, ignorant of news and ideas that aren’t already interesting to people around you, and incapable of bringing ideas to your friends that they haven’t already heard.)

Examining ways around the homophily trap, Torkington looks closely as collaborative filtering, the technology that underlies most online recommendation systems. Rate a couple dozen movies, and the system looks for other users who liked the same movies you did (and, sometimes, disliked the same movies you did) and recommends to you the movies they like that you haven’t yet rated. That’s all well and good, but if you happen to like the same movies that other white computer geeks from rural America like, you’re very unlikely to be recommended a movie that’s the favorite of urban latino fashionistas.

Torkington suggests that social software should consider ways to make these serendipitious recommendations. It would be trivial, for instance, for Netflix to offer a feature titled “People different from you love these movies”. They’ve already calculated your nearest neighbor – in the process, they calculate your furthest neighbor. The question is whether these recommendations would be at all interesting to you – it’s not a recommendation of bad movies, or even movies that might be a bad fit for you, but movies that are loved by those different from you…

While Torkington takes a swing at the New York Times in framing homophily, newspapers like the Times have a terrific mechanism to encourage serendipity. In many major newspapers, the lower right-hand side of the front page is reserved for a story that readers would otherwise likely miss. (Friday’s paper is a good example. On a day where leading stories were about steroids in baseball, Al Qaeda and the US presidential race, the serendipity box featured a fascinating story about a Liberian mother in Staten Island sending her son back to Liberia rather than lose him to gang violence in the US.) These stories aren’t selected by algorithms – they’re chosen by editors who want to feature content in the paper that might otherwise be ignored, which frequently includes stories on topics other than Iraq, US elections or terror. Dan Gillmor describes this feature as “institutionalized serendipity“.

It’s less clear where the institutionalized serendipity lives on the New York Times’s website. The NYTimes homepage features several times as many stories on its webpage than on the front page of the paper edition, but it’s much less clear which ones you’re encouraged to read. There’s more choice and less guidance… which isn’t a bad description for the information universe opened by the Internet. And the guidance that’s offered may be a homophilic form of guidance – in the bottom right of the homepage is a box that offers a list of the ten most popular stories, as measured by email traffic, blog links and searches. In other words, these are the stories that fellow websurfers found most interesting, not the stories the editors felt you should read, even if you didn’t know you were interested in them.

The serendipity box in the paper New York Times is a form of persuasive technology – it convinces us to pay attention to information we’d otherwise ignore. As BJ Fogg notes on the homepage on Stanford University’s Persuasive Technology Lab, the idea that technologies are persuading us, not just people’s arguments, can be an uncomfortable topic. But it’s also a very powerful tool.

By way of example: I visit the Googleplex in Mountain View a couple of times a year. Being a cheapskate, I usually try to arrange to be there for lunch to take advantage of Google’s amazing cafeterias. My favorite of the lunchrooms offers a huge salad bar, complete with chefs who dress your salad on your behalf. This is an odd phenomenon. At first glance, I assumed that this was Google’s way of saying, “We’re so wealthy and successful, we can afford to save our geeks the hassle of dressing their own salads.”

But that’s not the reason why. Dressing is where many of the calories in a salad come from, and most people over-dress their salads, turning a healthy meal into a more fattening one. Google’s salad chefs put a modest amount of dressing on your salad and toss it in steel bowls, so that your salad is thoroughly dressed, but not unheathily so. Look closer at the salad bar and it becomes clear that the entire experience is engineered to encourage you to assemble a healthy salad. Vegetables like peppers and carrots are closest to you; cheese and olives are as far away as possible, forcing you to make an uncomfortable reach to add that tasty fat to your innocent greens. The salad bar is a persuasive technology designed to change your eating habits.

I find this amusing, and vaguely sinister, but end up conceding that it’s probably a good thing, or would be if I ate at Google every day. In the same sense, I wish more websites would take institutionalized serendipity more seriously. Like green peppers, information you didn’t know you needed is good for you, and should be periodically put onto your plate, even if you didn’t request it.

Encountering new ideas isn’t a supply problem in today’s internet – it’s a demand problem. There’s a near infinity of people unlike you creating content and putting it online for you to encounter. But it’s entirely possible that you’ll never encounter it if you don’t actively look for it… or unless the systems you use to find ideas start forcing you outside your usual orbits into new territories. Don’t fear the serendipity.