Cass Sunstein’s “Infotopia”

Cass Sunstein’s book, Republic.com, published in 2001, offered a useful challenge to some of the cyber-utopian promises celebrated at the turn of the millenium. Nicholas Negroponte, in “Being Digital“, offered the fond hope that future news readers would consult “The Daily Me”, a customized set of news items designed to meet their specific tastes. Sunstein seizes on this possibility and offers a strong caution: if we can choose our own media, it’s possible we will use this power to insulate ourselves in an information cocoon, where we systematically avoid dissenting voices and have increasingly less common experience with our fellow citizens. Sunstein worries that a society of these isolated individuals will have difficulty participating in a democracy because citizens need a) some exposure to materials they would not have sought out and b) some common experience as a precursor for joint decisionmaking.

Republic.com sparked fierce debate amongst scholars of the Internet. Many – myself included – felt that Sunstein’s predictions were somewhat alarmist. While it was certainly possible to use the Internet to isolate yourself, many net users were finding themselves reading media sources they’d never encountered before: newspapers and broadcast media from all around the world as well as the voices of individual bloggers and authors. At the same time, memes swept the net, reaching large portions of internet readers, providing a new (if often very goofy) form of common experience for many users. Sure, we weren’t all watching Walter Cronkite each night, but we hadn’t become radically isolated cocoon-dwellers either.

Adamic and Glance’s visualization of linking patterns in top American political blogs leading up to the 2004 election.

Reacting in part to Sunstein’s claims, a number of researchers looked closely at issues of political and ideological polarization in political blogs. Lada Adamic and Natalie Glance released an excellent paper and visualization (above) in 2005 subtitled, “Divided they Blog”, which suggested that top political bloggers were far more likely to link to bloggers with similar ideologies than to cross ideological lines. On the other hand, there some significant linking across ideological lines, suggesting that even in a circumstance where bloggers have full control over the media they create, they will, in some cases, point to ideologically divergent views. Eszter Hargittai conducted a similar experiment some months later and discovered that in many cases, the links across ideological lines were cases where the other points of view were cited dismissively.

Half a decade after Republic.com, Sunstein is still concerned with the formation of ideological cocoons. In his new book, Infotopia, he’s become a cyber-enthusiast to an extent that would have been hard to imagine a few years ago. Specifically, he’s excited about the ways new online tools make it possible for groups of people to assemble information and accumulate knowledge. He’s become a devotee of Friedrich Hayek, the Austrian economist who saw markets, first and foremost, as a way to aggregate information held by a large group of people. There’s ample evidence that Hayek was right in an examination of the failure of planned economies – smart men sitting in a room do a far worse job of setting the price of copper ore or bread than the collected actions of thousands of consumers, iterated over time.

Markets aren’t the only way to aggregate information from a large group of people. Deliberative groups, where a set of people get together and share the knowledge they have on a problem or an issue, are favored by many political theorists, including Jürgen Habermas, who bases much of his political philosophy on the establishment of a public sphere where deliberation can occur. Sunstein is deeply suspicious of the optimistic claims made for deliberation, and cites a wealth of studies that demonstrate that deliberation, in many cases, leads to bad decisions and the reinforcement of extreme views.

(You can think of Infotopia as a caged deathmatch between Hayek and Habermas, streamed live on the Internet. Habermas taps out somewhere around page 200.)

Inspired in part by James Surowiecki’s “The Wisdom of Crowds”, Sunstein looks at the uncanny ability of large groups to jointly guess the number of beans in a jar. While individual guesses can be far from the mark, the mean guess is often surprisingly close to the actual number. This phenomenon is well explained by the Condorcet Jury Theorem, which suggests that a group of individuals – all of which have a better than even chance of making a correct decision – will have a greater chance of making the correct decision as the size of the group increases. In other words, if we’re pretty good at guessing the number of beans in a jar, lots of us working together are likely to be excellent at the same task.

Unfortunately, the converse is true as well. If each of us have less than a 50% chance of being right about a decision, a group of us will be worse at making a correct decision, with our probability of accuracy increasing towards zero as the size of the group increases. Ignorance can lower our chances of making an accurate decision, but so can political bias and preconception.

One would hope that deliberation could solve this problem – if a small number of people in a group are knowledgeable about a subject, perhaps they can convince the others of the accuracy of their claims and move the group to a result better than the mean of all their preconceptions. This turns out to be true for at least one set of problems: “eureka problems”, where the correctness of a solution is obvious to everyone once it’s been articulated. (Most chess problems, for instance, are “eureka” problems – they end in checkmate, which any player can see once it’s been diagrammed.)

But deliberation on other types of problems has a more complex outcome. Sunstein organized an experiment in deliberation with David Schkade (of the University of San Diego) and Reid Hastie (of the University of Chicago). The professors invited a set of Colorado citizens from two communities – liberal Boulder and conversative Colorado Springs – to come to local universities and deliberate three divisive political topics: global warming, affirmative action and civil rights. The groups – 5-7 randomly selected citizens from the same community – had a strong tendency to become more politically polarized over the course of the discussion. Liberals became more liberal, conservatives more conservative, and the range of ideological diversity in each group decreased.

Explaining the finding, Sunstein offers a number of explanations, each backed up by other research studies. In a group setting, people will often gravitate towards a strongly stated opinion, especially if their own opinions aren’t fully formed. An ideologically coherent group is likely to repeat a great deal of evidence for one side of an issue, giving more reinforcement for that viewpoint. People find it difficult to defy the will of a group, and may polarize to avoid interpersonal conflict.

Sunstein makes a great deal of this finding in the book, though the paper he and colleagues authored suggests that the constraints of the experiment need to be considered very carefully. The groups the researchers constructed were fairly ideologically homogenous, and no attempt was made to give the deliberating groups any information that might challenge their pre-established points of view. Experiments in deliberation which forced ideological mixing and gave both sides information to present their case had very different outcomes – Sunstein points to some of the work done by James Fishkin which finds some reasons for optimism in more guided deliberation. Sunstein and his co-authors argue that the geographic – and hence ideological – filtering they use in the experiment is a very realistic one in modern-day America, where individuals are increasingly choosing to live in communities where their neighbors share their ideologies. Still, it’s a disappointment that Sunstein doesn’t address some of the possible upsides of deliberation more closely.

Moving beyond the deliberation experiment, Sunstein unpacks some notable examples of groupthink in recent American history – the conclusion that there were WMDs in Iraq, the decision that led to the Challenger disaster – and argues that in each case, there was sufficient information in the deliberating group to make the factually correct decision. The social dynamics of deliberation – specifically the desire not to be seen as the lone dissenting voice in the room – prevented the information from being accurately aggregated.

The implications of this argument are quite dramatic – Sunstein argues that the American vision of democracy is based on deliberation. Representatives are not required to unflaggingly vote the way their constituents would poll on an issue specifically so that deliberation in Congress can change opinions. Much of the importance of a free press is the way the press creates a space in which deliberation can occur. If the deliberative process is broken due to ideological cocooning, this could be a serious problem for the future of the republic.

(Reading between the lines, there’s a strong critique of ideological cocooning in the current US administration in Infotopia, though Sunstein leaves it as an exercise for the reader to apply his ideas on cocooning to President Bush and his advisors…)

While pessimistic about using deliberation to accumulate information, Sunstein is very enthusiastic about the powers of prediction markets. He looks closely at the Iowa Electronic Market, where users bet real money on the outcome of local and national elections. The results of the market are consistently better than the results of opinion polls, a finding that’s held true over many years of experimentation. Sunstein features other prediction markets, used in major companies to predict the dates of product launches, or the abandoned Policy Analysis Market, which planned on letting individuals around the world predict political changes, terrorist attacks and other events important to global governments.

Sunstein reads Hayek through Condorcet to explain why prediction markets work so well. When people are putting money on the line – real, or imaginary – they’re more likely to be right than when they’re guessing at beans in a jar. In sufficiently flexible markets, a small number of well-informed actors can steer prices in the right direction (often making money in the process from the less well-informed). And because market participation is generally non-deliberative, the social factors Sunstein identifies as destructive to deliberation don’t come into play.

Markets aren’t the solution to every problem – it’s hard to use a price system to figure out policy solutions for Iraq, for instance. And Sunstein cautions that Hayek is overly optimistic about the fallibility of markets – bubbles and manipulation can lead to situations where markets give incorrect information. But the ability of markets to get a diverse group of people to share information – which Sunstein feels deliberation too often fails at – is the gold standard Sunstein measures other Internet phenomena against.

Using this criterion, Sunstein is optimistic about the potential of wikis in general and Wikipedia specifically. Part of this enthusiasm comes from Jimmy Wales’s embrace of Hayek – Sunstein quotes Wales as declaring, “Possibly one can understand Wikipedia without understanding Hayek… But one can’t understand my ideas about Wikipedia without understanding Hayek.” As in a Hayekian marketplace, Wikipedians add small bits of knowledge, accumulating them towards a shared goal of accurate, neutral encyclopedia articles. Wikis don’t have price signals, though, which means there’s no way of ensuring authors put they money where their mouths are. If I support a blatantly false proposition in a prediction market, I lose money – in Wikipedia, I might lose status, or I might not – the consequences may be lower. Also, in wikis, the final author has a great deal more power than a single buyer in a market. These cautions aside, Sunstein is a fan.

He’s a fan of open source software as well, drawing some of his inspiration from Eric Raymond’s “The Cathedral and the Bazaar“. In economic terms, the planned economies Hayek critiqued so fiercely are cathedrals – bazaars are, of course, free markets. Sunstein – accurately – recognizes that open source projects are hardly as anarchic and free as the bazaar. Most major F/OSS projects have strict hierarchies about who gets to work on core code. But because most projects are open to outside input, at least through bug reports, they serve as aggregators of dispersed information and satisfy Sunstein’s criterion.

Blogs, on the other hand, do not. Sunstein reacts strongly to “blog triumphalism”, and specifically to an assertion by court of appeals Judge Richard Posner (an excellent blogger on the Becker-Posner blog) that, “Blogging is… a fresh and striking exemplification of Friedrich Hayek’s thesis that knowledge is widely distributed among people and that the challenge to society is to create mechanisms for pooling that knowledge.” Blogs, Sunstein argues, fall far short of this promise – instead, they fall victim to many of the same deliberative traps he’s previously identified. “Even the best blogs lack anything like prepublication peer review, and their speed and informality often ensure glibness, superficiality, confusion, and blatant errors. Many blogs in law and politics are close to talk radio, or to brisk and irresponsible conversations over the lunch table.”

While one is tempted to argue with Sunstein at least on the point of “speed and informality” now that I’m several hours into this blogpost(!), I’d prefer to concede Sunstein’s entire argument and point out, instead, that most bloggers wouldn’t make a claim as strong as Posner’s for their online writing. Unlike Wikipedians, who are working towards the common goal of a free, fair and accurate encyclopedia, or F/OSS authors, who are working towards the common goal of a free, functioning piece of software, bloggers usually don’t have a common goal. Holding them to the standard of Hayekian aggregation of accurate information is like criticizing a football team for failing to produce a grand ballet. Yes, some writers have described football as balletic, but your average offensive tackle is trying to pancake a defensive lineman into the ground, not create high art. Your average blogger is trying to express her personal or political opinions, not further the advance of global knowledge.

Blogs make Sunstein nervous because they’re often the sort of strident, biased voices he worried about in Republic.com. Reading DailyKos or Little Green Footballs, it’s not hard to see how some blog communities can look like echo chambers and information cocoons. As one of the founders of a blog community designed to expose readers to perspectives they might not otherwise have seen, I find myself wondering how Sunstein’s reaction to blogging might change had he not focused on the US political blogosphere. I also wonder if a focus on aggregators of blogs, rather than on individual blogs themselves, might have changed Sunstein’s perspective – in all other cases, the phenomena he considers are collective phenomena, while most weblogs are not.

Whether or not I agree with all of Sunstein’s conclusions, his quest for systems that aggregate knowledge across networks is an exciting way to look at the contemporary Internet. A large number of the most interesting projects taking place on the Internet use strategies to aggregate information from multiple users to create new knowledge – this is the magic behind Google’s PageRank algorithm, Digg’s headlines and Amazon’s collaborative filtering recommendations. Analyzing these systems in terms of their effectiveness in getting people to reveal hidden knowledge is, in my opinion, an excellent framework for evaluation. (I’m very interested, for instance, in thinking through how the folksonomy and taxonomy systems David Weinberger is exploring in his forthcoming “Everything Is Miscellaneous” use different mechanisms to assemble information from different actors to organize information.)

It’s also useful to confront Sunstein’s fear of information cocoons again, five years later. Sunstein’s examples of cocooning are interpersonal ones in this book, governments and firms that manage themselves in ways to avoid confronting uncomfortable truths, as opposed to individuals burying themselves in sympathetic media. But media cocooning is a problem for individuals as well, consumers of online and offline media. I suspect it’s possible to use some of the Hayekian thinking about collecting diverse information to create media aggregators capable of breaking cocoons and exposing people to views and perspectives they might otherwise have missed.

Jed Purdy has an excellent review of Infotopia in The New American Prospect, which also looks at Yochai Benkler’s “The Wealth of Networks”.