What if search drove newspapers?

Zoe Fraade-Blanar presented a wonderful piece of work as her MFA thesis project for NYU’s Interactive Telecommunications Program. “Current” is a Java application designed to sit on the desktop of a journalist and monitor trending topics on Google and the appearance of those topics within Google News. The application looks for stories that have widespread reader interest (i.e., they are trending on Google Trends) and little press coverage – these, Zoe theorizes, are the stories most profitable for news organizations to cover.

Zoe had invited me to the critique that precedes the final show of these projects. I happened to be in NYC that day, so I went. I had mixed emotions about her piece – on the one hand, she appeared to have reduced the journalistic filtering process to its basest form, connecting supply and demand in a way that reminded me of Demand Media. In my comments on her piece, I wondered whether she was making a political statement, or whether this was a serious tool. She explained to me – and later to Brooke Gladstone at On the Media – that the tool was designed to let serious media outlets cover the stuff they needed to pay the bills so they could cross-subsidize serious investigative reporting.

A screenshot of Current in action

On the other hand, the visualization was just so damned lush. And it was such a clever way to put two sources of data together. When we got through the first round of presentations and took a coffee break, I asked her if she’d be willing to work with Hal, David and me on Media Cloud at Harvard… and I’m very happy that she said yes.

While I thought Current was designed as much as a provocation as a tool, it turns out that major media companies just see Zoe’s idea as the path forward. The New York Times ran an article this weekend about Yahoo’s new media property, The Upshot, which uses search data to determine what readers are interested in and deploys a team of two editors and six bloggers to create content to meet the readers’ interests. In other words, instead of asking Zoe if she was trying to make a political point, I should have been asking her if she’d filed for a patent.

Search-driven media strikes fear into the heart of many journalism watchers. My friend Dan Gillmor gave a talk at South by Southwest titled “Are Content Farms Good or Evil? Yes.” On the good side, they actually pay writers, and they listen to their readers (which, he argues, serious newspapers sometimes for get to do.) On the evil side, they’re damned cynical and are likely flooding search engines with mediocre but highly optimized content, perhaps drowning out other, higher quality content.

I’d propose another way in which search-driven content creation might be evil – it’s a step towards news outlet as search engine and away from news outlet as source of serendipity.

The front page of a newspaper is a statement not just about what’s happened in the world in the previous 24 hours, but what the editor believes is important for you to know about. There’s always more that happens in the world that can fit on a paper page – or even a much larger web page – and the editorial decisions made shape a vision of what you need to know as a reader and what you can safely ignore. Smart editors use this ability to engineer serendipity, pushing readers towards topics they might not have known they were interested in, featuring more obscure content that’s got good storytelling and a high likelihood of capturing a (previously uninterested) reader’s interest. (I wrote about this idea at more length in a post called The Architecture of Serendipity.)

While there’s something appealingly populist about the idea of building a media property around what people are searching for – give the people what they want! – it would likely be a bad way to run a newspaper. You’d overfocus on topics people “knew” were important and miss emerging stories. And you’d give up the critical ability to push topics and parts of the world that readers might not be interested in, but need to know about to be an engaged, informed citizen.

My concern is this – we’ve got great tools to help us find what we’re interested in online – search engines. We’re building strong tools to let us see what our friends and people who share our interests are interested in – Twitter, Facebook, Reddit, Digg. Who’s building tools to help us encounter stories we didn’t know we were interested in, and which our friends haven’t already found? Who’s building online tools that go beyond search and social towards serendipity?