“Simple statistics” and the blogging of humanitarian disasters

I was talking with a friend on the phone the other day, and he described some of my work as being about “very simple statistics”. I briefly took offense – while I’m not a trained social scientist, I sometimes use reasonably sophisticated statistical methods, and I try hard to make sure my work is statistically accurate and rigorous.

And then I realized that the few times my research has captured people’s interest – as with the recent interest in LpkC – it’s been because I’ve thrown out a very simple statistical observation, rather than a complex multivariable model. LpkC makes sense to people because they can do the math in their heads – GAP has largely failed to capture people’s imagination, I suspect, because it requires people to understand logarithmic regression. Perhaps my profound ignorance of advanced statistics is a feature, not a bug.

I’m getting ready for a speech in Chiba, Japan next week, at a workshop preceding the 14th International WWW Conference – I get to lead off a day-long workshop on “the Weblogging Ecosystem”. (If any of my readers are in Tokyo or Chiba, please email me at ethanzATgmailDOTcom – I’ll be in town from the 8th – 11th.) Looking at some of the paper topics submitted, I’m realizing that the gathering is going to be pretty long on sophisticated mathematical analysis. And so I’m redoubling my efforts to frame simple statistics on the topics I’m interested in: how old and new media pay attention to international news.

AlertNet (a humanitarian news service sponsored by the Reuters Foundation) and Factiva released a study in March that showed media attention to the December 26th tsunami “crowding out” other major humanitarian emergencies. They surveyed a set of relief professionals and asked them to name their top underreported humanitarian emergencies, then searched 200 english-language newspapers for stories about these crises and about the tsunami for six weeks after December 26th. While they found roughly 35,000 tsunami stories, they found 34,000 stories on the other ten crises, combined… over the past previous year.

While the study was attention-grabbing, it’s hardly a surprise for anyone who read Galtung and Ruge’s seminal 1965 paper, “The Structure of Foreign News”. The tsunami story featured many of the news factors Galtung and Ruge identified that make a story likely to gain significant attention: brief time duration, unexpectedness, an affect on wealthy people as well as poor people. Many of the stories Alertnet is studying are long stories – there’s no sudden development in the story of AIDS in Africa that makes for “news”, just an ongoing, predictable un-newsworthy crisis that affects poor people.

The question I’ve been trying to answer with my research these past two years – are the problems of media attention identified by Galtung and Ruge forty years ago getting any better now that bloggers are part of the media picture? I, and other cyberoptimists, like to believe that the ability of anyone to participate in the making of the news will help news “get better”.

But if “get better” means “pay more attention to stories the mainstream media ignores”, there’s not much evidence that this is happening. I repeated the AlertNet study using information from Blogpulse, checking to see how the eleven stories in question were covered by bloggers. (It’s impossible for me to accurately repeat the study without knowing the exact keywords the AlertNet study used – I haven’t been able to find a published paper listing those keywords, and AlertNet hasn’t yet been able to point me in the right direction. So I’m guessing. But I’ll fully disclose the bias of those guesses.)

Searching Blogpulse for mentions of tsunami from March 1, 2004 through February 28, 2005, I find 191,707 appearances. A quick examination of results suggests that at least one in ten is not about the December 26th tsunami. So I’ve ended up using a more restrictive boolean search – “tsunami AND (relief OR india OR indian OR “boxing day” OR 26 OR Indonesia OR “Sri Lanka” OR Thailand OR Aceh OR bangladesh OR disaster OR assistance”) – which yields 96,962 results.

Using searches designed to be generous (“uganda OR LRA or Lord’s Resistance Army” to match stories about the conflict in Northern Uganda, realizing that this is going to match stories about Uganda/Kenya soccer matches), I’ve checked the ten undercovered stories identified by AlertNet and found, collectively, 75,261 blog references over the past year… or 78% as many as the narrow search for tsunami references. In other words, even after I’ve stacked the deck to make bloggers look good, bloggers paid less attention to these undercovered stories – in comparison to attention paid to the tsunami – than mainstream media outlets did. Using the simpler search for tsunami as the denominator, and blogs wrote 39% as many stories about these ten undercovered stories than about the tsunami.

One possible explanation for these results that bloggers covered the tsunami with an intensity that dwarfed the mainstream media, and that blogger coverage of the 10 “ignored” stories looks paltry only in comparison to their massive coverage of the tsunami. And there’s some evidence for this argument.

How much did mainstream media report on the tsunami in relation to other major stories from March 2004 – February 2005? One way to measure this is to take an arbitrary, popular search term – “Iraq”, in this case – and see how other search terms compare to that base term. For the New York Times in the period we’re considering, “Iran” gets 20.53% as many search hits as “Iraq” (1160/5649). The same pair of terms on Blogpulse turns up a 20.19% comparison (66314/328470), a remarkably similar distribution. A few other terms have similar ratios – “frist” (as in “Senator Bill”) has a 3.8% “iraq ratio” on the NYT, and a 5% ratio on Blogpulse. Kerry (as in “Senator John”) gets 102.2% on NYT and 78.1% on Blogpulse.

But the ratios aren’t always that close, even on apparently similar stories. Afghanistan gets a 32.9% “iraq ratio” on the NYTimes, while registering a 15.1% on Blogpulse. Israel shows a similar disparity, with 38.7% on the Times and 25.1% on Blogpulse. (Just to be very clear – there’s no reason to assume that bloggers and the New York Times pay the same amount of attention to Iraq. Lacking a good way to compare the two, I’m grabbing an arbitrary reference point and doing comparisons to that arbitrary point. If bloggers and the Times cover Iraq to radically different degrees, it likely makes the comparisons I’m making here meaningless…)

And here’s where it gets weird. A search for “tsunami” on the New York Times in the yearlong interval we’re considering gets 503 matches, or 8.9% as many matches as searches for “Iraq”. The same search on Blogpulse yields 191,707 matches, or 58.3% as many mentions as “Iraq”. Even using the “adjusted” (narrower, boolean) search for “tsunami” on Blogpulse, we get 96,962 results, or 29.5% as many matches as searches for “Iraq”. If we use Iraq as our baseline, bloggers talked about the tsunami a great deal more than the New York Times did.

Alas, bloggers interest in other humanitarian stories doesn’t appear to follow the same pattern. While the Times covered Darfur heavily – 5% of the coverage that mentions Iraq – bloggers covered it with 2.1% as many mentions as Iraq. While Uganda has similar ratios in the two sets (1.8% of Iraq in the Times, 1.2% in Blogpulse), Haiti’s ratios are quite far apart – 6.6% in the Times versus 1.8% in Blogpulse.

Bloggers, as a whole, undifferentiated group (Blogpulse currently tracks 10.5 million blogs, most of which are personal journals, rather than citizens’ journalism) seem to pay less attention to the developing world than mainstream media sources. The map below is a graphic illustration of this:

This map compares media attention on Blogpulse and Google News over the past 14 days. Countries colored in red were represented more strongly in Blogpulse than in Google News – in other words, stories on Portugal represented a larger percentage of all stories on Blogpulse than they did on Google News. For the most part, bloggers paid more attention to Canada, Mongolia, Egypt, Turkey, Jordan, Iceland and a few southeast Asian nations (Vietnam, Laos, Cambodia) than the news sources tracked by Google News did. (Mali, unfortunately, isn’t getting lots of blog traffic. Instead, lots of bloggers mistype the word “mail”. The same thing tends to happen with “Togo”.) Google News is paying more attention to most of Africa, Central Asia, Eastern Europe, Indonesia and some of Central America.

Given bloggers’ tendency to report on developing nations less than mainstream news sources, and the NY Times/Blogpulse comparisons using Iraq as a baseline, I’m tempted to conclude that bloggers reported on the AlertNet 10 under-reported stories no more frequently than mainstream papers. But it looks like bloggers seized the tsunami issue with passion, mentioning the tsunami, tsunami relief concerts and aid efforts to a greater degree than mainstream news sources.

This is heartening news for those of us who see blogs as a way to “hack” the media. As blogs become increasingly important to media professionals, breaking new stories and reinforcing others, blogs are serving as a feedback mechanism, letting professional media sources know what stories audiences are interested in and willing to hear more about. For groups that want to increase attention to “forgotten stories”, it may make sense to try to get bloggers to write and talk about these issues in the hopes of conveying reader interest to mainstream, professional news sources.

Why did the tsunami capture blogger attention to such a great degree? That’s a whole other blog post, and I need to pack for my trip to Japan. But I suspect that the number of personal narratives – by people affected by the disaster, people writing personal rememberances of friends lost, and people writing about their experiences doing relief work – resonated with bloggers to a degree that “straight news” stories rarely do. This explanation is an extension of the explanation I’ve been offering for the popularity of the Christian Science Monitor with bloggers – bloggers are story-tellers and like news that tells stories.

As always, this is research in progress and I’m not sure my conclusions are the right ones. Please feel free to set me straight, or join me in the process of figuring these results out.