A little over a year ago, I started posting data I was scraping off different news sites, inviting other researchers who were interested in media attention to play with the data and publish their own conclusions. A year later, someone’s finally taken me up on the offer, in a wonderful and useful way.
I got an email a couple days ago from Michael Adler, a guy I hadn’t had any previous contact with, virtually or otherwise. He was writing to tell me that he’d written a set of scripts that processed my data, put it into a SQL database and used gnuplot to produce an elegant series of time series graphs. Would I like to see the results?
Would I?! Rewriting my code so that I could see results over time has been on my to-do list for about nine months now. I’ve already installed shackles on my desk so I could chain myself until I got the code written. While I’ve been struggling to turn Amazon’s sales rank into actual sales figures (more on that next week, I hope), someone I’d never met was doing my research for me. I love the Internet…
Michael was kind enough to let me mirror his results on my site, and has sent me his scripts, which I’m hoping to ruthlessly plunder and turn into lots of tasty database driven code later this summer. (Perhaps the shackles will still get some use.)
In the meantime, I’m enjoying watching how news media work in parallel on stories like Iraq or how attention slowly builds around stories like the crisis in Sudan. I’m already seeing some evidence that blogs (represented by Daypop stats on Michael’s charts) are moving in lockstep with major media sources. Thanks, Michael, for giving me lots more to think about.