Update 9:30pm Eastern. Shannon Love at the Chicago Boyz blog called foul on the Lancet 2004 study early on and, with vindication, reacts to David Kane’s new analysis of the 2004 Lancet Iraq death toll study: “Kane shows that if the Falluja cluster is included in the statistical calculations, the confidence interval dips below zero, which is a big no-no. Since the study’s raw data remain a closely guarded secret, Kane cannot be absolutely certain that the inclusion of the Falluja cluster renders the study mathematically invalid…but that’s the way to bet. In science, replication is the iron test. I find it revealing that no other source or study has come close to replicating the original study. All my original points still stand. Ah, vindication is sweet.”
Via Cox & Forkum, a fitting cartoon flashback:
One of the most useful roles of the blogosphere is its service as an open-source intelligence-gathering medium. You can draw on the expertise of people around the world at the touch of a button. We saw this with typography experts during the Rathergate scandal; Photoshop experts during the Reutersgate debacle; and military experts during the Jesse Macbeth unmasking.
Now, it’s the statisticians and math geeks’ turn. Remember that massively-publicized 2004 Lancet Iraq death toll study? It was cited in nearly 100 scholarly journals and reported by news outlets around the world. “100,000 Civilian Deaths Estimated in Iraq” blared the Washington Post in a typical headline.
There were attempts made by lay journalists to debunk the 2004 study (as well as the 2006 follow-up study that purported to back up the first). But none of those dissections comes close to a damning new statistical analysis of the 2004 study authored by David Kane, Institute Fellow at the Institute for Quantitative Social Science at Harvard University. I read of Kane’s new paper at this science blog and e-mailed him for permission to reprint his analysis in its entirety here so that a wider blog readership could have a look. He has given me his permission and adds that he welcomes comments and feedback. He’ll be presenting the paper at the Joint Statistical Meetings in Salt Lake City on Monday — the largest conference of statisticians in North America.
Much of the math here is mind-numbingly complicated, but Kane’s bottom line is simple: the Lancet authors “cannot reject the null hypothesis that mortality in Iraq is unchanged.” Translation: according to Kane, the confidence interval for the Lancet authors’ main finding is wrong. Had the authors calculated the confidence interval correctly, Kane asserts that they would have failed to identify a statistically significant increase in risk of death in Iraq, let alone the widely-reported 98,000 excess civilian deaths.
An interesting side note: as Kane observes in his paper, the Lancet authors “refuse to provide anyone with the underlying data (or even a precise description of the actual methodology).” The researchers did release some high-level summary data in highly aggregated form (see here), but they released neither the detailed interviewee-level data nor the programming code that would be necessary to replicate their results.
Kane has sent his paper to Lancet. But the blogosphere need not wait for Lancet to complete its review. If you’ve got a statistics background or know someone who does, have a look. Kane’s e-mail address is dkane-at-iq.harvard.edu. He’s a blogger himself at EphBlog.
***blog comments powered by Disqus
August 22, 2012 06:38 PM by Michelle Malkin
December 26, 2012 10:39 AM by Michelle Malkin
October 17, 2012 01:48 AM by Michelle Malkin
June 12, 2012 08:42 AM by Michelle Malkin
Federal appeals court rules Obama’s Labor Relations Board non-recess recess appointments were illegal
January 25, 2013 07:42 PM by Doug Powers
Gay Patriot» And still some (journalists) swoon at the mere mention of his name
Green Room» Dude
Gay Patriot» Thoughts on the Boy Scouts & gays
Daily Caller» The former president of France once got attacked by a panda
Green Room» Will new evidence emerge in Manson case?
Gay Patriot» The Slow Death of Irony Continues in Colorado
Daily Caller» Geraldo: Fewer probes of journalists, more probes of ‘radicalized’ mosques, terrorist threats
Daily Caller» Pizza deliverer caught eating toppings off customer’s pizza