Lancet and 100 kilodead

This issue crossed my desk again today, so I thought I'd do a little looking around. (I really need to remark on these items as soon as they come up, so I can find them again.)

(The Lancet article is behind a pay-wall.)

The validity of a number, like 100,000, depends on a couple of things. First, the statistical underpinnings. A sample size that's too small yields an unacceptably large margin of error. In this case, the 95% confidence interval, assuming everything else was done right, is 8,000 to 194,000.

The other factor is methodology. If your sample is not representative of the universe you're trying to draw conclusions about, your numbers are trash. And this may be the case.

There were other problems. The survey team simply could not visit some of the randomly chosen clusters; the roads were blocked off, in some cases by coalition checkpoints. So the team picked other, more accessible areas that had received similar amounts of damage. But it's unclear how they made this calculation. In any case, the detour destroyed the survey's randomness; the results are inherently tainted. In other cases, the team didn't find enough people in a cluster to interview, so they expanded the survey to an adjoining cluster. Again, at that point, the survey was no longer random, and so the results are suspect.

How well do the results in this paper stack up against other data? Well, one problem is mentioned in the Slate article:

Based on their survey of how many people in the sampled households died before the war, they calculated that the mortality rate in prewar Iraq was 5 deaths per 1,000 people per year. The mortality rate after the war started—not including Fallujah—was 7.9 deaths per 1,000 people per year. In short, the risk of death in Iraq since the war is 58 percent higher (7.9 divided by 5 = 1.58) than it was before the war.

Now, the thing is, the headlines scream about 100,000 Iraqis dead since the beginning of the war, implying that all 100,000 deaths are due to the war. In fact, taking these numbers at face value, the increase in deaths is much closer to 37,000.

But there are two problems with this calculation. First, Daponte (who has studied Iraqi population figures for many years) questions the finding that prewar mortality was 5 deaths per 1,000. According to quite comprehensive data collected by the United Nations, Iraq's mortality rate from 1980-85 was 8.1 per 1,000. From 1985-90, the years leading up to the 1991 Gulf War, the rate declined to 6.8 per 1,000. After '91, the numbers are murkier, but clearly they went up. Whatever they were in 2002, they were almost certainly higher than 5 per 1,000. In other words, the wartime mortality rate—if it is 7.9 per 1,000—probably does not exceed the peacetime rate by as much as the Johns Hopkins team assumes.

OK, let's try some other numbers.

Pre-war death rate	"Extra" deaths
6	53,500
6.5	36,400
6.8	27,300
7.0	21,700
7.5	9,000
8.0	-2,100

Between the uncertainty over the pre-war and post-war death rates, and the large margin of error in the final number, the figure of 100,000 dead seems to be close to meaningless.

Further comments here, here, here(arguments about who debunks whom), and here.

Enjoy!

(Um. I see the 98,000 figure was the "excess dead". I've fixed the numbers in my table.)