Take a breath

Steve Milloy, owner of the Junk Science Page, looks at the report, in JAMA, linking smog levels with an increased death rate.

He does not look very favorably.

But had the reporter been able to go beyond simple regurgitation of the study’s press release, Reuters’ might well have reported “Researchers tried to scare public with statistical malpractice.” The researchers compared the non-injury-related death rates and smog measurements for 95 urban areas for the period 1987-2000. They reported a one-half percent (0.5 percent) increase in premature death (mortality) per 10-part per billion increase in ground-level ozone (smog) in the urban areas. Reducing smog levels by 35 percent, they claim, could save about 4,000 lives per year. <snip> First, if smog is deadly in New York City, then it should be deadly everywhere. But even granting the researchers every benefit of the doubt with respect to the validity of their analysis, among the 95 urban areas included in the study, the correlation between smog and mortality is only statistically meaningful in five of those 95 urban areas (New York City, Newark, Philadelphia, Dallas-Ft. Worth and Chicago). That means in 95 percent of the urban areas studied, there was no meaningful correlation between smog and mortality.

I don't know what significance level the researchers used in their study. The usual cut-off is 0.05, meaning that, if there were no link at all, the observed result would have happened due to chance in 5% of observations by pure chance. That means that if you looked at correlations between smog levels and death rates in 100 cities, you'd expect to find five observed correlations that made it to that level of significance.

Five out of a hundred is 0.05, and five out of 95 is 0.0526 – a difference of only only five percent. In a sample of 95 cities, we should not be surprised to see five cities show significant numbers for the correlation between smog levels and death rates, even if there's no link at all.

Now, to be sure, high levels of smog are harmful, and smog at any level can be unpleasant. But how many hundreds of billions of dollars should we spend on something that may have no measurable effect? If you want to improve public health, there are lots of ways to spend the any given hundred billion dollars that would give you far more bang for the gigabuck.