An Ill-Conceived Health-Care Ranking - WSJ.com

The Numbers Guy at the Wall Street Journal looks at the story behind the US Health Care system's 37^th place ranking in An Ill-Conceived Health-Care Ranking - WSJ.com

Few people who cite the ranking are aware that some public-health officials were skeptical of the report from the outset. The ranking was faulted because it judges health-care systems for problems -- cultural, behavioral, economic -- that aren't controlled by health care.

"It's a very notorious ranking," says Mark Pearson, head of health for the Organization for Economic Cooperation and Development, the 30-member, Paris-based organization of the world's largest economies. "Health analysts don't like to talk about it in polite company. It's one of those things that we wish would go away."

So what's wrong with it?

The WHO ranking was ambitious in its scope, grading each nation's health care on five factors. Two of these were relatively uncontroversial: health level, which is roughly the average healthy lifespan of a nation's residents; and responsiveness, which is a sort of customer-service rating encompassing factors such as the system's speed, choice and quality of amenities. The other three measure inequality in health-care outcomes; responsiveness; and individual spending.

These last three measures struck some analysts as problematic, because a country with unhealthy people could rank above a healthier one where there was a bigger gap between healthy and unhealthy people. It is certainly possible that spreading health care as evenly as possible makes a society healthier, but the rankings struck some health-care researchers as assuming that, rather than demonstrating it.

An even bigger problem was shared by all five of these factors: The underlying data about each nation generally weren't available. So WHO researchers calculated the relationship between those factors and other, available numbers, such as literacy rates and income inequality. Such measures, they argued, were linked closely to health in those countries where fuller health data were available. Even though there was no way to be sure that link held in other countries, they used these literacy and income data to estimate health performance.

Philip Musgrove, the editor-in-chief of the WHO report that accompanied the rankings, calls the figures that resulted from this step "so many made-up numbers," and the result a "nonsense ranking." Dr. Musgrove, an economist who is now deputy editor of the journal Health Affairs, says he was hired to edit the report's text but didn't fully understand the methodology until after the report was released. After he left the WHO, he wrote an article in 2003 for the medical journal Lancet criticizing the rankings as "meaningless."