Mother Jones Claims the Rate of Mass Shootings Has Tripled Since 2011. Is That True? - Hit & Run : Reason.com

Mother Jones Claims the Rate of Mass Shootings Has Tripled Since 2011. Is That True? - Hit & Run : Reason.com

The best critique I've seen of the original Mother Jones article was written by Michael Siegel. You should read the whole thing, but this is the key passage:

It is a truism of science that the more narrowly you define your sample and the more you shrink the number of data points, the less reliable your conclusions will be. If you were to analyze all gun shootings and violence over the last thirty years, you'd have hundreds of thousands of data points to base your conclusions on. You could, as I like to say, achieve Victory Through Sheer Data Volume. But when you start parsing the data down further and further, you become more prone to random variation and even bias.

Even if we take Mother Jones' data at face value, we can see we're dealing with less than 120 victims every year and frequently less than 20. That's an awfully small number to be drawing conclusions from. To illustrate why, take the Virginia Tech killings. 56 people were killed or wounded. That is more than all but five entire years in their database. Something like that is simply going to swamp the statistics.

But we shouldn't even take Mother Jones' data at face value because it is highly suspect. First, it seems to be based on media coverage, which is not exactly an objective source and almost certainly leaves shootings out....Everywhere, they make arbitrary cuts to exclude murders that may not fit their conclusions. They limit the sample to lone shooters, but make exceptions for Columbine and Westside. They exclude gang activity and other crimes but include the Fort Hood Shootings, which were an act of terrorism....They arbitrarily throw in a few spree killings.

This is simply not a representative sample. It's cherry-picked to fit a definition, but leaves huge gaping biases all over the place. Mother Jones doesn't even acknowledge this.

All this would be fine if you wanted to create an illustrative or representative sample. This is even fine if you want to draw some broad and overwhelming conclusions such as that most spree killers get their guns legally. But the low numbers and the biases blow up in your face when you try to do a more rigorous analysis....They've narrowed the sample so far down that they are essentially looking at noise.

The new analysis looks at the intervals between each incident rather than the annual numbers of crimes and victims. But aside from the fact that the list has been updated through 2014, this is the same data as the original article, with all the problems that Siegel and others pointed out before.

The best alternative measurement that I'm aware of comes from Grant Duwe, a criminologist at the Minnesota Department of Corrections. His definition of mass publicshootings does not make the various one-time exceptions and other jerry-riggings that Siegel criticizes in the Mother Jones list; he simply keeps track of mass shootings that took place in public and were not a byproduct of some other crime, such as a robbery. And rather than beginning with a search of news accounts, with all the gaps and distortions that entails, he starts with the FBI's Supplementary Homicide Reports to find out when and where mass killings happened, then looks for news reports to fill in the details. According to Duwe, the annual number of mass public shootings declined from 1999 to 2011, spiked in 2012, then regressed to the mean.

Finally, a note on why this matters. Violent crime rates have been moving downward for decades now, and mass shootings—by any definition—are a very rare phenomenon. I've heard arguments from one direction that say there's no point in putting such a small risk under a microscope when the most pressing threats to people's lives lie elsewhere. I've heard arguments from another direction that say even one crime this horrible is too many, and that the effect of noting how infrequently it happens is just to discourage people from trying to prevent it.

To the first set of arguments, I say that when the press and politicians present a problem like this as a rising crisis, it's worthwhile to see whether it is indeed rising. To the second set of arguments, I say that absolutely nothing I've said here means we shouldn't try to prevent future mass murders. Plane crashes are extremely rare, but airlines still look for ways to make them even less likely. If a measure genuinely makes people safer without creating an intolerable trade-off, I'm for it.

Such measures are most likely to be incremental changes adopted at particular places (such as schools) and then imitated elsewhere, not big anti-crime bills rushed into law by national politicians eager to be seen Doing Something. But there may well be ways federal or state officials can make that experimentation and imitation easier. Good ideas are good—and bad ideas are bad—whether or not mass shootings are getting more common.