Sunday, January 30, 2005

But how about the science?

Sternberg did publish Meyer's piece in the Proceedings of the Biological Society of Washington. People are in an uproar over the equivalent of astrology being published in that journal. But how is the science?

After citing a 6,000 word review which dissects the bad science in the paper, showing "the big picture view of why it is poor science that shouldn't have made it past any qualified reviewers", P.Z. Myers takes a look at one paragraph to show why (I'll say it – in his opinion) "why it represents poor, biased scholarship."
I'm motivated in part by a ridiculous critique from Joe Carter. One of the things he does (in his second point, if you bother to read it) is a practice creationist pseudoscientists are getting very good at, and that Meyer also practices in his paper: throwing a bunch of scientific references at the reader that, in Carter's case, the creationist has never read, or in Meyer's case, may have read but misrepresents. How many people would bother to check that these esoteric references are being reported accurately? How many of us who actually are comfortable with the scientific literature have the time to cross-check and report all of the misrepresentations being made? I sure don't. That's why I'm just going to pick on one paragraph. <snip> The idea behind scientific citations is that they should be papers supporting the ideas being discussed, and are shortcuts for the author—instead of tediously enumerating all the evidence to support a claim, he points the reader to another source that documents it. There is a bit of trust involved; a scientific paper may easily throw 50 references at the reader, and it's a difficult chore to check them all. (This, by the way, is one reason peer-review is supposed to be done by individuals qualified in the field; they are likely to have already read many of the cited papers, are more or less familiar with their contents, and reviewing one paper doesn't necessarily involve scurrying to the library and reading 50 more papers to see if they were correctly represented.)

So how well do the citations check out? Not very.

Now you might be able to see what a qualified reviewer would see when reading the Meyer paper. It's full of these peculiar disconnects from the reality of the scientific literature—he's constantly citing little fragments of papers while ignoring the bulk of the work. It's a more rarefied version of more typical creationist quote mining, made slightly more sophisticated and much more difficult to check, and designed to wow the rubes rather than persuade anyone knowledgeable in the subject.

I'm willing to believe Sternberg has been thrown out of his office, and that his job may be in jeopardy. But I suspect demonstrated incompetence is probably a stronger motive than any sort of "witch hunt".

No comments: