Locke's Madman

In almost every issue of the authoritative magazine Science of the American Association for the Advancement+ of Science, you’ll find examples of these biases in one or more of the reports.

The line drawn through the data points is sheer madness, reminiscent of the Texas Sharpshooter Fallacy (from a story about a Texan who fires several shots at the side of a barn, paints a target centered on the hits and claims to be a sharpshooter). The authors pay lip service to the tenuousness of their premise and then heedlessly forge ahead to their conclusions. It’s true that different fields of science allow for different standards of evidence, but about 4-5 months later another report appeared in Science that lifted the conclusions from this first report and stated them as fact, omitting the original caveats. The madness is spreading.

I’m often tempted to trace a dartboard and see what researchers make of the dots on the graph.

So if we start our research initiative based on one of these peer-reviewed research articles (or its predecessor in one of the more technical journals) then we've lost the game from the very beginning.

Of course, if we're interested in 'controlling the narrative' around a particular topic, for example the best surrogate measure for XYZ disease, then we need to publish as many of these studies as budget will allow. Later authors will not have the time nor budget to dig too deeply into the data and will cite our findings as though they were accepted fact. Individuals will mistake quantity of (flawed) studies for quality of overall direction. See Lousy Studies.

Home Page April 2010