Related to my previous post, I followed a link from Chris Blattman’s blog (link) to the Marginal Revolution blog where Alex Tabarrok had posted a great discussion of “false findings” in science (link). The post was from 2005, and it was triggered by John Ioannidis’s now well-known paper on “why most published research findings are false” (link). Tabarrok proposes that economics may be in less worse shape because economics hypotheses tend to be better motivated by theory than the type of atheoretical “see what sticks” hypothesis testing that seems, from a casual glance, to characterize other literatures. I’m not sure how one might assess Tabarrok’s claim on that one, but there is a line in his post that I really, really like: “Trust empirical papers which test other people’s theories more than empirical papers which test the author’s theory.” I think this is spot on. Alas, publishing and presentation standards in my discipline, political science, seem to punish people who take such an approach. Rather, it seems that the expectation is for people to present tests of “their argument” and to use tests to, Tah-dah!, demonstrate the validity of “their argument.” I find it anti-scientific.
We have the same problem in psychology. It’s disturbing how much of modern practice seems to be “anti-scientific.”