{"id":1865,"date":"2015-01-07T12:45:53","date_gmt":"2015-01-07T17:45:53","guid":{"rendered":"https:\/\/cyrussamii.com\/?p=1865"},"modified":"2015-01-07T12:45:53","modified_gmt":"2015-01-07T17:45:53","slug":"can-we-hold-theory-fixed-for-a-minute","status":"publish","type":"post","link":"https:\/\/cyrussamii.com\/?p=1865","title":{"rendered":"Can we hold theory fixed for a minute?"},"content":{"rendered":"

In the current PS, Luke Keele (link<\/a>) makes a great point about the need for journals to let researchers take existing theories and subject them to multiple rounds of empirical scrutiny:<\/p>\n

Of course, there is nothing wrong with new theories;
\nhowever, I believe that an overemphasis on theory can
\nimpede the ability of the discipline to establish causal
\nrelationships. One point of emphasis in the identification
\nrevolution is that causal inference is difficult. A series of
\nregression models is hardly the last word in whether a causal
\ntheory holds. Within causal inference, it is generally understood
\nthat only a series of different studies using disparate research designs can provide solid evidence for a causal
\nrelationship. This understanding implies that, as opposed
\nto a single test, a theory requires a number of tests from
\ndifferent research designs. We might assert that whereas
\ntheory is critically necessary, too many theories might be
\nharmful. It is probably the case that the gain from an incomplete
\ntest of a new theory is less than the gain from a new
\ntest of an existing theory.<\/p>\n

Economics provides a useful example. The question
\nof whether attending college increases income is one of
\nlong-standing interest to economists. The theory behind the
\nquestion is relatively simple. Although I hesitate to state that
\nthe causal hypothesis that college leads to higher incomes is
\ndefinitively settled, a review of the many different research
\ndesigns used makes a convincing case for a causal effect. If all
\nof the researchers conducting these studies had been told by
\nreviewers that new theory was needed, little progress would
\nhave been made.<\/p>\n

When we overvalue novel theories, we tend to dismiss
\nattempts to answer old questions with new research designs.
\nWe can value papers with new theories and little or no empirics.
\nWe also can value papers with little in the way of new theory
\nbut that present novel research designs that provide new
\nempirical evidence about causal relationships. We could argue
\nconvincingly that both topics require such attention that it
\nmay be difficult to present both novel theory and empirics
\nin the same paper. Currently, I would state that, in general,
\na paper that does not engage in new theory development will
\nhave a difficult time being published in a top journal. I don’t
\nthink that is healthy. If we really want researchers to clearly
\nestablish causal relationships that is often worth doing
\nalone in a single paper.\n<\/p><\/blockquote>\n

In my (and Luke’s) home discipline of political science, I have frequently seen reviewers at top journals suggesting rejection for papers because they don’t think the theoretical contribution is novel enough, even if the research design is compelling. (I am not talking about reviews for my own work either, but that of my much more capable colleagues who have discussed their reviews with me.) But journals seem to be okay with papers that propose new theories with no empirical tests whatsoever. Weird, isn’t it?<\/p>\n

I am not saying pure theory papers shouldn’t be published—quite the contrary in fact. What makes sense is some division of labor in the discipline. I recognize the importance of pure theory papers. I also recognize the importance of compelling empirical work that offers a new and credible test of an existing theoretical claim. The usual refrain that I hear when I say this is that “top economics journals don’t seem to have this problem,” and with that I agree. I wonder why there is such a difference.<\/p>\n

I agree with pretty much everything else in Luke’s paper too. It is part of a symposium on whether “big data,” “causal inference,” and “formal theory” are conflicting trends, an obviously ridiculous proposition, but one that triggered some nice essays by the contributors.<\/p>\n","protected":false},"excerpt":{"rendered":"

In the current PS, Luke Keele (link) makes a great point about the need for journals to let researchers take existing theories and subject them to multiple rounds of empirical scrutiny: Of course, there is nothing wrong with new theories; however, I believe that an overemphasis on theory can impede the ability of the discipline … <\/p>\n

Continue reading “Can we hold theory fixed for a minute?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1865","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/1865","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1865"}],"version-history":[{"count":14,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/1865\/revisions"}],"predecessor-version":[{"id":1879,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/1865\/revisions\/1879"}],"wp:attachment":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1865"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1865"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1865"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}