{"id":306,"date":"2010-11-30T13:49:14","date_gmt":"2010-11-30T18:49:14","guid":{"rendered":"https:\/\/cyrussamii.com\/?p=306"},"modified":"2011-01-05T12:30:16","modified_gmt":"2011-01-05T17:30:16","slug":"psi-as-a-teachable-moment-on-methods-and-data-analysis","status":"publish","type":"post","link":"https:\/\/cyrussamii.com\/?p=306","title":{"rendered":"“Psi” as a teachable moment on methods and data analysis"},"content":{"rendered":"

In case you haven’t followed the chatter about Daryl Bem’s forthcoming paper on evidence of “precognition” and “premonition” (a.k.a. “psi” effects, or more colloquially, psychic intelligence), you can read a synopsis at the Freakonomics blog (link<\/a>). \u00a0The comments on the blog page are quite amusing. \u00a0More interesting is how Wagenmakers et al. have leapt on this as a “teachable moment” for discussing perils and pitfalls in commons modes of contemporary data analysis. Their lengthy critique of Bem’s research (link<\/a>) is really interesting. \u00a0They discuss fallacies that arise when confirmatory and exploratory analyses are confused. \u00a0They also discuss how a low probability of the data given the null hypothesis ($latex p(D|H_0)$) does not necessarily translate into good reason to <\/em>discontinue belief in the null in favor of some alternative ($latex H_1$). \u00a0Your willingness to increase your belief in the alternative depends on your priors,<\/p>\n

$latex p(H_1|D) = \\frac{p(D|H_1)p(H_1)}{p(D|H_0)p(H_0)+p(D|H_1)p(H_1)}$.<\/p>\n

So, if your belief in the null is strong, then even strong evidence against the null can do little to change your beliefs. \u00a0As Wagenmakers et al state, “[t]his distinction provides the mathematical basis for Laplace\u2019s Principle that extraordinary claims require extraordinary evidence.” Failure to appreciate this is known as the “fallacy of the transposed conditional,” referring to the fallacious consideration of $latex p(D|H)$ as being essentially equivalent to $latex p(H|D)$.\u00a0\u00a0They also discuss a Bayesian method of hypothesis testing that helps to guard against the problem that, with large data, “everything is significant.” The method has one examine the relative strength of evidence against the null versus evidence against some clearly specified alternative.\u00a0All of this would make a great introductory case study for methods students.<\/p>\n

Update: What Wagenmakers et al call the “fallacy of the transposed conditional” is also the basis of Kahnemann and Tversky’s well-known “representation heuristic” (link<\/a>).<\/p>\n","protected":false},"excerpt":{"rendered":"

In case you haven’t followed the chatter about Daryl Bem’s forthcoming paper on evidence of “precognition” and “premonition” (a.k.a. “psi” effects, or more colloquially, psychic intelligence), you can read a synopsis at the Freakonomics blog (link). \u00a0The comments on the blog page are quite amusing. \u00a0More interesting is how Wagenmakers et al. have leapt on … <\/p>\n

Continue reading ““Psi” as a teachable moment on methods and data analysis”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-306","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/306","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=306"}],"version-history":[{"count":14,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/306\/revisions"}],"predecessor-version":[{"id":407,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/306\/revisions\/407"}],"wp:attachment":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=306"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=306"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=306"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}