{"id":3154,"date":"2020-04-21T16:43:11","date_gmt":"2020-04-21T20:43:11","guid":{"rendered":"https:\/\/cyrussamii.com\/?p=3154"},"modified":"2020-04-21T16:43:16","modified_gmt":"2020-04-21T20:43:16","slug":"using-pre-analysis-plans-to-learn-better-and-to-learn-together","status":"publish","type":"post","link":"https:\/\/cyrussamii.com\/?p=3154","title":{"rendered":"Using pre-analysis plans to learn better and to learn together"},"content":{"rendered":"<p>Below is a Twitter thread in which I offer a perspective from my experience through EGAP (<a href=\"http:\/\/egap.org\/\">egap.org<\/a>) on how to make effective use of pre-analysis plans and also research designs.  The basic idea is that your research design and pre-analysis plan should serve as the basis of a <i>discussion<\/i> in which you can refine your design and analysis and gain buy-in from skeptics.  A research design or pre-analysis plan that is never discussed publicly before it is implemented is a huge missed opportunity.<\/p>\n<p>The thread was in response to a paper by Duflo et al. (linked in the thread) who focus mostly on pre-analysis plans as ways to bind yourself, without giving much consideration to the idea of using them as the basis of having an <i>ex ante<\/i> conversation about the research.<\/p>\n<p>The thread is here:<\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">As current Executive Director of <a href=\"https:\/\/twitter.com\/EGAPTweets?ref_src=twsrc%5Etfw\">@EGAPtweets<\/a> let me explain how EGAP does research designs and pre-analysis plans (RDs\/PAPs), which I think is different from what these authors understand as the point of RDs\/PAPs. 1\/9 <a href=\"https:\/\/t.co\/f4DwqxVsWL\">https:\/\/t.co\/f4DwqxVsWL<\/a><\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564056248066051?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">RDs\/PAPs should be a vehicle for having an ex ante *conversation* about what you are doing, as a way to refine your study and also get buy-in and agreement. 2\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564057468649475?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">This is similar to what Duflo et al discuss as the &#8220;interested party&#8221; rationale. I would say that this is more general than the set of cases that Duflo et al consider. 3\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564058202689536?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">Much of the work that is carried out via <a href=\"https:\/\/twitter.com\/EGAPTweets?ref_src=twsrc%5Etfw\">@EGAPtweets<\/a> consists of expensive trials of policy innovations conducted with NGOs and governments. We don&#8217;t want to waste opportunities or mess these up. Feedback prior to implementation is *really* important. 4\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564058924027904?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">RDs\/PAPs that are never reviewed or discussed prior to their implementation are kind of pointless in my opinion. 5\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564059670659073?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">The primary role of <a href=\"https:\/\/twitter.com\/EGAPTweets?ref_src=twsrc%5Etfw\">@EGAPtweets<\/a> is to organize sessions to discuss RDs\/PAPs. Also regional workshops (like NEWEPS, etc). do the same. The short PAPs that these authors advocate wouldn\/t make for interesting sessions, and being able to do such sessions is the point. 6\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564060408877056?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">It seems that in econ this isn&#8217;t the practice, as seminar time is for finished and polished work. I honestly think many just don&#8217;t know what they are missing and would value <a href=\"https:\/\/twitter.com\/EGAPTweets?ref_src=twsrc%5Etfw\">@EGAPtweets<\/a> style ex ante feedback. 7\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564061142896648?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">Research on COVID has made clear the need for these RD\/PAP discussions, given problems with design and then also analysis of the trials and surveys. If such research were subject to pre-field and pre-analysis discussions among critical audiences, learning would be faster. 8\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564061843226625?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">As it is now, few people believe the results that are being produced because the designs and analyses are weak. Here the stakes are very real. 9\/9<\/p>\n<p>\u2014 Cyrus Samii (@cdsamii) <a href=\"https:\/\/twitter.com\/cdsamii\/status\/1252564062656794625?ref_src=twsrc%5Etfw\">April 21, 2020<\/a><\/p><\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Below is a Twitter thread in which I offer a perspective from my experience through EGAP (egap.org) on how to make effective use of pre-analysis plans and also research designs. The basic idea is that your research design and pre-analysis plan should serve as the basis of a discussion in which you can refine your &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/cyrussamii.com\/?p=3154\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Using pre-analysis plans to learn better and to learn together&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3154","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/3154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3154"}],"version-history":[{"count":3,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/3154\/revisions"}],"predecessor-version":[{"id":3157,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/3154\/revisions\/3157"}],"wp:attachment":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3154"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}