The goal of this class is to teach students skills to design field experiments and social science RCTs in particular. The includes the following:
- Posing a compelling “problem” that warrants intervention.
- Defining and motivating an intervention on the basis of good theory and available evidence.
- Designing a compelling field test of the intervention.
The approach of the class is hands-on: as a class, we will go through 3 cycles of the RCT design process. Each cycle is intended to yield a research design that would satisfy requirements for, e.g., an NSF or similar such grant proposal. Each week will involve making progress in successive steps of RCT design. During class time, we will present material to each other for discussion.
The plan for the semester will be laid out week by week below.
For class on Sept 10, prepare the following
- Characterize your research interests in terms of a “problem” (or a few problems).
- Reference existing theory and evidence to motivate potential interventions to address the problem(s).
Prepare notes to present your problems and intervention concepts in class. We will discuss them together.
In preparation for learning nuts and bolts of experimental design, read the following:
- Gerber, Alan S., and Donald P. Green. Field experiments: Design, analysis, and interpretation. WW Norton, 2012. Ch 1-2.
- Duflo, Esther, Rachel Glennerster, and Michael Kremer. “Using randomization in development economics research: A toolkit.” Handbook of development economics 4 (2007): 3895-3962.
Also, Evidence in Governance and Politics has an online resource book that has various resources for field experimental design. Start perusing the material there: [link]
Last class we defined three problems that will serve as the basis of the RCTs that we will design over the course of the semester:
- Non-participation and non-cooperation of rightwing partisans in surveys by established polling organizations.
- Police brutality.
- Competing historical narratives that impede progress in addressing inequality or vulnerability.
We started to sketch out potential interventions, based on existing theory and evidence. For class on Sept 17, each of the group should prepare 4 slides sketching out the following:
- Characterize the nature of the problem as it applies to a particular location and population that will be the site of intervention.
- Frame the problem on the basis of a compelling social theory and describe what the theory suggests about addressing the problem. It should be a well-specified theory that describes a mechanism or behavioral parameters on which we might intervene to address the problem.
- Propose an intervention based on this analysis, defining the units of intervention. Will the intervention target individuals or groups of people?
- Describe how you would judge whether the intervention is successful. Ideally you would want to measure effects using the same kinds of outcomes that you would use to characterize the problem in the first place.
In addition, here are some more nuts and bolts readings in preparation for more technical aspects of design work:
- Gerber, Alan S., and Donald P. Green. Field experiments: Design, analysis, and interpretation. WW Norton, 2012. Ch 3,4,12.
- Congdon, W.J., J.R. Kling, J. Ludwig, S. Mullainathan, Chapter 4 – Social Policy: Mechanism Experiments and Policy Evaluations, Editor(s): Abhijit Vinayak Banerjee, Esther Duflo, Handbook of Economic Field Experiments, North-Holland, Volume 2, 2017, Pages 389-426.
In class each group will present their slides. The goal will be for everyone to get a good sense of the components of the proposed RCTs, because the next phase of the class will be for the whole class (working smaller groups) to work on developing each of the RCTs in turn.