Using pre-analysis plans to learn better and to learn together

Below is a Twitter thread in which I offer a perspective from my experience through EGAP ( on how to make effective use of pre-analysis plans and also research designs. The basic idea is that your research design and pre-analysis plan should serve as the basis of a discussion in which you can refine your design and analysis and gain buy-in from skeptics. A research design or pre-analysis plan that is never discussed publicly before it is implemented is a huge missed opportunity.

The thread was in response to a paper by Duflo et al. (linked in the thread) who focus mostly on pre-analysis plans as ways to bind yourself, without giving much consideration to the idea of using them as the basis of having an ex ante conversation about the research.

The thread is here:


Open source environments for structural estimation

If you click on the tweet below, you will get a conversation on open source options (essentially Python, Julia, and R) for students interested in getting started with structural estimation:

Among other things, people pointed to the following resources to get you started:


Spillover effects in experimental data: review essay and R package

With Stephanie Zonszein, Dean Eckles, and Peter Aronow, we have a new review article on estimating spillover effects with experimental data, with accompanying R package:

At seminars one often hears “what about SUTVA violations?” Don’t just wave your hands, rather:

  1. Learn what’s identified even w/ SUTVA violations of unspecified form–e.g.,

  2. Estimate the spillover effects–that’s what this review piece and accompanying R package are about.


Conformal inference tutorial

I am doing some work on conformal prediction methods, which allow for doing predictive regression-based inference with minimal assumptions. Mostly to help myself understand the methods in algorithmic terms, I created the following tutorial: link.

An accessible introduction is offered in this paper by Lei et al. (2017, arxiv), which accompanies the R package, conformalInference (github). They demonstrate conformal inference methods in connection with high dimensional regression and covariate selection.

In the causal inference literature, Chernozhukov et al. (2017, arxiv) use conformal methods for robust inference with synthetic control and related panel methods. Coauthors and I are doing some more work in this area.

Chernozhukov et al. (2018, arxiv) also have new work extending conformal inference to time series and other dependent-data settings.