Cyrus Samii

Excited to share “Design-Based Inference for Spatial Experiments with Interference”, joint with Peter M. Aronow and Ye Wang: arxiv

In settings with complex spatial effects and interference, the paper defines a type of marginal effect, the “average marginalized response,” that has a clear interpretation and can be identified with a spatial experiment and a simple contrast.

It took time to work out details for robust inference, and finally got there with Ye working out reasonable conditions that justify the spatial HAC variance estimator, and then by connecting to a breakthrough CLT result from Ogburn et al. (2020; arxiv link).

We are working on the public release of the R package and also a more didactic paper that walks through applications. Stay tuned for those.

Below is a Twitter thread in which I offer a perspective from my experience through EGAP (egap.org) on how to make effective use of pre-analysis plans and also research designs. The basic idea is that your research design and pre-analysis plan should serve as the basis of a discussion in which you can refine your design and analysis and gain buy-in from skeptics. A research design or pre-analysis plan that is never discussed publicly before it is implemented is a huge missed opportunity.

The thread was in response to a paper by Duflo et al. (linked in the thread) who focus mostly on pre-analysis plans as ways to bind yourself, without giving much consideration to the idea of using them as the basis of having an ex ante conversation about the research.

The thread is here:

As current Executive Director of @EGAPtweets let me explain how EGAP does research designs and pre-analysis plans (RDs/PAPs), which I think is different from what these authors understand as the point of RDs/PAPs. 1/9 https://t.co/f4DwqxVsWL

— Cyrus Samii (@cdsamii) April 21, 2020

RDs/PAPs should be a vehicle for having an ex ante *conversation* about what you are doing, as a way to refine your study and also get buy-in and agreement. 2/9

— Cyrus Samii (@cdsamii) April 21, 2020

This is similar to what Duflo et al discuss as the “interested party” rationale. I would say that this is more general than the set of cases that Duflo et al consider. 3/9

— Cyrus Samii (@cdsamii) April 21, 2020

Much of the work that is carried out via @EGAPtweets consists of expensive trials of policy innovations conducted with NGOs and governments. We don’t want to waste opportunities or mess these up. Feedback prior to implementation is *really* important. 4/9

— Cyrus Samii (@cdsamii) April 21, 2020

RDs/PAPs that are never reviewed or discussed prior to their implementation are kind of pointless in my opinion. 5/9

— Cyrus Samii (@cdsamii) April 21, 2020

The primary role of @EGAPtweets is to organize sessions to discuss RDs/PAPs. Also regional workshops (like NEWEPS, etc). do the same. The short PAPs that these authors advocate wouldn/t make for interesting sessions, and being able to do such sessions is the point. 6/9

— Cyrus Samii (@cdsamii) April 21, 2020

It seems that in econ this isn’t the practice, as seminar time is for finished and polished work. I honestly think many just don’t know what they are missing and would value @EGAPtweets style ex ante feedback. 7/9

— Cyrus Samii (@cdsamii) April 21, 2020

Research on COVID has made clear the need for these RD/PAP discussions, given problems with design and then also analysis of the trials and surveys. If such research were subject to pre-field and pre-analysis discussions among critical audiences, learning would be faster. 8/9

— Cyrus Samii (@cdsamii) April 21, 2020

As it is now, few people believe the results that are being produced because the designs and analyses are weak. Here the stakes are very real. 9/9

— Cyrus Samii (@cdsamii) April 21, 2020

If you click on the tweet below, you will get a conversation on open source options (essentially Python, Julia, and R) for students interested in getting started with structural estimation:

For the structural estimation folks: for students starting from scratch, what would be the recommended open source alternative to Matlab to learn programing estimation routines (e.g., like Nevo 2000)? Julia?

— Cyrus Samii (@cdsamii) January 26, 2020

Among other things, people pointed to the following resources to get you started:

• https://quantecon.org/
• https://pyblp.readthedocs.io/en/stable/

With Stephanie Zonszein, Dean Eckles, and Peter Aronow, we have a new review article on estimating spillover effects with experimental data, with accompanying R package:

• Paper: https://arxiv.org/abs/2001.05444
• R package: https://github.com/szonszein/interference

At seminars one often hears “what about SUTVA violations?” Don’t just wave your hands, rather:

1. Learn what’s identified even w/ SUTVA violations of unspecified form–e.g., https://arxiv.org/abs/1711.06399.

2. Estimate the spillover effects–that’s what this review piece and accompanying R package are about.

As per Athey et al. (2019, Annals of Statistics), random forests can be interpreted as kernel estimators. If you haven’t seen this before, here is a toy example: .html