The essence of the design-based approach is that one establishes optimality principles in terms of research design parameters, while making minimal assumptions about the distributional properties of the variables in the analysis. The motivation for a design-based perspective is three-fold.
First, I do a lot of field research. Most field research projects seek to obtain data on a variety of outcome variables. Each of these outcome variables might differ in its distributional properties. In such cases, one wants optimal design principles that are robust to such variety. The design- based approach achieves precisely this robustness goal. This is the line of thinking one associates with Leslie Kish, William Cochran, and the literature on “official statistics.”
Second, the design-based approach aims to minimize errors of inference (for example, inaccurate confidence intervals) that arise when one uses methods that rely on distributional assumptions that are inaccurate. The design based approach achieves such error minimization by defining inferential properties primarily in terms of the design parameters that are directly controlled, and therefore known, by the researcher. This is the line of thinking that one associates with David Freedman.
Third, the design-based approach minimizes reliance on data parameters that are only revealed after the data are collected. This allows for the pre-specification and ex ante agreement among scholars on the inferential leverage that a design offers. For example, it allows for ex ante specification and agreement on what hypotheses can be tested and with what power. A design document and pre-analysis plan can guard against ex post manipulations and “results fishing,” arguably allowing for more credible accumulation of scientific knowledge. This is the line of thinking that one associates with current proponents of pre-analysis plans like Edward Miguel and Macartan Humphreys.
