{"id":2041,"date":"2015-06-04T13:21:06","date_gmt":"2015-06-04T17:21:06","guid":{"rendered":"https:\/\/cyrussamii.com\/?p=2041"},"modified":"2015-06-04T13:29:59","modified_gmt":"2015-06-04T17:29:59","slug":"more-on-external-and-construct-validity","status":"publish","type":"post","link":"https:\/\/cyrussamii.com\/?p=2041","title":{"rendered":"More on external (and construct) validity"},"content":{"rendered":"

The Papers and Hot Beverages (PHB) blog had a nice discussion (link<\/a>) of some of the points I raised in my previous post about “pursuing external validity by letting treatments vary” (link<\/a>). PHB starts by proposing that we can rewrite a simple treatment effects model along the lines of the following (modified from PHB’s expression to make things clearer):<\/p>\n

$latex Y = \\mu + \\rho T + \\epsilon$
\n
$latex \\hspace{2em} = \\mu+\\left(\\sum_k \\alpha_k x_k + \\sum_j \\sum_k \\kappa_{jk} z_j x_k\\right) T + \\epsilon$.<\/p>\n

The idea is that the treatment may bundle various components, captured by the $latex x$ terms, each of which has its own effect. Moreover, each of these components may interact with features of the context, captured by the $latex z$ terms.<\/p>\n

The proposal to explore external validity by “letting treatments vary” amounts to trying to identify the effect of one of the $latex x$ components by generating variation in that component that is independent of the other $latex x$ components. Of course, in doing so, one does not resolve the problem of covariation with the $latex z$ components. So in this way, I understand why PHB was not “convinced” about the strategy of letting treatments vary as being sufficient for testing a parsimonious proposition that focuses on the effect of a particular component of a treatment bundle in a manner that does not incorporate contextual conditions. Of course, I was not trying to propose that such a strategy is sufficient in this way. Just that it is another way to think about accumulating knowledge across studies. <\/p>\n

We can also go further and provide a more complete characterization of the problem of interpreting a treatment effect. Indeed, PHB’s characterization imposes some restrictions relative to the following:<\/p>\n

$latex Y = \\mu + \\rho T + \\epsilon$<\/p>\n

$latex = \\mu + \\left( \\sum_{k} \\alpha_k x_k + \\sum_{k}\\sum_{k’\\ne k} \\beta_{kk’}x_kx_{k’} \\right.$
\n
$latex \\left. + \\sum_{j} \\sum_{k} \\kappa_{jk}z_jx_k + \\sum_{j} \\sum_{k} \\sum_{k’\\ne k} \\delta_{jkk’}z_jx_kx_{k’} \\right)T + \\epsilon $<\/p>\n

The $latex \\alpha$s are effects of elements in $latex x$ that depend on neither other elements of the treatment bundle $latex x$ nor the context $latex z$. The $latex \\beta$s are the ways that elements of the treatment bundle modify each others’ effects regardless of context. The $latex \\kappa$s are ways that the context modifies the effects of elements of $latex x$ separately. Finally, the $latex \\delta$s are ways that context modifies the ways that elements of $latex x$ modify the effects of each other. <\/p>\n

When using causal estimates to develop theories, we typically want to interpret manipulations of $latex T$ in parsimonious terms. The upshot is that in trying to be parsimonious we may ignore elements of $latex x$ or $latex z$. Even if the effect of $latex T$ is well identified, our parsimonious interpretation may not be valid. <\/p>\n

This is a mess of an expression. But I find it strangely mesmerizing. It gives some indication of how complicated is the work of interpreting causal effects.<\/p>\n","protected":false},"excerpt":{"rendered":"

The Papers and Hot Beverages (PHB) blog had a nice discussion (link) of some of the points I raised in my previous post about “pursuing external validity by letting treatments vary” (link). PHB starts by proposing that we can rewrite a simple treatment effects model along the lines of the following (modified from PHB’s expression … <\/p>\n

Continue reading “More on external (and construct) validity”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2041","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/2041","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2041"}],"version-history":[{"count":19,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/2041\/revisions"}],"predecessor-version":[{"id":2060,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=\/wp\/v2\/posts\/2041\/revisions\/2060"}],"wp:attachment":[{"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2041"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2041"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cyrussamii.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2041"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}