Nyasha, a PhD candidate from the Netherlands, writes,<\/p>\n
I am evaluating a food aid program for HIV\/AIDS afflicted families and individuals in Zambia. This is the data I have:<\/p>\n
\n
- 4 zones were selected in an urban area. This purposive selection was based on HIV prevalence data, these 4 zones appear to have slightly higher rates than others .<\/li>\n
- Proxy means testing was then used to select households in these 4 zones. The selection criteria was based on a combination of household and individual characteristics.<\/li>\n
- I have data from 200 \u201ctreated \u201c households from the 4 selected zones and data from 200 similar \u201c control\u201d households from 4 zones that were not selected.<\/li>\n
- Within these households, I am interested in assessing outcomes at individual level. I have personal medical data for HIV patients (300 observations), household consumption data (400 observations) and personal labour supply data for everyone in each household (1935 observations). First chapter of my thesis looks at patient level data, second chapter looks at household consumption data and the last one looks at individual labour supply data.<\/li>\n<\/ol>\n
A few questions:<\/p>\n
\n
- How I can proceed with creating propensity scores. I would like analyse the program\u2019s impact on both household level outcomes and individual level outcomes. Do I use a propensity score model defined at household level when I am assessing individual outcomes? Is it justified to use three different propensity score models for each level of analysis?<\/li>\n
- How do I incorporate the geographical selection in the matching? When I include the zone dummies in the logit model (psmatch2), I have a separation problem, with 6 of the dummies being dropped from the model. Should I continue to use this logit model? Or should I completely drop the zone dummies?<\/li>\n
- What do I do with HIV rate. The difference is there between selected zones and non-selected zones. However HIV rates do not appear to be correlated with any of my outcomes. I was thinking of using it as an instrument in IV regressions? Is it a justifiable instrument, especially when the variable is only available for 8 cluster zones? My preliminary diagnostic tests show its a valid instrument.<\/li>\n
- Can I also include the HIV rate variable as a cluster\/geographic level covariate in PSM? Or do I exclude it as it appears more to be an instrument?<\/li>\n<\/ul>\n<\/blockquote>\n
Some of the questions are probably best posed to someone working in
\nyour discipline. But let me respond to the general question about
\nmatching on multilevel data. From what I understand, your treated and
\ncontrol individuals are from different communities. Whether or not this is
\nthe case, there are different ways to do the matching with multilevel
\ndata:<\/p>\nOne way is to first use community level data to match communities that
\ninclude treated people with communities that didn’t. Then, across
\neach of the matched pairs or matched sets, find individual level
\nmatches for each of the treated individuals by drawing controls only
\nfrom the matched community. That would be a two-stage matching
\napproach, and it makes sense if you think that community level factors
\nare really important.<\/p>\nAnother way is to simply load in the community level variables along
\nwith the individual level variables and match on everything at the
\nsame time. It makes sense
\nwhen you think that community level variables are no more or less
\nimportant than individual level variables.<\/p>\nIn practice, the two approaches may generate nearly identical solutions. But such may not be the case for you, in
\nwhich case you need to decide whether you think the community level
\nvariables are of paramount importance or not.<\/p>\nThe matching can be done with pscores, coarsened exact matching,
\nnearest neighbor, genetic matching or something
\nelse—whatever you like. There are benefits and downsides to each. I have used genetic matching because in theory
\nit obtains the best outcome that either pscores or mahalanobis
\ndistance nearest neighbor matching can obtain. I have also used coarsened exact matching because of its transparency and ease of interpretation. Another alternative would be to use a generalized weighting algorithm, but I don’t think there is readily available software for it yet (although some of Jens Hainmueller’s current work seems to be promising).<\/p>\nOn some of your other question: the separation problem with the zone
\ndummies seems to be due to the fact that some zones had no treated or
\nno controls. If there are some zones that have both, you might
\nrestrict your analysis to those zones. You might then do another
\nanalysis that then adds in matched data created according to the first
\noption above.<\/p>\nIndeed, if you think something has the properties of an instrument,
\nthen you do not want to include it in the matching algorithm. That
\ncan result in bias.<\/p>\nNyasha followed up,<\/p>\n
Another question I have is this- four zones or communities in my data only have treated only. The other four only have controls. That is why I think I am encountering the separation problem when I add community dummies into the logit equation.<\/p><\/blockquote>\n
Most certainly that will create such problems.<\/p>\n
I have few other community observed characteristics which I
\nhave included already, but how best do I then control for unobserved effects
\n(especially endogenous program placement) at community level, if I cannot
\ninclude the community dummies in the propensity score model?<\/p><\/blockquote>\nAlas, you cannot. You need to make an assumption that the measured
\ncovariates capture all of the relevant differences between the
\ncommunities, and then match using these measured covariates.<\/p>\nShould I also carry out further regression analysis on the matched sample, where I then include the community dummies (as fixed effects)?<\/p><\/blockquote>\n
You will not be able to do this because of perfect collinearity with
\nthe treatment indicators. There is really no way to account for
\nunmeasured community level factors. The best you can do is use the
\nmeasured information to match, and you can also include these
\ncommunity level covariates whatever regressions you use. Then,
\nperhaps you can conduct a sensitivity analysis.<\/p>\nOr should I also look into using an IV with community fixed effects? Would this work for cross section data?<\/p><\/blockquote>\n
Again, you won’t be able to do it because the community fixed effects
\nwill be perfectly collinear with the treatment indicator, so a second
\nstage regression with community fixed effects would not be identified.<\/p>\nDid I miss something here?<\/p>\n","protected":false},"excerpt":{"rendered":"
Nyasha, a PhD candidate from the Netherlands, writes, I am evaluating a food aid program for HIV\/AIDS afflicted families and individuals in Zambia. This is the data I have: 4 zones were selected in an urban area. This purposive selection was based on HIV prevalence data, these 4 zones appear to have slightly higher rates … <\/p>\n