Cyrus Samii

This week the cover feature of The Economist magazine argues that it is “time to scrap affirmative action” (link). In the US, the article anticipates the Supreme Court’s hearing cases from Michigan and Texas that may have major impact on the practice of affirmative action in college admissions. (See this post for a summary of affirmative action’s evolution in the US: link.) The magazine also considers affirmative action in the US against experiences elsewhere, with separate stories focusing on South Africa and Malaysia. This comparative perspective is welcomed. I recently taught an undergraduate seminar studying affirmative action policies around the world (course page). The seminar considered arguments on either side of affirmative action debates in different countries, trying to develop for students a nuanced perspective.

As is often the case with Economist features, I came away annoyed by the superficiality of its journalists’ treatment of the issue. First, the journalists point to the fact that while affirmative action policies are targeted toward historically “disadvantaged” (a euphemism…) groups, the beneficiaries are often relatively better-off members of those groups. This is taken to mean that affirmative action is broken, insofar as it does not necessarily help the neediest. This kind of argument often comes up, and a commonly proposed “solution” is to do away with group based affirmative action and pursue instead policies aimed at improving prospects for the poor—that is, to replace “race” with “class” as the basis for preferential policies.

This line of argument is highly problematic to me. Why should families that are members of historically disadvantaged groups become disqualified for reparation because of their success? Reparation is still meaningful for such families: a family that is middle class despite discrimination may well have been upper middle class had there been no discrimination. Many reasonable theories of justice would take such a gap to be quite worthy of redress (cf. the work by Lowry, Weisskopf, and Fryer on the syllabus for my seminar linked above). Arguments going back over a century to DuBois’s “Talented Tenth” essay (link) go even further to propose that redress of this variety restores among society’s elite positions for members of groups that have been discriminated-against. Note that a natural implication of such restoration is an increase in income inequality within the formerly discriminated-against group. So long as this is a reflection of the income “ceiling” being lifted rather than an income “floor” falling, such an increase in inequality poses no ethical dilemma with respect to concerns related to redressing legacies of discrimination. (Whether inequality per se nonetheless deserves attention is a separate issue.)

In a nutshell, there is a problem with conflating redress of legacies of racial discrimination on the one hand with poverty relief or removing barriers to class-based mobility on the other. A switch from race-based to class-based affirmative action would inevitably dilute reparation of race-based discrimination in favor of transfer to groups who were not victims of institutionalized discrimination. In the US, say, the problem is that sustaining group- rather than class-based preferences is a very hard position to sustain with the mass electorate, because it conflicts with the self interest of the (non black) majority. As such, sustaining this position would require keeping it out of the hands of the mass electorate—the usual minority protection arguments. In places like South Africa and, now, India, the situation is different, as “disadvantaged” status applies to majorities.

The other problem I had with the Economist’s treatment of the issue was their failure to engage adequately with the deep empirical literature on this topic. In discussing affirmative action in the US, the journalists spend a lot of time on Sander and Taylor’s work on the so-called “mismatch” hypothesis, without considering solid critiques of these findings by some of the sharpest minds in social science: link. That fact that none of these critiques were mentioned is a sad statement either of how journalists abuse scientific evidence to make points that they find appealing on the basis of taste or ideology, how little research is actually done in the production of pieces for highly influential venues like the Economist, or how catchy but potentially fallacious arguments generate buzz that drown out critiques (cf. Rogoff-Reinhart).

This is not to say that I am an uncritical believer in the expansion of affirmative action policies. I take the potential for perverse incentives seriously and understand how thorny it can be to design mechanisms for redress in a worse-than-second-best world. What I do hope for is that such policies are considered on the basis of relevant considerations and a serious review of the evidence.

For those of you in the greater NYC area, we will be hosting the annual Northeast Methodology Program meeting at NYU on Friday, May 3. The program starts at noon with lunch, followed by an afternoon of presentations. This year’s presentations focus on methods for analyzing text. The lineup includes the following:

• Justin Grimmer (Stanford University), “The Impression of Influence: How Legislator Communication and Government Spending Cultivate a Personal Vote”
• Burt L. Monroe, Eitan Tzelgov, and Douglas R. Rice (Penn State University), “Measurement of Topics and Topicky Concepts in Text”
• Nick Beauchamp (Columbia University), “Someone is Wrong on the Internet: Political Argument as the Exchange of Conceptually Networked Ideas”

Imbens and Kolesar have a new NBER working paper, “Robust standard errors in small samples: Some practical advice”: gated link. They propose a new “default” choice for standard errors and confidence intervals to accompany OLS estimates of treatment effects, based on recommendations of Bell and McCaffrey (2002): ungated link. Their ideas are consistent with a previous blog post here, and actually discuss explicitly some of the points that were raised in the comments: link (see especially the comments section). In fact, many of the points that Imbens and Kolesar make are anticipated in a recent paper by Lin, who discusses the performance of OLS-based methods in estimating treatment effects in randomized experiments: ungated link. (See also the discussion of the paper at the Development Impact Blog: link)

The “robust inference for treatment effects” problem can be broken down into two components: (i) getting good standard errors for your point estimates of the treatment effect and (ii) relating these to a reasonable approximation of the sampling (or randomization) distribution of the point estimate to construct hypothesis tests or confidence intervals. Standard practice today for robust inference is captured by Stata’s default under the “, robust” option (or, for cluster-robust, the “, cluster(id)” option). For the non-clustered inference scenario, these defaults use (i) White’s heteroskedasticity consistent covariance estimator to obtain the standard errors, and then (ii) approximate the sampling distribution of the point estimates using t(n-k), where n is the number of data points and k the number of regressors. While White’s estimator is consistent, in finite samples the bias may be pronounced, a point that White himself developed in MacKinnon and White (1985): ungated link. In that 1985 paper, MacKinnon and White introduced the leverage-adjusted HC2 estimator that removes the finite sample bias. It is precisely this HC2 estimator that Imbens and Kolesar recommend as the default that people should use. Incidentally, Peter and I have a paper from last year that also demonstrates that HC2 provides a robust and conservative approximation to the exact randomization variance for unit-randomized experiments, and we also recommend its use: ungated link.

What about component (ii), an approximation of the sampling (or randomization) distribution? The t(n-k) approximation is from an analogy to the case of homoskedastic normal errors, where t(n-k) is in fact the exact sampling distribution. As discussed in the previous blog post (referenced above), the hope is that this gives an adequate amount of “tail fattening” to the reference distribution for the non-homoskedastic case. However, as Imbens and Kolesar demonstrate in a simple example, this approximation is obviously bad when the treatment variable is highly skewed. Consider a case where we have n1 treated, with n1 very large, and n0 control, with n0 very small. We estimate the treatment effect by regressing outcomes on a constant and treatment dummy, and so the treatment effect estimate is equivalent to taking the difference in treated and control means. Then, the treated mean will be highly precise, with a sampling/randomization variance of about 0. The estimate of the control mean will be very imprecise. The sampling/randomization variance of the treatment effect estimate will be driven almost exclusively by the variability in the control mean. The appropriate degrees of freedom for the approximate sampling/randomization distribution is probably closer to n0-1 rather than n-2 = n1 + n0 – 2. This could make a big difference. This is an old problem, and a classical way to deal with it is to use Welch’s approximation to the degrees of freedom for the sampling distribution (link). Welch’s approximation addresses the problems that arise due to skew. (Lin studies Welch-based approxomations in the simulations in the paper linked above, and he also explained it in the comments to the blog post linked above.) The problem with Welch’s degrees-of-freedom approximation is that it relies on estimates of the conditional error variances, which can be quite imprecise in finite samples. This is where Bell and McCaffrey come in. They propose to use a Welch approximation to the degrees of freedom that assumes homoskedasticity, allowing one to avoid having to plug in estimates of the conditional error variances. Of course the homoskedasticity assumption is typically false, but using it for the degrees of freedom approximation at least partially handles the skew problem without introducing new volatility problems. It stands as a reasonable compromise. Simulation studies in the Bell and McCaffrey paper as well as in the Imbens and Kolsar paper shows that it performs well (even outperforming bootstrap methods, such as the wild-t bootstrap, at least as the latter is conventionally applied).

The two papers develop these ideas for more general regression scenarios, including the clustered inference scenario. As for practice, estimating HC2 is easy (it is already and option with the “, robust” command in Stata, and in R it should be simple to program). I don’t know if the Bell-McCaffrey degrees of freedom approximations are pre-canned, but the expressions are not so complicated.

I am doing some writing on post-conflict elections and democratization based on separate field studies in Nepal and Liberia. I created this graph to provide some context for the research. I used the UCDP Armed Conflicts dataset (link) to identify post-conflict “peace spells” in the period of 1990 to 2009. These peace spells are defined as periods of peace after ceasefires or peace agreements in civil wars and other types of intra-state armed conflict. I then added to these post-conflict peace spells Polity scores from the Quality of Governance dataset (link). (We should all take a moment to pay respect to the Swedish taxpayer, whose dollars allow such seminal datasets to be created and maintained!) The Polity score runs from -10 to 10 and measures a polity’s progress from autocracy (-10) to democracy (10). I modified these scores such that if a peace spell ended in a resumption of armed conflict, the year of resumption was coded as a -10, and then the time series was terminated. The x-axis plots years into the post-conflict peace spell, and the y-axis plots the modified Polity scores. The trajectories are smoothed (using a quadratic fit) to make them easier to read. The red X’s show a resumption of armed internal conflict. If a line ends before 10 years without a red X appearing, that means that it has come up against the last year recorded in the data (2009).

Breaking things out by region like this allows one to see the remarkable success of Latin American cases in going from armed conflict to democracy. It stands far apart from the other regions. The density of activity for sub-Saharan Africa reflects the extent to which the continent has endured armed political strife over the past two decades. At the same time, we see dramatic heterogeneity in the political dynamics that unfold after fighting stops.

An heuristic visualization for use in my class on comparative political economy of affirmative action policies. Federal standing is scored simply by adding a point for each federal action or Supreme Court decision promoting the application of affirmative action, and subtracting a point for each action that curtails it. It is a gross simplification of course, but by appearances, it seems to capture the trends well. For example, while there have been steady curtailments since the early 1990s, it would be inaccurate to suggest federal standing is where it was at the time of enacting the Civil Rights Act in 1964. The federal level actions causing the score to increase or decrease are also shown.

Below the federal standing score is a plot of the Martin-Quinn score (link) of the median Supreme Court justice, a measure of the court’s liberal orientation (shown as blue, higher values) versus conservative orientation (red, lower values). Below that is a timeline of the party of the president.

John David Skrentny, in his magisterial account of the evolution of affirmative action politics in the US (link), highlights crisis management in the face of the race riots of the mid-1960s and US leadership’s concerns over international reputation during the Cold War as crucial determinants of affirmative action’s rise in that period. We can see that, at least by measure of party and court ideology, the institutional/ideological context was also especially receptive. At the same time, trends pushing upward the federal standing of affirmative action continued despite Nixon assuming office and a trend toward conservatism on the court.