Inverse covariance weighting versus factor analysis

These are two ways to take a bunch of variables that are supposed to measure common latent factors and reduce them to a single or a few indices. What is the difference? I get the question fairly often, so I thought I’d put this post up.

The two approaches do different things. Inverse covariance weighting applies an assumption that there is one latent trait of interest, and constructs an optimal weighted average on the basis of that assumption. Factor analysis tries to partial out an array of orthogonal latent factors.

An intuitive way to think of it is like this:

Suppose you have data that consists of three variables: College Math Grade, Math GRE, and Verbal GRE. The two math variables will be highly correlated, and the verbal variable will be somewhat correlated with the math scores.

The inverse covariance weighted average of these three variables would result in an index that gives about 25% weight to each math score and then 50% weight to the verbal score. It “rewards” the verbal score for providing new information that the math scores don’t. The resulting index could be interpreted as a “general scholastic aptitude” index.

A factor analysis of these three variables would yield two orthogonal factors, the first factor of which would give almost 50% weight to each math variable and almost zero weight to the verbal variable, and the second would give almost zero weight to each math variable and almost 100% weight to the verbal variable. So you would get a “pure math” factor and a “pure verbal” factor.

Which one is better? It depends on the goals of your analysis.

I discuss this a bit more in my lecture on “measurement” in the quant field methods class (see links at top right). There is some R code there to play around with these concepts too.


4 thoughts on “Inverse covariance weighting versus factor analysis

  1. Cyrus

    Berk — last time I checked Michael Anderson at UC-Berkeley had a Stata ado on his site to accompany his outstanding 2008 paper. Also, the replication materials for Casey et al.’s “Reshaping Institutions” paper has some code. I have my own R code and ado too and will post it as soon as I have the chance.

  2. Berk Ozler

    Great – thanks. I’ve used Anderson’s code for q-values before. I’ll check out his site and the Casey et al…


  3. Peter

    Thanks for the post. Very helpful.

    Last I checked Anderson doesn’t have the code for the inverse covariance weighting on his website.

Leave a Reply

Your email address will not be published. Required fields are marked *