What to read to understand Bayesian statistics?

Easy: Sander Greenland (link). I had read and worked with Bayesian methods for some time before encountering his work. (One of my dissertation committee members was kinda well-known in Bayesian methods: link.) As I saw it, the way Bayesian methods were developed in political science, my field of application, was mostly about building big latent variable models and then using a Bayesian approach mostly for reasons of computational practicality—that is, simulation methods were often easier than trying to characterize the likelihood, and so one just threw some non-informative priors at the problem to get the machinery going and then let JAGS or BUGS rip…. But my research didn’t really benefit from latent variable modeling. I also came to interpret the retreat to simulation as a case of people trying to fit models they didn’t understand, which struck me as a suspect.* Bayesian methods I concluded simply weren’t for me.

Then I read this: PDF Gated. And this: PDF Gated. (The latter builds on Efron and Morris’s seminal work on Stein estimation: PDF1 PDF2). The papers are simple and accessible, but no less deep for that. Greenland performs a wonderful synthesis of various hierarchical, empirical Bayes, and “informative priors” Bayes methods. After reading these I eagerly consumed pretty much anything he wrote and I recommend that you do the same.

Also worth reading is Efron and Greenland’s recent Stat. Sci. exchange (as well as others’ comments, including Gelman’s): link.

*This isn’t to impugn all Bayesian latent variable modeling of course! Heck, I am working with some text analysis methods now that benefit greatly from Bayesian latent variable methods, or at least variational approximations.

Share