Post by Eva Vivalt, @evavivalt.

I recently had the chance to sit down with Elizabeth Tipton, assistant professor at Columbia University.

Talking about the pros and cons, she made an interesting point. While some people argue that one shouldn’t combine studies because they are all so different, in practice we already do this in our heads all the time. One difference between meta-analysis and this informal mental process, and an advantage of it, is that all the assumptions are spelled out clearly up front and examined. People have all sorts of psychological biases that could affect how they are combining results in their heads.

One might wonder how much one should weight certain studies and how much results can generalize from one context to another. This is an empirical question that can only be answered by looking at the data. We need more study, not less. Will post soon about work being done on this subject.