Post by Eva Vivalt, @evavivalt.
I recently had the chance to sit down with Elizabeth Tipton, assistant professor at Columbia University.
Talking about the pros and cons, she made an interesting point. While some people argue that one shouldn’t combine studies because they are all so different, in practice we already do this in our heads all the time. One difference between meta-analysis and this informal mental process, and an advantage of it, is that all the assumptions are spelled out clearly up front and examined. People have all sorts of psychological biases that could affect how they are combining results in their heads.
One might wonder how much one should weight certain studies and how much results can generalize from one context to another. This is an empirical question that can only be answered by looking at the data. We need more study, not less. Will post soon about work being done on this subject.
We are getting ready to release a new set of results as well as an update to our meta-analysis app which would provide a more detailed look at the data. Exciting times! Check back soon for the changes.
The second round of meta-analyses is well underway! We are moving from the screening stage to the data extraction phase of the process. We are also excited to be starting to work with our new summer interns! It is a really solid team and we look forward to the collaboration.
We are also aggressively looking for new staff. If you are interested in international development and would like to play a key role in a small but rapidly growing organization, this would be a great opportunity for you! We are looking for dynamic people to take on the research roles described here, mainly working on our on-going meta-analyses. Work can be done remotely, though there is a preference for candidates willing to work in the Washington, DC office. The ideal candidate has a good understanding of international development and standard economic methods of causal attribution and a passion for the field. If interested, please submit a resume to firstname.lastname@example.org; the application deadline has been extended to July 1 but early applications are preferred.
We are hiring for new researcher and coordinator roles. If interested, please submit your CV by e-mail, explaining your interest and background. Please refer to past job listings for more information.
Gain hands-on experience in the most popular topics of development impact research
Conduct meta-analysis research and gain statistics experience
Make a tangible difference in the international development community
AidGrade is offering a summer internship opportunity for undergraduates interested in development and aid effectiveness. Join our dynamic research team!
As an AidGrade intern, you will not only gain research experience but also have many other avenues for growth – managing volunteer researchers, organizing events, and making connections with other international development academic research groups. Have other skills such as web design or social media? Working with a start-up will give you the freedom and responsibility to take on self-initiated projects!
Duration: Summer, with a minimum commitment of 4 weeks. Part-time positions available.
Qualifications: Undergraduate with strong interest in international development. Statistical or meta-analysis background preferred, but not required.
Responsibilities: The intern will be responsible for sifting through impact evaluations of development programs and collecting data from them. Responsibilities will grow with the intern and training will be provided. Work can be done remotely, but there is a preference for candidates willing to work in the Washington, DC office. The ideal candidate has a good understanding of or a strong passion for international development and standard economic methods of causal attribution.
Deadline: Monday, May 20, 2013. Please submit a resume to Timothy Catlett. Applications will be reviewed on a rolling basis and early application is encouraged.
Contact Christine Shen to learn more! Other volunteer opportunities may be available to those with different qualifications.
COMMENTS FROM FORMER VOLUNTEERS
“I have learned a great deal about both the subjects I researched and the process of reviewing and analyzing the studies… invaluable!”
“I feel I have been given ownership over my work and am excited about seeing it be used!”
We only put up a demo version of our data tools earlier so as to get feedback before committing to larger changes.
There are some issues we want to address in the next iteration of our site:
1) We want to emphasize the context-dependence of results. The “compare programs” and “examine a program” links under the “donors” tab misleads people into thinking there is one true effect that a program is likely to have when they can access results that might be more applicable to their situation by using the “build your own meta-analysis” tool under the “meta-analysis” tab. Are there better ways of emphasizing this, apart from the warnings on the aforementioned pages?
2) We want people to be able to see the actual papers that the results are coming from more easily, especially for the meta-analysis app.
3) We would like your feedback as to what characteristics you would like to be able to filter on in the meta-analysis app (e.g. randomized, in a certain geographic area, etc.). As described there, we can’t show all the study characteristics, or the list of choices for users to make would be too long and unwieldy. To help enable better selection of characteristics for inclusion in the app, we will also look at the data to see which factors seem particularly correlated with the results of an intervention.
There are undoubtedly other things that also need improvement.
Do you have an idea as to how we can address these or other issues? Please let us know. We might be able to implement your design!