Monthly Archives: December 2012

We’re continuing to answer the questions we received. Please use this link to send us your questions! This question comes from Paul Niehaus, assistant professor at the University of California, San Diego.

The full comment: “This is something we think about a lot in summarizing evidence on cash transfers – there is of course no such thing as “the effect” of cash transfers since by design they are flexible and people use them very differently in different contexts. Even within our own randomized controlled trial sample you see very clearly that poorer people with starving kids prioritize nutrition while less poor households invest more in land, livestock, housing.”

We’re trying to code up the different factors that describe a study’s setting. So, if someone wanted to look only at studies in India, for example, they could do that by going to our meta-analysis app and choosing to include only those studies that were done in India in their analysis. (Note: right now you will only see a couple of sample filters as the demo version. Behind the scenes, we have coded up an extensive list of variables for each study, including country and other characteristics of the sample studied. We are still discussing how to put these up in a user-friendly way, as nobody wants to wade through dozens of options. Please feel free to send us your comments about which filters you most want to see added or any usability suggestions.)

Now, adding filters would reduce the number of studies that would be included substantially, so the user will have to make trade-offs. If they want to pick the closest study to their specific situation, they can find just that one study. If they think their situation is somewhere in between several situations, they can include all the studies in the situations they believe might be relevant.

While we’ve been collecting data on a long list of characteristics of studies and the samples they are based on, we’re not going to catch all the characteristics one might possibly care about on our first try. As we grow, you’ll be able to suggest missing filters, which we can go back and add in. Again, the analogy is that this work is like building a Wikipedia from scratch – slow at the start, but, with the help of many individuals, we’ll serve as a good resource.

Here we’re starting to answer the questions we received. Please use this link to send us your questions! This question comes from Josh Tasoff, assistant professor at Claremont Graduate University.

Only a few of the programs we have considered can be evaluated using the same outcome measures. For example, we’ve seen that if your only goal is to improve test scores, school meals programs might be your best bet, with deworming, scholarship programs, and unconditional cash transfers bunched close together and conditional cash transfers lagging behind. Then there are the other education-related outcomes, the health outcomes, the economic outcomes, and so on.

How can we combine all our estimates into one conclusion as to which programs one ought support? The simple answer is that depending on one’s values, one will want to prioritize different programs. This goes against the typical approach, advocated by GiveWell and other charity navigators, of choosing essentially one or two programs to rule them all.

Charity navigators might pick one or two programs because they worry about the “opportunity cost” of donating to a program. Essentially, if you donate to one program, you probably won’t donate as much to another program, so if you want your donation to go to the most effective programs you had better identify the most effect program of all and donate to that one program.

GiveWell et al. correctly note that there are limits to this reasoning. For example, any one particular NGO has capacity constraints, and at a certain point, giving more money to it will become less effective than giving to a different NGO or cause.

However, they miss a deeper philosophical issue, which is that, at the end of the day, we have no way to compare vastly disparate outcomes such as educational outcomes and health outcomes. Which outcomes we value is a personal consideration.

There are ways we can each elicit our preferred set of programs for our own values. If we really wanted, we could introspect and choose several outcomes we cared about, or we could do some surveys to formally model our own preferences (though the latter would take quite some time!). For example, discrete choice methods, commonly used by marketers, were made to model preferences.

What about using someone else’s preferences or group preferences? This is also possible. Here we get into social choice, an entirely different domain of economics. The short answer is that there is no easy way to combine multiple individuals’ preferences. On the other hand, we can estimate a general set of preferences for a representative individual; we need only use our discrete choice models or other modeling techniques on a population rather than an individual. With these individualized or group weights, we could then rank different programs and compile a bundle of programs that, combined, hit all the right outcomes in the best possible ratio.

Most people won’t care for all this and will prefer to just go for one or two causes that are close to home, but we will be doing some work in this area, including but not limited to putting a survey up on this site to elicit some preferences, and we will report back once we have some results.