How do you compare programs across outcomes to form one measure of the best program to donate to?

Here we’re starting to answer the questions we received. Please use this link to send us your questions! This question comes from Josh Tasoff, assistant professor at Claremont Graduate University.

Only a few of the programs we have considered can be evaluated using the same outcome measures. For example, we’ve seen that if your only goal is to improve test scores, school meals programs might be your best bet, with deworming, scholarship programs, and unconditional cash transfers bunched close together and conditional cash transfers lagging behind. Then there are the other education-related outcomes, the health outcomes, the economic outcomes, and so on.

How can we combine all our estimates into one conclusion as to which programs one ought support? The simple answer is that depending on one’s values, one will want to prioritize different programs. This goes against the typical approach, advocated by GiveWell and other charity navigators, of choosing essentially one or two programs to rule them all.

Charity navigators might pick one or two programs because they worry about the “opportunity cost” of donating to a program. Essentially, if you donate to one program, you probably won’t donate as much to another program, so if you want your donation to go to the most effective programs you had better identify the most effect program of all and donate to that one program.

GiveWell et al. correctly note that there are limits to this reasoning. For example, any one particular NGO has capacity constraints, and at a certain point, giving more money to it will become less effective than giving to a different NGO or cause.

However, they miss a deeper philosophical issue, which is that, at the end of the day, we have no way to compare vastly disparate outcomes such as educational outcomes and health outcomes. Which outcomes we value is a personal consideration.

There are ways we can each elicit our preferred set of programs for our own values. If we really wanted, we could introspect and choose several outcomes we cared about, or we could do some surveys to formally model our own preferences (though the latter would take quite some time!). For example, discrete choice methods, commonly used by marketers, were made to model preferences.

What about using someone else’s preferences or group preferences? This is also possible. Here we get into social choice, an entirely different domain of economics. The short answer is that there is no easy way to combine multiple individuals’ preferences. On the other hand, we can estimate a general set of preferences for a representative individual; we need only use our discrete choice models or other modeling techniques on a population rather than an individual. With these individualized or group weights, we could then rank different programs and compile a bundle of programs that, combined, hit all the right outcomes in the best possible ratio.

Most people won’t care for all this and will prefer to just go for one or two causes that are close to home, but we will be doing some work in this area, including but not limited to putting a survey up on this site to elicit some preferences, and we will report back once we have some results.

Posted in FAQ

One thought on “How do you compare programs across outcomes to form one measure of the best program to donate to?

  1. Pingback: Some friendly concerns with GiveWell | AidGradeAidGrade