Experimental%20Methodology%20Table.pdf. Www.povertyactionlab.org/publication/cost-effectiveness. Program Theory Assessment. Why Evaluate? The purpose of evaluation is not always clear, particularly for those who have watched surveys conducted, data entered, and then the ensuing reports filed away only to collect dust. This is most common when evaluations are imposed by others. If, on the other hand, those responsible for the day-to-day operations of a program have critical questions, evaluations can help find answers. As an example, the NGO responsible for distributing chlorine pills may speak with their local field staff and hear stories of households diligently using the pills, and occasionally see improvements in their health.
But each time it rains heavily, the clinics fill up with people suffering from diarrheal diseases. The NGO might wonder, “if people are using chlorine to treat their water, why are they getting sick when it rains? Speaking to individuals at different levels of the organization as well as to stakeholders can uncover many stories of what is going on. Impact Evaluation. Programs and policies are designed to achieve a certain goal (or set of goals). For example, a chlorine distribution program may be implemented specifically to combat high-incidence of waterborne illness in a region. We may want to know whether this program is succeeding in its goal. This isn’t the same thing as asking, “Does chlorine kill bacteria?” Or “Is the consumption of chlorine harmful?” Those questions can be answered in a real laboratory. For our program to achieve its goal of stopping illness, money must be allocated, tablets must be purchased, distribution mechanisms must be put in place, households must receive the tablets, households must use the tablets, and households must not consume untreated water.
As a normal part of operations, e.g. basic bookkeeping, certain information is produced, such as how many boxes of chlorine tablets have been shipped. Impact evaluations gauge the success of a program—where success can be broadly or narrowly defined. What is Evaluation? The word “evaluation” can be interpreted quite broadly. It means different things to different people and organizations. Engineers, for example, might evaluate or test the quality of a product design, the durability of a material, efficiency of a production process, or the safety of a bridge. Critics evaluate or review the quality of a restaurant, movie or book. A child psychologist may evaluate or assess the decision-making process of toddlers. The researchers at J-PAL evaluate social programs and policies designed to improve the well being of the world’s poor. This is known as program evaluation. Put simply, a program evaluation is meant to answer the question, “how is our program or policy doing?”
J-PAL works with governments, NGOs, donors, and other partners who are more interested in learning the answer to the questions: How effective is our program? At a very basic level, randomized evaluation can answer the question: was the program effective? Methodology Overview. What is Randomization? In its most simple sense, randomization is what happens when a coin is flipped, a die is cast, or a name on a piece of paper is drawn blindly from a basket, and the outcome of that flip, cast, or draw determines what happens next. Perhaps, the outcome of the coin flip determines who has to do some chore; the role of the die determines who gets a pile of money; the draw of a name determines who gets to participate in some activity, or a survey. When these tools (the coin, the die, the lottery) are used to make decisions, the outcome can said to be left to chance, or, randomized. Why do people let chance determine their fate?
Sometimes, because they perceive it as fair. Other times, because uncertainty adds an element of excitement. Statisticians use randomization because, when enough people are randomly chosen to participate in a survey, conveniently, the attributes of those chosen individuals are representative of the entire group from which they were chosen. 1. 2.