Combining survey and ethnographic methods to improve evaluation of conditional cash transfer
Michelle Adato
International Food Policy Research Institute, Washington DC, United States of America
PP: 222 - 236
Abstract
Drawing on evaluations of conditional cash transfer programs for the governments of Nicaragua and Turkey, this article explores how survey and ethnographic methods have been combined to improve program evaluation.
Quantitative and qualitative research designs for the two countries are described, benefits of each method discussed, and examples provided of how findings of the different approaches complemented, explained, illuminated, or contradicted each other. The surveys provide reliable measures of program impacts on human capital, while the qualitative research provides explanations of why we do or do not find these impacts, and explores how social processes and social relations are affected by and in turn shape responses to the programs.
While many official evaluations now require mixed methods, and these have impacts on policy, there is still a considerable way to go with respect to how methods are integrated and how mixed approaches are appreciated in social program evaluation.
Keywords
qualitative research, ethnography, survey research, mixed-method research, evaluation, conditional cash transfers, social protection, Nicaragua, Turkey
Article Text
Social protection encompasses a broad set of systems for protecting people against risks to their livelihoods, meant to keep people them from falling into, or deeper into, poverty. Increasingly, state and donor-supported approaches to social protection in low income countries have aimed to reduce short-term poverty, while simultaneously contributing to long-term poverty-reduction and development processes. This is largely achieved through investments in assets, from infrastructure to human capital. In the last ten years, a particular type of program has become increasingly popular, found now in over twenty countries around the world: 'Conditional cash transfer' programs (CCTs) provide families with a direct cash transfer for purchase of food and other basic necessities, conditioned on their participation in education and health services. With the introduction of CCTs has come a practice of systematically building rigorous impact evaluations into the interventions, where donors and governments often require the use of quantitative and qualitative research methods. Drawing on recent experience of the International Food Policy Research Institute's evaluations of conditional cash transfer programs] for the governments of Nicaragua and Turkey, this paper explores how survey and ethnographic and survey methods have been combined to provide reliable measures program of impacts on human capital, poverty, health, nutrition, and education, along coupled with explanations of why we do or do not find impacts, and exploration of social processes and social relations that are affected by and in turn shape responses to the programs in-depth, nuanced explanations for those changes (or lack of change), and exploration of social processes related to gender and other social relations.
2. Conditional Cash Transfer Programs in Nicaragua and Turkey
The focus of CCTs on nutrition, health and education is based on evidence of the importance of these investments at these early ages. Numerous studies have demonstrated the interactions between early childhood nutrition, health, physical and cognitive development, and between those factors and adult earnings (Martorell 1995; Grantham-McGregor et al 2007; Pollitt et al 1995; Behrman 2000). CCT programs seek to promote nutrition, health and education through cash incentives. Though program designs vary, they normally share these features: 1) The provide cash, sometimes accompanied by in-kind transfers, targeted to the country's poorest households; 2) benefits are conditioned upon family members' participation in education and health services;[2] 3) They often involve improvements in the supply and quality of services; 4) There is a strong gender dimension, emphasizing women's control of resources, and girls education.
Nicaragua's CCT, Red de Protección Social (RPS) was a comparatively small program in Latin America, with an initial budget in 2000 of about US$11 million. The second phase expansion in 2002 was designed for another three years, with a budget of $22 million. In 2004, 21,619 families were enrolled in the program (Maluccio and Flores 2005). Beneficiaries were required to bring children under five for growth monitoring and vaccinations; and to attend health and nutrition workshops. In phase 2, certain health services were extended to pregnant and lactating women, and children 6-9 years. Children under five also received vitamin A, iron, and anti-parasite treatment. Education benefits were conditioned on primary school-age children's enrollment and 85 percent attendance rate.
Turkey's CCT program was part of the Social Risk Mitigation Program (SRMP) that targeted 1.1 million beneficiaries and allocated $360 million toward a social safety-net targeted to the poorest 6 percent of families (World Bank 2001). It was fully operational by 2004. Payments were higher for secondary school enrollment and attendance than for primary school, and higher for girls than for boys. Health benefits were conditioned on vaccinations and regular check-ups for children. There was also a benefit for pregnant women conditioned on check-ups, and giving birth in a hospital.
3. Combining Survey and Ethnographic Methods for the Evaluation of CCT programs ...continues...
[1] These evaluations were conducted by the International Food Policy Research Institute.
[2] For discussions of benefits and drawbacks of conditioning transfers, see Adato and Bassett 2007.
References
Adato M, Roopnaraine T, Smith N, Altinok E, Çelebioğlu N and Cemal S (2007) An evaluation of the conditional cash transfer program in Turkey: Second qualitative and anthropological study, Washington DC: International Food Policy Research Institute.
Adato M, Meinzen-Dick R, Hazell P and Haddad L (2007) Integrating social and economic analyses to study impacts on livelihoods and poverty: Conceptual frameworks and research methods. In Adato M and Meinzen-Dick R (eds) Agricultural Research, Livelihoods, and Poverty, IFPRI, Johns Hopkins University Press.
Adato M (2006) 'Empowerment and social cohesion in conditional cash transfer programs'. Presented at the Third International Conference on Conditional Cash Transfers, Istanbul, June 26-30.
Adato M and Roopnaraine T (2004) A Social analysis of Red de Protección Social in Nicaragua, Washington DC: International Food Policy Research Institute.
Adato M (2000) The Impact of PROGRESA on community social relationships. Washington DC: International Food Policy Research Institute.
Adato M, Coady D and Ruel M (2000) An operations evaluation of PROGRESA from the perspective of beneficiaries, promotoras, school directors, and health staff, Washington DC: International Food Policy Research Institute.
Adato M and Mindek D (2000) PROGRESA and women's empowerment: Evidence from six Mexican states. In Adato M, de la Brière B, Mindek D and Quisumbing A (eds) The Impact of PROGRESA on women's status and intrahousehold relations, Washington DC: International Food Policy Research Institute.
Ahmed A, Adato M, Kudat A, Gilligan D, Roopnaraine T, and Çolasan R (2007) Impact evaluation of the conditional cash transfer program in Turkey, Washington DC: International Food Policy Research Institute.
Ahmed A, Kudat A and Çolasan R (2006) 'Evaluation the conditional cash transfer program in Turkey'. Presented at the Third International Conference on Conditional Cash Transfers, Istanbul, June 26-30.
Ahmed A, Gilligan D, Kudat A, Çolasan R, Tatlidil H and Ozbilgin B (2006) Interim impact evaluation of the conditional cash transfer program in Turkey: A Quantitative assessment, Washington DC: International Food Policy Research Institute.
Ayala Consulting (2006) 'Country profiles', report prepared for the Third International Conference on Conditional Cash Transfers, Istanbul June 26-30.
Behrman JR (2000) Literature Review on Interactions between Health, Education and Nutrition and the Potential Benefits of Intervening Simultaneously in All Three, Washington DC: International Food Policy Research Institute.
Bernard HR (2002) Research methods in anthropology: Qualitative and quantitative approaches, Walnut Creek CA: Altamira Press.
Brewer J and Hunter A (1989) Multimethod research: A synthesis of styles, Newbury Park CA: Sage.
De la Brière B and Quisumbing A (2000) The Impact of PROGRESA on intrahousehold decision-making and relative schooling achievements of boys and girls. In Adato M, de la Brière B, Mindek D and Quisumbing A (eds) The Impact of PROGRESA on women's status and intrahousehold relations, Washington DC: International Food Policy Research Institute.
Burawoy M (1998) The extended case method, Sociological Theory 16(1): 4-33.
Creswell JW (1995) Research design: Qualitative and quantitative approaches, Thousand Oaks CA: Sage.
Denzin NK (1978) The logic of naturalistic inquiry. In Denzin NK (ed) Sociological methods: A sourcebook, New York: McGraw-Hill.
Grantham-McGregor S, Cheung YB, Cuerto S, Glewwe P, Richter L and Strupp B (2007) Developmental potential in the first 5 years for children in developing countries, The Lancet 369(9555): 60‑70.
Greene JC, Caracelli VJ and Graham WF (1989) Towards a conceptual framework for mixed-method evaluation design, Educational Evaluation and Policy Analysis 11.
Hallman K, Lewis D and Begum S (2007) Assessing the impact of vegetable and fishpond technologies on poverty in rural Bangladesh. In Adato M and Meinzen-Dick R (eds) Agricultural Research, Livelihoods, and Poverty, Baltimore: Johns Hopkins University Press.
Jick TD (1979) Mixing qualitative and quantitative methods: Triangulation in action, Administrative Science Quarterly 24.
Kudat A (2006) Evaluating the conditional cash transfer program in Turkey: A qualitative assessment, Washington DC: International Food Policy Research Institute.
Levy S (2006) Progress against poverty: Sustaining Mexico's Progresa-Oportunidades program, Washington DC: The Brookings Institution.
Maluccio J (forthcoming) Household targeting in practice: The Nicaraguan Red de Protección Social, Journal of International Development.
Maluccio J and Flores R (2005) Impact evaluation of a conditional cash transfer program: The Nicaraguan Red de Protección Social, Research Report 141, Washington DC: International Food Policy Research Institute.
Maluccio J, Adato M, Flores R and Roopnaraine T (2005) Nicaragua: Red de Proteccion Social-Mi Familia, Breaking the cycle of poverty, Washington DC: International Food Policy Research Institute.
Martorell R (1995) Results and implications of the INCAP follow-up study, Journal of Nutrition 125: 1127S-1139S.
Mitchell JC (1987) Case studies. In Ellen R (ed) Ethnographic Research, London: Academic Press.
Morris S, Olinto P, Flores R, Nilson EAF and Figueiró AC (2004) Conditional cash transfers are associated with a small reduction in the rate of weight gain of preschool children in Northeast Brazil, The Journal of Nutrition 134: 2336-2341.
Place F, Adato M and Hebinck P (2007) Understanding rural poverty and investment in agriculture: An Assessment of integrated quantitative and qualitative research in Western Kenya, World Development 35(2): 312-325.
Pollitt, E, Gorman MS, Engle PL, Rivera JA and Martorell R (1995) Nutrition in early life and the fulfillment of intellectual potential, Journal of Nutrition 125(4)(Supplement): S1111‑S1118.
Ravallion M (2001) The mystery of the vanishing benefits: An introduction to impact evaluation, The World Bank Economic Review 15(1): 115-140.
Rawlings L and Rubio G (2005). Evaluating the impact of conditional cash transfer programs, The World Bank Research Observer 20(1).
Skoufias E (2005) PROGRESA and its impacts on the welfare of rural households in Mexico. IFPRI Research Report No 139, Washington DC: International Food Policy Research Institute.
Tashakkori A and Teddlie C (1998) Mixed methodology: Combining qualitative and quantitative approaches, Thousand Oaks CA: Sage.
World Bank (2006) 3rd International Conference on Conditional Cash Transfers: Conference Proceedings, Washington DC: The World Bank. Available at http://web.worldbank.org/WBSITE/EXTERNAL/WBI/WBIPROGRAMS/SPLP/0,,content
MDK:20892674~pagePK:64156158~piPK:64152884~theSitePK:461654,00.html
World Bank (2001) Project Appraisal Document (PAD) on a Proposed hybrid investment/adjustment loan in the amount of $500 Million to the Republic of Turkey for a social risk mitigation project/loan. Washington DC: World Bank.
eContent Home
