Taking the Long View in Evaluating International Development Assistance
Taking the Long View in Evaluating International Development Assistance
R. Gregory Michaels, Chemonics International
Alphonse Bigirimana, Chemonics International
Evaluation 2010
American Evaluation Association Annual Conference
November, 2010
Abstract
Program evaluations addressing the sustainability of international development assistance have not adequately informed understanding of development effectiveness (the focus, for example, of the Evaluation Network of the Organisation for Economic Co-operation and Development’s Development Assistance Committee (OECD DAC)). Program evaluations are typically limited to the specific project life span, missing possibilities of evaluating projects well after closure. In contrast, appraising project results several years after the project intervention ends offers a robust standard for evaluating sustainability — what actually worked and what did not in the long run. Ex post project sustainability investigations offer an untapped mine of information.
This paper presents the methodologies, findings, and lessons learned from two case studies of assessments examining the sustainability of USAID-funded project results several years after closure. The first case study reports on two retrospective assessments of USAID efforts to promote agricultural exports from Central America (1986-1995). The second case study reports on an assessment of the performance of a Moroccan wastewater treatment facility constructed in 2000. These case studies illustrate the power of ex-post sustainability appraisals to offer valuable insights into the durability of development.
I. Introduction
Recent debate in the evaluation of international development assistance emphasizes the importance of analytical rigor in drawing conclusions drawn about what works and what does not. Two approaches provide decision makers with evidence about project effectiveness: impact evaluation and process evaluation. Potential contributions and
R. Gregory Michaels, Chemonics International
Alphonse Bigirimana, Chemonics International
Evaluation 2010
American Evaluation Association Annual Conference
November, 2010
Abstract
Program evaluations addressing the sustainability of international development assistance have not adequately informed understanding of development effectiveness (the focus, for example, of the Evaluation Network of the Organisation for Economic Co-operation and Development’s Development Assistance Committee (OECD DAC)). Program evaluations are typically limited to the specific project life span, missing possibilities of evaluating projects well after closure. In contrast, appraising project results several years after the project intervention ends offers a robust standard for evaluating sustainability — what actually worked and what did not in the long run. Ex post project sustainability investigations offer an untapped mine of information.
This paper presents the methodologies, findings, and lessons learned from two case studies of assessments examining the sustainability of USAID-funded project results several years after closure. The first case study reports on two retrospective assessments of USAID efforts to promote agricultural exports from Central America (1986-1995). The second case study reports on an assessment of the performance of a Moroccan wastewater treatment facility constructed in 2000. These case studies illustrate the power of ex-post sustainability appraisals to offer valuable insights into the durability of development.
I. Introduction
Recent debate in the evaluation of international development assistance emphasizes the importance of analytical rigor in drawing conclusions drawn about what works and what does not. Two approaches provide decision makers with evidence about project effectiveness: impact evaluation and process evaluation. Potential contributions and
Bibliography: Advisory Committee on Foreign Voluntary Foreign Aid (ACVFA). 2009. ACVFA Recommendations on Monitoring and Evaluation. Retrieved from: http://pdf.usaid.gov/pdf_docs/PDACQ104.pdf. Blue, Richard, Clapp-Wincek, Cynthia, and Benner, Holly Britan, Gerald M. 2009. Strengthening Evaluations at USAID. Presentation at the Advisory Committee on Voluntary Foreign Aid Meeting, June 9, 2009. Washington, DC. Clapp-Wincek, Cynthia and Blue, Richard Easterly, William. 2002. The Cartel of Good Intentions: The Problem of Bureaucracy in Foreign Aid. Center for Global Development, Washington, DC. Foresti, Marta et al. 2009. A Comparative Study of Evaluation Policies and Practices in Development Agencies. Agence Francaise de Developpement, Paris, France. Joint Japan/World Bank Graduate Scholarship Program. 2007. Tracer Study VII. May. Retrieved from: http://siteresources.worldbank.org/INTWBISFP/Resources/551491-1108589837615/tracer_studies7.pdf Michaels, G., Bigirimana, A., Soudi, B Millennium Challenge Corporation. 2009. Policy for Monitoring and Evaluation. Washington, DC. Retrieved from: http://www.mcc.gov/documents/guidance/policy-051209-mande.pdf Natsios, Andrew Office of the Director of US Foreign Assistance. 2009. Evaluation Guidelines for Foreign Assistance. Washington, DC. Scarborough, A., Michaels, R. G., and Brower, B. 2009. PROEXAG and EXITOS: Their Long-term Impact on Non-traditional Agricultural Exports. Unpublished paper. Chemonics International Inc., Washington, D.C. Teller, Charlie. 2008. Are We Learning About What Really Works? Lost Opportunities and Constraints of Producing Rigorous Evaluation Designs of Health Project Impact. The White House The White House. Office of the Press Secretary. 2010. US Global Development Policy Fact Sheet. http://www.whitehouse.gov/the-press-office/2010/09/22/fact-sheet-us-global-development-policy United Nations Children Fund United Nations Development Program. 2006. UNDP Evaluation Policy. New York, NY. United Nations Evaluation Group. 2005. Norms for Evaluation in the UN System. New York, NY. 29 April. Retrieved from: http://www.eclac.cl/dppo/noticias/paginas/4/37534/NormsForEvaluationinTheUNSystem.pdf United Nations Population Fund US Agency for International Development, 2009. FY 2009 Foreign Operations Performance Report and FY 2011 Performance Plan. ashington, DC. US Agency for International Development. Performance Monitoring and Evaluation TIPS. 1997, Volume 1. The Role of Evaluation at USAID. Washington, DC. US Office of Management and Budget. 2007. Detailed Information on the Assistance to Developing Countries Information Assessment. Retrieved from: http://www.whitehouse.gov/omb/expectmore/detail/10009034.2007.html US Office of Management and Budget US Office of Management and Budget. 2009. Rating the Performance of Federal Programs. Washington, DC. Wiebe, Franck. 2009. Key Elements of Evaluation at MCC. Washington, DC. Source: Scarborough et al., Chemonics International, 2009.