Assessing the contribution of impact assessment to donor decisions for international agricultural research

Ex post impact assessments (epIAs) have long been produced by research centres of the Consultative Group on International Agricultural Research (CGIAR) with a principal stated goal of informing the funding decisions of donor agencies, but there has been little formal analysis of the extent to which epIAs actually do so. To address this issue, the present analysis investigates how epIA results contribute to donor decisions via three techniques: comparison of epIA results with subsequent funding patterns; an email survey of CGIAR donors; and interviews of donor representatives. Comparison of aggregate estimates from large economic epIAs with funding patterns revealed little correlation between assessed impact and subsequent relative funding levels. Email survey responses indicate high demand for metrics directly related to poverty and which are ‘far down the impact pathway’. EpIAs are also reported as important in allocation decisions. Interviews of donor officials revealed that factors such as political priorities, perceived scientific quality and desires for continuity often influence funding decisions more than consideration of past impacts. In this context, the influence of epIAs is often indirect and ‘conceptual’. Copyright , Beech Tree Publishing.

[1]  Laura C. Leviton,et al.  Research On the Utilization of Evaluations , 1981 .

[2]  Karolynn Siegel,et al.  The Utilization of Evaluation Research , 1985 .

[3]  Steven E. Stemler,et al.  An Overview of Content Analysis. , 2001 .

[4]  K. Sheehan.  Response Variation in E-Mail Surveys: An Exploration , 1999 .

[5]  B. Beattie.  Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting , 1995 .

[6]  Michael Quinn Patton,et al.  Utilization-Focused Evaluation , 1979 .

[7]  Laura Langbein,et al.  Factors Influencing the Use of Health Evaluation Research in Congress , 1991 .

[8]  J. Baker.  Evaluating the Impact of Development Projects on Poverty: A Handbook for Practitioners , 2000 .

[9]  Nancy B. Shulock,et al.  The paradox of policy analysis: If it is not used, why do we produce so much of it? , 1999 .

[10]  Carol H. Weiss,et al.  Knowledge Creep and Decision Accretion , 1980 .

[11]  Carol Hirschon Weiss,et al.  The Interface between Evaluation and Public Policy , 1999 .

[12]  Lyn M. Shulha,et al.  Evaluation Use: Theory, Research, and Practice since 1986 , 1997 .

[13]  B. Turnbull.  The mediating effect of participation efficacy on evaluation use. , 1999, Evaluation and program planning.

[14]  David A. Raitzer,et al.  Benefit-cost meta-analysis of investment in the International Agricultural Research Centers of the CGIAR , 2008 .

[15]  David A. Raitzer,et al.  Donor Demands and Uses for Evidence of Research Impact - the Case of the CGIAR , 2005 .

[16]  H. Gregersen,et al.  Enhancing ex post impact assessment of agricultural research: the CGIAR experience , 2008 .

[17]  Kim Bartel Sheehan,et al.  E-mail Survey Response Rates: A Review , 2006, J. Comput. Mediat. Commun..

[18]  B. Shiferaw,et al.  NRM impact assessment in the CGIAR: meeting the challenge and implications for ICRISAT. , 2003 .

[19]  Robert F. Boruch,et al.  Contributions of Evaluation To Education Programs and Policy , 1983 .

[20]  Julian M. Alston,et al.  A meta-analysis of rates of return to agricultural R&D : ex pede Herculem? , 2000 .

[21]  U. Lele.  The CGIAR at 31: An Independent Meta-Evaluation of the Consultative Group on International Agricultural Research , 2004 .

[22]  Jock R. Anderson.  Selected policy issues in international agricultural research: On striving for international public goods in an era of donor fatigue , 1998 .

[23]  Julian M. Alston,et al.  Dynamics in the creation and depreciation of knowledge, and the returns to research: , 1998 .

[24]  Melvin M. Mark,et al.  Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions , 2003 .

[25]  Mywish K. Maredia,et al.  Ex Post Evaluation of Economic Impacts of Agricultural Research Programs: a Tour of Good Practice , 2000 .

[26]  Hermann Waibel,et al.  Impact assessment of agricultural research in West Africa: an application of the propensity score matching methodology , 2009 .

[27]  Javier M. Ekboir,et al.  Why impact analysis should not be used for research evaluation and what the alternatives are , 2003 .