It's All About MeE: Using Structured Experiential Learning ('e') to Crawl the Design Space

Organizations that fund development projects whether they be governments, multi-laterals, bilateral agencies, or NGOs have to make hard decisions about what to fund. In this decision there is an inherent tension between funding activities that have solid evidence about effectiveness and funding innovative activities that promise even greater effectiveness but are untested. Evidence based approaches that promote greater use of Rigorous Impact Evaluations (including randomized control trials) and evidence from those evaluations in policy and programming have added more rigor to the E (evaluation) in traditional M&E. Here we extend the basic idea of rigorous impact evaluation?the use of a valid counter-factual to make judgments about causality to evaluate project design and implementation. This adds a new learning component of experiential learning or a little e to the M&RIE so that instead of just M&E development projects are all about MeE. Structured experiential learning allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real time performance information with direct feedback into project design and implementation. The key insight is that within- project variations can serve as their own counter-factual which dramatically reduces the incremental cost of evaluation and increases the usefulness of evaluation to implementing agencies. The right combination of MeE provides for rigorous learning while the providing needed space for innovation.

[1]  Lyn Squire,et al.  Economic Analysis of Projects , 1976 .

[2]  L. Squire,et al.  Beyond Rate of Return: Reorienting Project Appraisal , 1997 .

[3]  Esther Duflo,et al.  Peer Effects , Pupil-Teacher Ratios , and Teacher Incentives : Evidence from a Randomized Evaluation in Kenya , 2007 .

[4]  Dani Rodrik,et al.  The New Development Economics: We Shall Experiment, but How Shall We Learn? , 2008 .

[5]  J. Hammer Economic Analysis for Health Projects , 1996 .

[6]  M. Székely Toward Results-Based Social Policy Design and Implementation , 2011 .

[7]  N. Stern,et al.  The theory of cost-benefit analysis , 1987 .

[8]  M. Ravallion The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation , 2001 .

[9]  Craig W. Thomas,et al.  Evidence for development effectiveness , 2009 .

[10]  J. Mirrlees,et al.  Manual of Industrial Project Analysis in Developing Countries. Vol. I Methodology and Case Studies. Annex to Vol. I Industrial Profiles.@@@Manual of Industrial Project Analysis in Developing Countries. Vol. II. Social Cost Benefit Analysis. , 1970 .

[11]  B. Greenberg Evaluation of Social Programs , 1968 .

[12]  Barry Bosworth,et al.  India policy forum , 2009 .

[13]  J. Behrman,et al.  Timing and Duration of Exposure in Evaluations of Social Programs , 2008 .

[14]  Frederic Sautet,et al.  The Origin and Evolution of New Businesses , 2002 .

[15]  Nava Ashraf,et al.  Household bargaining and excess fertility: An experimental study in Zambia. , 2014 .

[16]  Steve Jones,et al.  Almost like a whale : the origin of species updated , 1999 .

[17]  Lant Pritchett,et al.  Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation , 2013 .

[18]  E. Ostrom Crossing the great divide: Coproduction, synergy, and development , 1996 .

[19]  J. Newhouse Free for All?: Lessons from the RAND Health Insurance Experiment , 1993 .

[20]  W. Easterly,et al.  The White Man's Burden: Why the West's Efforts to Aid the Rest Have Done So Much Ill and So Little Good , 2006 .

[21]  R. Hausmann The Other Hand: High Bandwidth Development Policy , 2008 .

[22]  Lant Pritchett,et al.  It pays to be ignorant: A simple political economy of rigorous program evaluation , 2002 .

[23]  Jim Y. Kim Delivering on development : harnessing knowledge to build prosperity and end poverty , 2012 .

[24]  Esther Duflo,et al.  Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives, Managerial Autonomy and Training , 2012 .

[25]  J. Hammer,et al.  Money for Nothing: The Dire Straits of Medical Practice in Delhi, India , 2005 .

[26]  Esther Duflo,et al.  Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya , 2008 .

[27]  Arianna Legovini Development impact evaluation initiative : a World Bank-wide strategic approach to enhance developmental effectiveness , 2010 .

[28]  Lant Pritchett,et al.  Capability Traps? The Mechanisms of Persistent Implementation Failure , 2010 .

[29]  M. Grindle Good Enough Governance: Poverty Reduction and Reform in Developing Countries , 2004 .

[30]  D. Ariely,et al.  PSYCHOLOGICAL SCIENCE Research Article Contagion and Differentiation in Unethical Behavior The Effect of One Bad Apple on the Barrel , 2022 .

[31]  William Easterly Planners versus Searchers in Foreign Aid , 2006, Asian Development Review.

[32]  Extrapolate-Ing: External Validity and Overidentification in the Late Framework , 2010 .

[33]  K. Mackay,et al.  IMPACT EVALUATION- THE EXPERIENCE OF THE INDEPENDENT EVALUATION GROUP OF THE WORLD BANK , 2006 .

[34]  A. Deaton Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development , 2009 .

[35]  Johnathan Roberts,et al.  The Modern Firm: Organizational Design for Performance and Growth , 2004 .

[36]  James A. Robinson,et al.  Institutions as a Fundamental Cause of Long-Run Growth , 2005 .

[37]  Lant Pritchett,et al.  Escaping Capability Traps Through Problem-Driven Iterative Adaptation (PDIA) , 2012 .

[38]  D. Rodrik One Economics, Many Recipes: Globalization, Institutions, and Economic Growth , 2007 .

[39]  Felipe Barrera-Osorio,et al.  Incentivizing Schooling for Learning: Evidence on the Impact of Alternative Targeting Approaches , 2013, The Journal of Human Resources.

[40]  P. Dupas,et al.  Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment , 2007 .

[41]  Partha Dasgupta,et al.  Guidelines for Project Evaluation. , 1972 .

[42]  J. Angrist,et al.  Extrapolate-Ing: External Validity and Overidentification in the Late Framework , 2010 .

[43]  F. Fukuyama,et al.  New Ideas on Development after the Financial Crisis , 2011 .

[44]  C. Kenny Getting Better: Why Global Development Is Succeeding--And How We Can Improve the World Even More , 2011 .

[45]  Esther Duflo,et al.  Monitoring Works : Getting Teachers to Come to School ∗ , 2007 .

[46]  Robert T. Jensen,et al.  The (Perceived) Returns to Education and the Demand for Schooling , 2010 .

[47]  Banco Mundial The demand for water in rural areas: determinants and policy implications. , 1993, The World Bank research observer.

[48]  N. Siegfried,et al.  Randomized Controlled Trials of HIV/AIDS Prevention and Treatment in Africa: Results from the Cochrane HIV/AIDS Specialized Register , 2011, PloS one.

[49]  Fadhel Kaboub Realistic Evaluation , 2004 .

[50]  J. Hammer,et al.  Weak Links in the Chain II , 2002 .

[51]  Esther Duflo,et al.  Can Institutions Be Reformed from within? Evidence from a Randomized Experiment with the Rajasthan Police , 2012 .

[52]  A. Munnell Lessons from the income maintenance experiments: an overview , 1986 .

[53]  William Easterly,et al.  The elusive quest for growth: Economists''adventures and misadventures in the tropics, MIT Press, , 2001 .

[54]  Dean Spears,et al.  Policy Lessons from Implementing India's Total Sanitation Campaign* , 2012 .

[55]  J. Hammer,et al.  Money for Nothing: The Dire Straits of Medical Practice in Delhi, India , 2005 .

[56]  M. Moore Creating public value : strategic management in government , 1995 .

[57]  Lant Pritchett,et al.  Solutions When the Solution is the Problem: Arraying the Disarray in Development , 2002 .

[58]  G. Mwabu,et al.  Pitfalls of Scaling-up Proven Education Interventions: Evidence from an RCT in Kenya , 2012 .

[59]  Sandra I McCoy,et al.  Weighing the gold in the gold standard: challenges in HIV prevention research , 2010, AIDS.

[60]  J. Hammer,et al.  Weak Links in the Chain , 2000 .

[61]  Daniel A. Kaufmann,et al.  Good Countries or Good Projects? Macro and Micro Correlates of World Bank Project Performance , 2011 .

[62]  Alaka Holla,et al.  Pricing and Access : Lessons from Randomized Evaluations in Education and Health , 2008 .

[63]  Michael A. Clemens What’s Wrong with the Millennium Development Goals? , 2005 .

[64]  Mark H. Moore,et al.  What Business Might Learn from Government about Strategic Management , 2004 .

[65]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[66]  S. Baird,et al.  Designing Cost-Effective Cash Transfer Programs to Boost Schooling Among Young Women in Sub-Saharan Africa , 2009 .

[67]  Henry Mintzberg,et al.  Of strategies, deliberate and emergent , 1985, Strategic Management Journal.

[68]  Jakob Svensson,et al.  Power to the People: Evidence from a Randomized Field Experiment of a Community-Based Monitoring Project in Uganda , 2007 .

[69]  Esther Duflo,et al.  PUTTING A BAND-AID ON A CORPSE: INCENTIVES FOR NURSES IN THE INDIAN PUBLIC HEALTH CARE SYSTEM. , 2008, Journal of the European Economic Association.

[70]  Sendhil Mullainathan,et al.  What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment , 2009 .

[71]  Onora ONeill Principles of action , 2013 .

[72]  Andrew M. Warner Cost - benefit analysis in World Bank projects , 2010 .