An impact evaluation framework based on sustainable livelihoods for energy development projects: an application to Ethiopia

Abstract Within the current global challenges, energy plays a key role for the achievement of basic human needs, socio-economic development, environmental protection and global security. At the light of this interconnection, a proper impact evaluation metric able to assess the main effects of energy projects at local level becomes necessary in order to highlight successful strategies. Relying on the Sustainable Livelihoods concept, this study proposes an Impact Evaluation Framework (IEF) to measure project impact as changes of target community’s livelihoods. First, the IEF establishes a Capitals-Based Evaluation Hierarchy, taking its rationale from the literature: this first step responds to the needs of providing a standard and harmonized structure applicable to different projects. Secondly, the IEF develops a further Five-Step Procedure to respond to the concurrent need of flexibility and customization of specific projects. The conceptual methodology of the IEF might be used at different stages of project design: as a supportive methodology donors use in their programme of funds allocation or as an instrument experts use to quantitatively support their ex-post project evaluation. In the paper, the IEF is presented in the light of this second application and the procedure applied to a real project in Ethiopia. The set of information obtained with the IEF is compared to the final expert evaluation, commissioned by the donor and performed at the end of the project, showing the usefulness of IEF as a supportive methodology in the evaluation process.

[1]  E. Colombo,et al.  Renewable energy for unleashing sustainable development , 2013 .

[2]  R. Went,et al.  Doing Good or Doing Better : Development Policies in a Globalising World , 2009 .

[3]  Benjamin Sovacool,et al.  New partnerships and business models for facilitating energy access , 2012 .

[4]  Elisabeth Ilskog,et al.  And then they lived sustainably ever after?--Assessment of rural electrification cases by means of indicators , 2008 .

[5]  Wendy Annecke,et al.  Monitoring and evaluation of energy for development: The good, the bad and the questionable in M&E practice , 2008 .

[6]  Theo Haupt,et al.  Key performance indicators and assessment methods for infrastructure sustainability—a South African construction industry perspective , 2007 .

[7]  Fred Carden,et al.  Outcome mapping : building learning and reflection into development programs [Arabic version] , 2002 .

[8]  Daniel Spreng,et al.  Energy use and energy access in relation to poverty , 2003 .

[9]  G. Munda,et al.  Qualitative multicriteria evaluation for environmental management , 1994 .

[10]  S. Mirasgedis,et al.  A multiple criteria decision-aid approach in defining national priorities for greenhouse gases emissions reduction in the energy sector , 2003, Eur. J. Oper. Res..

[11]  Jaya Bhanot,et al.  Moving towards tangible decision-making tools for policy makers: Measuring and monitoring energy access provision , 2012 .

[12]  Richard Margoluis,et al.  Monitoring and Evaluation in Conservation: a Review of Trends and Approaches , 2005 .

[13]  Yusuke Kamiya,et al.  An impact evaluation of the safe motherhood promotion project in Bangladesh: evidence from Japanese aid-funded technical cooperation. , 2013, Social science & medicine.

[14]  Julia C. Terrapon-Pfaff,et al.  How effective are small-scale energy interventions in developing countries? Results from a post-evaluation on project-level , 2014 .

[15]  Evan A. Thomas,et al.  Beyond Broken Pumps and Promises: Rethinking Intent and Impact in Environmental Health , 2017 .

[16]  P. Yu A Class of Solutions for Group Decision Problems , 1973 .

[17]  Toshihiko Nakata,et al.  Assessment of access to electricity and the socio-economic impacts in rural areas of developing countries , 2008 .

[18]  Jingyi Han,et al.  Small-scale bioenergy projects in rural China: Lessons to be learnt , 2008 .

[19]  V. Stevanovic,et al.  Sustainable development of the Belgrade energy system , 2009 .

[20]  Jiangjiang Wang,et al.  Review on multi-criteria decision analysis aid in sustainable energy decision-making , 2009 .

[21]  Naim Afgan,et al.  Multi-criteria evaluation for CHP system options , 2006 .

[22]  E. Georgopoulou,et al.  A multicriteria decision aid approach for energy planning problems: The case of renewable energy option , 1997 .

[23]  G. Rasul Food, water, and energy security in South Asia: A nexus perspective from the Hindu Kush Himalayan region☆ , 2014 .

[24]  Elisabeth Ilskog,et al.  Indicators for assessment of rural electrification--An approach for the comparison of apples and pears , 2008 .

[25]  James S. Dyer,et al.  Attribute weighting methods and decision quality in the presence of response error: A simulation study , 1998 .

[26]  D. McDonald,et al.  Developing guidelines to enhance the evaluation of overseas development projects. , 1999, Evaluation and program planning.

[27]  Takehiro Hatakeyama,et al.  Sustainable development indicators: Conceptual frameworks of comparative indicators sets for local administrations in Japan , 2018 .

[28]  A. Azapagic,et al.  Life cycle environmental impacts of UK shale gas , 2014 .

[29]  Stefano Corsi,et al.  The Environmental and Social Impact Assessment: a further step towards an integrated assessment process , 2015 .

[30]  Benjamin K. Sovacool,et al.  Conceptualizing and evaluating best practices in electricity and water regulatory governance , 2011 .

[31]  Maurizio Cellura,et al.  Decision making in energy planning: the ELECTRE multicriteria analysis approach compared to a FUZZY-SETS methodology , 1998 .

[32]  D. French,et al.  Sustainable Development Goals , 2021, Encyclopedia of the UN Sustainable Development Goals.

[33]  L. eng,et al.  Sustainable livelihoods guidance sheets , 1999 .

[34]  Pute Wu,et al.  A framework of indicator system for zoning of agricultural water and land resources utilization: A case study of Bayan Nur, Inner Mongolia , 2014 .

[35]  G. Hong,et al.  Sustainability assessment of renewable energy projects for off-grid rural electrification: The Pangan-an Island case in the Philippines , 2012 .

[36]  Stefan Bouzarovski,et al.  A global perspective on domestic energy deprivation: Overcoming the energy poverty-fuel poverty binary , 2015 .

[37]  Alessio Ishizaka,et al.  How to derive priorities in AHP: a comparative study , 2006, Central Eur. J. Oper. Res..

[38]  J. Martins The Impact of the Use of Energy Sources on the Quality of Life of Poor Communities , 2005 .

[39]  Nain H. Afgan,et al.  Sustainability assessment of a hybrid energy system , 2008 .

[40]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[41]  F. Ellis Rural livelihood diversity in developing countries: evidence and policy implications , 1999 .

[42]  W. Savedoff,et al.  Results-based aid in the energy sector : an analytical guide , 2015 .

[43]  Kyran O'Sullivan,et al.  Energy Policies and Multitopic Household Surveys: Guidelines for Questionnaire Design in Living Standards Measurement Studies , 2006 .

[44]  A. Brent,et al.  Renewable rural electrification: Sustainability assessment of mini-hybrid off-grid technological systems in the African context , 2010 .

[45]  Petros A. Pilavachi,et al.  Multicriteria evaluation of power plants impact on the living standard using the analytic hierarchy process , 2008 .

[46]  Harro von Blottnitz,et al.  Renewable energy for sustainable urban development: Redefining the concept of energisation , 2010 .

[47]  Mikiko Cars Project Evaluation in Development Cooperation: A Meta-Evaluative Case Study in Tanzania , 2006 .

[48]  Naim Afgan,et al.  MULTI-CRITERIA ASSESSMENT OF NEW AND RENEWABLE ENERGY POWER PLANTS , 2002 .

[49]  Maurizio Cellura,et al.  Decision-making in energy planning. Application of the Electre method at regional level for the diffusion of renewable energy technology , 2003 .

[50]  Narasimha D. Rao,et al.  Assessing rural energy sustainability in developing countries , 2014 .

[51]  Bernardo B. N. Strassburg,et al.  Evaluating impacts of development and conservation projects using sustainability indicators: Opportunities and challenges , 2014 .

[52]  Toshihiko Nakata,et al.  Analysis of the energy access improvement and its socio-economic impacts in rural areas of developing countries , 2007 .

[53]  Naim Afgan,et al.  Sustainability assessment of hydrogen energy systems , 2004 .

[54]  Olivier Serrat,et al.  Sustainable Livelihoods Approach , 2008 .

[55]  Benjamin K. Sovacool,et al.  Who governs energy? The challenges facing global energy governance , 2009 .

[56]  Aie,et al.  World Energy Outlook 2011 , 2001 .

[57]  Alberto Giretti,et al.  Environmental impacts related to the commissioning and usage phase of an intelligent energy management system , 2015 .

[58]  Jiangjiang Wang,et al.  Integrated evaluation of distributed triple-generation systems using improved grey incidence approach , 2008 .

[59]  Danae Diakoulaki,et al.  Multi-criteria decision analysis and cost–benefit analysis of alternative scenarios for the power generation sector in Greece , 2007 .

[60]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[61]  Patricia Jaramillo,et al.  Energy supply for sustainable rural livelihoods. A multi-criteria decision-support system , 2007 .

[62]  Zhixin Yu,et al.  Life-cycle assessment of energy consumption and environmental impact of an integrated food waste-based biogas plant , 2015 .

[63]  Mingchu Luo,et al.  Mixed-methods design for an objective-based evaluation of a magnet school assistance project , 2005 .

[64]  J. De Ruyck,et al.  Environmental impact of decentralized power generation in Santa Clara City, Cuba: An integrated assessment based on technological and human health risk indicators , 2013 .

[65]  Ana Delicado,et al.  Community perceptions of renewable energies in Portugal: Impacts on environment, landscape and local development , 2016 .

[66]  A. Hezri,et al.  Sustainability indicator system and policy processes in Malaysia: a framework for utilisation and learning. , 2004, Journal of environmental management.

[67]  Adisa Azapagic,et al.  Environmental impacts of microgeneration: Integrating solar PV, Stirling engine CHP and battery storage , 2015 .

[68]  Kathryn McKelvey,et al.  Uncertainty in energy planning: Estimating the health impacts of air pollution from fossil fuel electricity generation , 2015 .

[69]  Claudia F. Benham,et al.  Change, opportunity and grief: Understanding the complex social-ecological impacts of Liquefied Natural Gas development in the Australian coastal zone , 2016 .

[70]  J. Baker Evaluating the Impact of Development Projects on Poverty: A Handbook for Practitioners , 2000 .

[71]  Stefano Tarantola,et al.  Handbook on Constructing Composite Indicators: Methodology and User Guide , 2005 .

[72]  Marcos Aurélio Vasconcelos de Freitas,et al.  Evaluation of the impact of access to electricity: A comparative analysis of South Africa, China, India and Brazil , 2011 .

[73]  T. Saaty Decision making — the Analytic Hierarchy and Network Processes (AHP/ANP) , 2004 .

[74]  Tetsuo Tezuka,et al.  Proposing an evaluation framework for energy policy making incorporating equity: Applications in Australia , 2016 .

[75]  Vijay Modi,et al.  Measuring energy poverty: Focusing on what matters , 2012 .

[76]  Laura B. Rawlings,et al.  Impact Evaluation in Practice: Second Edition , 2010 .

[77]  Thomas L. Saaty,et al.  Decision-making with the AHP: Why is the principal eigenvector necessary , 2003, Eur. J. Oper. Res..

[78]  M. Diesendorf,et al.  Model for empowering rural poor through renewable energy technologies in Bangladesh , 2001 .

[79]  Laia Ferrer-Martí,et al.  Evaluating and comparing three community small-scale wind electrification projects , 2012 .

[80]  Patricia Jaramillo,et al.  A multicriteria approach to sustainable energy supply for the rural poor , 2012, Eur. J. Oper. Res..

[81]  Rodolfo Dufo-López,et al.  Tecno-economic assessment of an off-grid PV-powered community kitchen for developing regions , 2012 .

[82]  Emanuela Colombo,et al.  Minimization of local impact of energy systems through exergy analysis , 2013 .

[83]  Benjamin K. Sovacool,et al.  The importance of comprehensiveness in renewable electricity and energy-efficiency policy , 2009 .

[84]  R. Chambers,et al.  Sustainable rural livelihoods: Practical concepts for the 21st century , 1992 .

[85]  Aie,et al.  World Energy Outlook 2013 , 2013 .

[86]  D. Diakoulaki,et al.  Multicriteria analysis vs. externalities assessment for the comparative evaluation of electricity generation systems , 1997 .

[87]  Rowan Bedggood,et al.  Social impact assessments: Developing a consolidated conceptual framework , 2015 .

[88]  I. Kelman,et al.  Arctic petroleum’s community impacts: Local perceptions from Hammerfest, Norway , 2016 .

[89]  Christopher J. Koroneos,et al.  Optimization of energy production system in the Dodecanese Islands , 2005 .

[90]  Gunther Bensch,et al.  Impact evaluation of productive use—An implementation guideline for electrification projects , 2011 .

[91]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .