A qualitative evaluation approach for energy system modelling frameworks

BackgroundThe research field of energy system analysis is faced with the challenge of increasingly complex systems and their sustainable transition. The challenges are not only on a technical level but also connected to societal aspects. Energy system modelling plays a decisive role in this field, and model properties define how useful it is in regard to the existing challenges. For energy system models, evaluation methods exist, but we argue that many decisions upon properties are rather made on the model generator or framework level. Thus, this paper presents a qualitative approach to evaluate frameworks in a transparent and structured way regarding their suitability to tackle energy system modelling challenges.MethodsCurrent main challenges and framework properties that potentially contribute to tackle these challenges are derived from a literature review. The resulting contribution matrix and the described application procedure is then applied exemplarily in a case study in which the properties of the Open Energy Modelling Framework are checked for suitability to each challenge.ResultsWe identified complexity (1), scientific standards (2), utilisation (3), interdisciplinary modelling (4), and uncertainty (5) as the main challenges. We suggest three major property categories of frameworks with regard to their capability to tackle the challenges: open-source philosophy (1), collaborative modelling (2), and structural properties (3).General findings of the detailed mapping of challenges and properties are that an open-source approach is a pre-condition for complying with scientific standards and that approaches to tackle the challenges complexity and uncertainty counteract each other. More research in the field of complexity reduction within energy system models is needed. Furthermore, while framework properties can support to address problems of result communication and interdisciplinary modelling, an important part can only be addressed by communication and organisational structures, thus, on a behavioural and social level.ConclusionsWe conclude that the relevance of energy system analysis tools needs to be reviewed critically. Their suitability for tackling the identified challenges deserves to be emphasised. The approach presented here is one contribution to improve current evaluation methods by adding this aspect.

[1]  B. Sovacool.  What Are We Doing Here? Analyzing Fifteen Years of Energy Scholarship and Proposing a Social Science Research Agenda , 2014 .

[2]  Semida Silveira,et al.  OSeMOSYS: The Open Source Energy Modeling System: An introduction to its ethos, structure and development , 2011 .

[3]  Dedan Kimathi,et al.  Assessing Quality of Open Source Software Based on Community Metrics , 2015 .

[4]  Iain Staffell,et al.  The importance of open data and software: Is energy research lagging behind? , 2017 .

[5]  Varun Rai,et al.  Agent-Based Modeling of Energy Technology Adoption: Empirical Integration of Social, Behavioral, Economic, and Environmental Factors , 2014, Environ. Model. Softw..

[6]  Nicole D. Sintov,et al.  Should we quit our jobs? Challenges, barriers and recommendations for interdisciplinary energy research , 2017 .

[7]  Tom Brown,et al.  PyPSA: Python for Power System Analysis , 2017, 1707.09913.

[8]  Journals unite for reproducibility , 2014, Nature.

[9]  Madeleine Fogde,et al.  Unpacking the Water-Energy-Food Nexus: Tools for Assessment and Cooperation Along a Continuum , 2013 .

[10]  Alexander Zerrahn,et al.  Long-run power storage requirements for high shares of renewables: review and a new model , 2017 .

[11]  Joseph F. DeCarolis,et al.  Using modeling to generate alternatives (MGA) to expand our thinking on energy futures , 2011 .

[12]  Marion Christ,et al.  Wind Energy Scenarios for the Simulation of the German Power System Until 2050: The Effect of Social and Ecological Factors , 2016, EnviroInfo.

[13]  Alastair R. Buckley,et al.  A review of energy systems models in the UK: Prevalent usage and categorisation , 2016 .

[14]  Mark Aberdour.  A people-focused , 2022 .

[15]  Maryse Labriet,et al.  ETSAP-TIAM: the TIMES integrated assessment model Part I: Model structure , 2008, Comput. Manag. Sci..

[16]  H. Dowlatabadi.  Integrated assessment models of climate change: An incomplete overview , 1995 .

[17]  J. DeCarolis,et al.  The case for repeatable analysis with energy economy optimization models , 2012, 2001.10858.

[18]  Neil Strachan,et al.  Characterising the Evolution of Energy System Models Using Model Archaeology , 2014, Environmental Modeling & Assessment.

[19]  Neil Strachan,et al.  Reinventing the energy modelling–policy interface , 2016, Nature Energy.

[20]  Katrin Schaber,et al.  Integration of Variable Renewable Energies in the European power system: a model-based analysis of transmission grid extensions and energy sector coupling , 2014 .

[21]  Mark Jennings,et al.  A review of urban energy system models: Approaches, challenges and opportunities , 2012 .

[22]  Simon Hilpert,et al.  The core concept of the Open Energy Modelling Framework (oemof) , 2016, EnviroInfo.

[23]  Clemens Wingenbach,et al.  An open source energy system simulation model as an instrument for public participation in the development of strategies for a sustainable future , 2014 .

[24]  Uwe Krien,et al.  The Open Energy Modelling Framework (oemof) - A new approach to facilitate open science in energy system modelling , 2018, Energy Strategy Reviews.

[25]  Nouredine Hadjsaid,et al.  Modelling the impacts of variable renewable sources on the power sector: Reconsidering the typology of energy modelling tools , 2015 .

[26]  Joseph F. DeCarolis,et al.  Modeling for insight using Tools for Energy Model Optimization and Analysis (Temoa) , 2013 .

[27]  M. Kerkhof,et al.  Learning and stakeholder participation in transition processes towards sustainability: Methodological considerations , 2005 .

[28]  M. Jefferson.  Closing the gap between energy research and modelling, the social sciences, and modern realities , 2014 .

[29]  Benjamin K. Sovacool,et al.  Looking the wrong way: Bias, renewable electricity, and energy modelling in the United States , 2016 .

[30]  Edward S. Rubin,et al.  A review of uncertainties in technology experience curves , 2012 .

[31]  Liz Varga,et al.  Energy and complexity: New ways forward , 2015 .

[32]  Roland De Guio,et al.  Modelling and uncertainties in integrated energy planning , 2015 .

[33]  Jens Saak,et al.  Best Practices for Replicability, Reproducibility and Reusability of Computer-Based Experiments Exemplified by Model Reduction Software , 2016, ArXiv.

[34]  Thomas Pregger,et al.  Model experiments and comparison for simulation of pathways to a fully renewable energy supply , 2017 .

[35]  Tobias Wittmann,et al.  Agent-Based Models of Energy Investment Decisions , 2008 .

[36]  Eleni Castro,et al.  Open Journal Systems and Dataverse Integration– Helping Journals to Upgrade Data Publication for Reusable Research , 2015 .

[37]  Clemens Wingenbach,et al.  AC Power Flow Simulations within an Open Data Model of a High Voltage Grid , 2016, EnviroInfo.

[38]  Gregory F. Nemet,et al.  Beyond the learning curve: factors influencing cost reductions in photovoltaics , 2006 .

[39]  Heidi Heinrichs,et al.  Benefits and challenges of participatory methods in qualitative energy scenario development , 2018 .

[40]  Darrel C. Ince,et al.  The case for open computer programs , 2012, Nature.

[41]  S. Pfenninger,et al.  Renewables, nuclear, or fossil fuels? Scenarios for Great Britain’s power system considering costs, emissions and energy security , 2015 .

[42]  Justin P. Johnson,et al.  Collaboration, Peer Review and Open Source Software , 2004, Inf. Econ. Policy.

[43]  N. V. Beeck.  Classification of Energy Models , 1999 .

[44]  J. A. Wardekker,et al.  Uncertainty communication in environmental assessments: views from the Dutch science-policy interface , 2008 .

[45]  Francesco Moresino,et al.  Coupling techno-economic energy models with behavioral approaches , 2017, Oper. Res..

[46]  Anna Ernst,et al.  Integrated assessment of a phase-out of coal-fired power plants in Germany , 2017 .

[47]  John P. Weyant,et al.  Modeling for insights, not numbers: the experiences of the energy modeling forum , 1982 .

[48]  Danny Pudjianto,et al.  Linking a storyline with multiple models: A cross-scale study of the UK power system transition , 2014 .

[49]  Christian Dieckhoff.  Modellierte Zukunft: Energieszenarien in der wissenschaftlichen Politikberatung , 2015 .

[50]  Brian Vad Mathiesen,et al.  A review of computer tools for analysing the integration of renewable energy into various energy systems , 2010 .

[51]  Henri Moll,et al.  A review of the bandwidth and environmental discourses of future energy scenarios: Shades of green and gray , 2017 .

[52]  B. Li,et al.  Modelling to generate alternatives with an energy system optimization model , 2016, Environ. Model. Softw..

[53]  Dieter Gerten,et al.  Ethical aspects in the economic modeling of water policy options , 2015 .

[54]  Jonathan G. Koomey,et al.  WHAT CAN HISTORY TEACH US? A Retrospective Examination of Long-Term Energy Forecasts for the United States* , 2002 .

[55]  Marion Christ,et al.  Integrating Social Acceptance of Electricity Grid Expansion into Energy System Modeling: A Methodological Approach for Germany , 2016, EnviroInfo.

[56]  Holger R. Maier,et al.  Future research challenges for incorporation of uncertainty in environmental and ecological decision-making , 2008 .

[57]  Felix Cebulla,et al.  Raising awareness in model-based energy scenario studies—a transparency checklist , 2016 .

[58]  Nate Blair,et al.  RE-ASSUME: A Decision Maker's Guide to Evaluating Energy Scenarios, Modeling, and Assumptions , 2013 .

[59]  S. Pfenninger.  Energy scientists must show their workings , 2017, Nature.

[60]  Iain Staffell,et al.  Opening the black box of energy modelling: strategies and lessons learned , 2017, ArXiv.

[61]  Frank C. Krysiak,et al.  The Demand Side in Economic Models of Energy Markets: The Challenge of Representing Consumer Behavior , 2015, Front. Energy Res..

[62]  Patrik Söderholm,et al.  Empirical challenges in the use of learning curves for assessing the economic prospects of renewable energy technologies , 2007 .

[63]  David O. Wood,et al.  Energy System Modeling and Forecasting , 1976 .

[64]  Varun Rai,et al.  Determinants of Spatio-Temporal Patterns of Energy Technology Adoption: An Agent-Based Modeling Approach , 2014 .

[65]  Benjamin K. Sovacool,et al.  The uniqueness of the energy security, justice, and governance problem☆ , 2012 .

[66]  On Inaccuracies in a Published Journal Article , 2016 .

[67]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1962 .

[68]  O. Edenhofer,et al.  Cartography of pathways: A new model for environmental policy assessments , 2015 .

[69]  Bernd Möller,et al.  Heat Roadmap Europe: Combining district heating with heat savings to decarbonise the EU energy system , 2014 .

[70]  Robbie Morrison,et al.  Energy system modeling: Public transparency, scientific reproducibility, and open development , 2018 .

[71]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .

[72]  Evangelos Panos,et al.  Meta-Analysis of Energy Scenario Studies: Example of Electricity Scenarios for Switzerland , 2015 .

[73]  Adam Hawkes,et al.  Energy systems modeling for twenty-first century energy challenges , 2014 .

[74]  Frauke Wiese.  renpass Renewable Energy Pathways Simulation System - Open Source as an approach to meet challenges in energy modeling , 2015 .