How to do structural validity of a system dynamics type simulation model: The case of an energy policy model

System dynamics based simulation models are becoming increasingly popular in the analysis of important energy policy issues including global warming, deregulation, conservation and efficiency. The usefulness of these models is predicated on their ability to link observable patterns of behavior of a system to micro-level structures. This paper argues that the structural validity of the simulation model--right behavior for the right reasons--is a stringent measure to build confidence in a simulation model regardless of how well the model passes behavior validity tests. That leads to an outline of formal structural validity procedures available but less explored in system dynamics modeling 'repertoire'. An illustration of a set of six tests for structural validity of a system dynamics model for energy policy analysis follows. Then using Theil inequality statistics, the behavior validity of the model is also tested. Finally, some conclusions on the increased appeal for simulation models for energy policy analysis and design are presented.

[1]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[2]  John D. Sterman,et al.  System Dynamics: Systems Thinking and Modeling for a Complex World , 2002 .

[3]  Hassan Qudrat-Ullah,et al.  Understanding the dynamics of electricity supply, resources and pollution: Pakistan's case , 2001 .

[4]  Geoff Coyle,et al.  The validation of commercial system dynamics models , 2000 .

[5]  Saul I. Gass,et al.  Feature Article - Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis , 1983, Oper. Res..

[6]  C F Hermann.  Validation problems in games and simulations with special reference to models of international politics. , 1967, Behavioral science.

[7]  George B. Kleindorfer,et al.  Validation in Simulation: Various Positions in the Philosophy of Science , 1998 .

[8]  Andrew Ford,et al.  Using system dynamics for conservation policy analysis in the Pacific Northwest , 1989 .

[9]  Isaac Dyner,et al.  System Dynamics Modelling for Residential Energy Efficiency Analysis and Management , 1995 .

[10]  Rogelio Oliva,et al.  Model calibration as a testing strategy for system dynamics models , 2003, Eur. J. Oper. Res..

[11]  Hassan Qudrat-Ullah,et al.  MDESRAP: a model for understanding the dynamics of electricity supply, resources and pollution , 2005 .

[12]  Hans Jochen Scholl,et al.  Agent-based and system dynamics modeling: a call for cross study and joint research , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[13]  Paul N. Finlay,et al.  The Paucity of Model Validation in Operational Research Projects , 1987 .

[14]  Y. Barlas.  Multiple tests for validation of system dynamics type of simulation models , 1989 .

[15]  Erik R. Larsen,et al.  Sensitivity of reserve margin to factors influencing investment behaviour in the electricity market of England and Wales , 1992 .

[16]  C. S. Holling.  Adaptive Environmental Assessment and Management , 2005 .

[17]  Erling Moxnes,et al.  Interfuel substitution in OECD‐European electricity production , 1990 .

[18]  Martin Greenberger,et al.  Models in the policy process , 1976 .

[19]  Jay W. Forrester,et al.  “The” model versus a modeling “process” , 1985 .

[20]  J. N. R. Jeffers,et al.  Mathematical Models in Ecology. , 1972 .

[21]  George P. Richardson,et al.  A petroleum life cycle model for the United States with endogenous technology, exploration, recovery, and demand , 1990 .

[22]  Roger F. Naill,et al.  A System Dynamics Model for National Energy Policy Planning , 1992 .

[23]  C. Hall,et al.  Ecosystem Modeling in Theory and Practice: An Introduction with Case Histories , 1990 .

[24]  Robert L. Eberlein,et al.  Simplification and understanding of models , 1989 .

[25]  Richard L. Van Horn,et al.  Validation of Simulation Results , 1971 .

[26]  D. M. Geraghty,et al.  Feedback loops: the effect of external agents on utility performance , 1982 .

[27]  James R. Emshoff,et al.  Design and use of computer simulation models , 1970 .

[28]  Pradeep K. Gupta,et al.  Strategic management and planning for electric utilities , 1985 .

[29]  Yaman Barlas,et al.  Philosophical roots of model validation: Two paradigms , 1990 .

[30]  Andrew Ford,et al.  Using simulation for policy evaluation in the electric utility industry , 1983 .