Verification and validation of simulation models

This paper discusses verification and validation of simulation models. The different approaches to deciding model validity are presented; how model verification and validation relate to the model development process are discussed; various validation techniques are defined; conceptual model validity, model verification, operational validity, and data validity are described; ways to document results are given; and a recommended procedure is presented.

[1]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[2]  Saul I. Gass,et al.  Documenting a Computer-Based Model , 1984 .

[3]  Tuncer Ören,et al.  Concepts and criteria to assess acceptability of simulation studies: a frame of reference , 1981, CACM.

[4]  Robert G. Sargent,et al.  An expository on verification and validation of simulation models , 1985, WSC '85.

[5]  George B. Kleindorfer,et al.  THE PHILOSOPHY OF SCIENCE AND VAL1DATION IN SIMULATION , 1993 .

[6]  Jack P. C. Kleijnen,et al.  EUROPEAN JOURNAL OF OPERATIONAL , 1992 .

[7]  Osman Balci,et al.  Validation fo Multivariate Response Trace-driven Simulation Models , 1983, Performance.

[8]  Robert G. Sargent,et al.  Simulation Model Validation , 1984 .

[9]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .

[10]  Robert G. Sargent,et al.  Investigation Into Scheduling for an Interactive Computing System , 1974, IBM J. Res. Dev..

[11]  David O. Wood,et al.  MIT model analysis program: what we have learned about policy model review , 1986, WSC '86.

[12]  Bernard P. Zeigler,et al.  Theory of Modelling and Simulation , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Jack P. C. Kleijnen,et al.  Strategic directions in verification, validation, and accreditation research , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[14]  M Panjabi,et al.  Validation of mathematical models. , 1979, Journal of biomechanics.

[15]  Janina Decker,et al.  Simulation and Model-Based Methodologies: An Integrative View , 1984, NATO ASI Series.

[16]  Robert G. Sargent,et al.  The use of graphical models in model validation , 1986, WSC '86.

[17]  Robert G. Sargent.  Validation of simulation models , 1979, WSC '79.

[18]  Peter L. Knepell,et al.  Simulation Validation: A Confidence Assessment Methodology , 1993 .

[19]  J. Banks,et al.  Discrete-Event System Simulation , 1995 .

[20]  I. A. Chaikovsky,et al.  Terminology for model credibility , 1979 .

[21]  Osman Balci,et al.  Verification, validation, and accreditation , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[22]  J. Kleijnen,et al.  Validation of models: statistical techniques and data availability , 1999, WSC'99. 1999 Winter Simulation Conference Proceedings. 'Simulation - A Bridge to the Future' (Cat. No.99CH37038).

[23]  Osman Balci,et al.  Some examples of simulation model validation using hypothesis testing , 1982, WSC.

[24]  Robert G Sargent,et al.  An Assessment Procedure and a Set of Criteria for Use in the Evaluation of 'Computerized Models and Computer-Based Modelling Tools'. , 1981 .

[25]  J. Kleijnen.  Statistical tools for simulation practitioners , 1986 .

[26]  BalciOsman,et al.  A methodology for cost-risk analysis in the statistical validation of simulation models , 1981 .

[27]  Saul I. Gass,et al.  Concepts of model confidence , 1981, Comput. Oper. Res..

[28]  Osman Balci,et al.  A methodology for cost-risk analysis in the statistical validation of simulation models , 1981, CACM.

[29]  Saul I. Gass,et al.  Feature Article - Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis , 1983, Oper. Res..

[30]  R. Sargent,et al.  Validation of Simulation Models via Simultaneous Confidence Intervals , 1984 .

[31]  Thomas H. Naylor,et al.  Verification of Computer Simulation Models , 1967 .

[32]  Dale K. Pace,et al.  Naval Modeling and Simulation Verification, Validation, and Accreditation , 1993, Proceedings of 1993 Winter Simulation Conference - (WSC '93).

[33]  Lee W. Schruben,et al.  Establishing the credibility of simulations , 1980 .

[34]  Osman Balci,et al.  Guidelines For Selecting And Using Simulation Model Verification Techniques , 1989, 1989 Winter Simulation Conference Proceedings.

[35]  O. Balci.  How to assess the acceptability and credibility of simulation results , 1989, WSC '89.

[36]  Saul I. Gass,et al.  Model accreditation: A rationale and process for determining a numerical rating , 1993 .

[37]  Robert G. Sargent,et al.  A tutorial on verification and validation of simulation models (1984) , 1984, WSC '07.

[38]  Julie C. Lowery.  Design of hospital admissions scheduling system using simulation , 1996, Winter Simulation Conference.

[39]  Robert G. Sargent.  Some subjective validation methods using graphical displays of data , 1996, Winter Simulation Conference.

[40]  Saul I. Gass,et al.  Letter to the Editor - Guidelines for Model Evaluation: An Abridged Version of the U.S. General Accounting Office Exposure Draft , 1980, Oper. Res..

[41]  Osman Balci,et al.  Validation of Multivariate Response Simulation Models by Using Hotelling's Two-sample T^2 Test , 1981 .