Independent Verification and Validation of an Industrial Simulation Model

The independent verification and validation (IV&V) of simulation models is largely restricted to applications in the military and public policy domains. There is little evidence of IV&V for industrial simulation models. This is largely because industrial simulations are normally of a much smaller scale and do not warrant a full IV&V. A procedure for IV&V of industrial simulation models is described that provides a viable alternative where the cost and time of IV&V must be contained. The procedure consists of the following activities: structured walkthrough, review of model assumptions, code examination, review of verification procedures, replications analysis, review of static analysis, review of output reporting, and investigation of results and experimentation. The IV&V of a Sellafield Limited supply chain model is described.

[1]  Ming Yang,et al.  Study on simulation credibility metrics , 2005, Proceedings of the Winter Simulation Conference, 2005..

[2]  William F. Waite,et al.  VERIFICATION , VALIDATION , AND ACCREDITATION OF HIGH LEVEL ARCHITECTURE FEDERATES AND FEDERATIONS-A VV & A STRATEGY FOR LEGACY MODELS , SIMULATIONS , ALGORITHMS AND SOFTWARE COMPONENTS - , 2012 .

[3]  Robert G Sargent,et al.  An Assessment Procedure and a Set of Criteria for Use in the Evaluation of 'Computerized Models and Computer-Based Modelling Tools'. , 1981 .

[4]  Tuncer Ören,et al.  Concepts and criteria to assess acceptability of simulation studies: a frame of reference , 1981, CACM.

[5]  Saul I. Gass,et al.  Feature Article - Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis , 1983, Oper. Res..

[6]  Osman Balci Guidelines for successful simulation studies , 1990, 1990 Winter Simulation Conference Proceedings.

[7]  Stewart Robinson,et al.  Measuring Service Quality in the Process of Delivering a Simulation Study: The Customer's Perspective , 1998 .

[8]  John A. Tufarolo,et al.  A case study of verification, validation, and accreditation for advanced distributed simulation , 1997, TOMC.

[9]  Barry Boehm,et al.  Characteristics of software quality , 1978 .

[10]  Saul I. Gass,et al.  An Assessment Procedure for Simulation Models: A Case Study , 1991, Oper. Res..

[11]  S. Wall,et al.  Retrieval of intermediate-level waste at Trawsfynydd nuclear power station , 2002 .

[12]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[13]  Michael Pidd,et al.  Provider and customer expectations of successful simulation projects , 1998, J. Oper. Res. Soc..

[14]  James E. Coolahan,et al.  Simulation Verification, Validation, and Accreditation , 2000 .

[15]  Sandi Delack Expanding Our Horizons , 2010, NASN school nurse.

[16]  Jack P. C. Kleijnen,et al.  EUROPEAN JOURNAL OF OPERATIONAL , 1992 .

[17]  Stewart Robinson,et al.  Simulation: The Practice of Model Development and Use , 2004 .

[18]  Osman Balci,et al.  A collaborative evaluation environment for credibility assessment of modeling and simulation applications , 2002, Proceedings of the Winter Simulation Conference.

[19]  James D. Arthur,et al.  Independent verification and validation: a missing link in simulation methodology? , 1996, Winter Simulation Conference.

[20]  Paul K. Davis,et al.  Generalizing Concepts and Methods of Verification, Validation, and Accreditation (VV&A) for Military Simulations, , 1992 .

[21]  Saul I. Gass,et al.  Model accreditation: A rationale and process for determining a numerical rating , 1993 .

[22]  Richard E. Nance,et al.  A specification language to assist in analysis of discrete event simulation models , 1985, CACM.

[23]  Dale K. Pace Naval modeling and simulation verification, validation, and accreditation , 1993, WSC '93.

[24]  Tuncer I. Ören,et al.  Quality Assurance in Modelling and Simulation: A Taxonomy , 1984 .

[25]  Osman Balci,et al.  A methodology for certification of modeling and simulation applications , 2001, TOMC.

[26]  James D. Arthur,et al.  Improving the model development process: expanding our horizons in verification, validation, and accreditation research and practice , 2002, WSC '02.

[27]  Gerald T. Mackulak,et al.  Simulation Project Characteristics in Industrial Settings , 1995 .

[28]  Jennifer Park,et al.  Verification, validation, and accreditation (VV&A): one voice --- unified, common & cross-cutting , 2007, SCSC.

[29]  Stewart Robinson,et al.  Modes of simulation practice: approaches to business and military simulation , 2002, Simul. Model. Pract. Theory.

[30]  Stewart Robinson,et al.  Simulation verification, validation and confidence: a tutorial , 1999 .

[31]  P. Sanders,et al.  DoD Modeling and Simulation (M&S) Verification, Validation, and Accreditation (VV&A), , 1996 .

[32]  Osman Balci,et al.  Verification, Validation, and Testing , 2007 .

[33]  Dale K. Pace,et al.  Naval Modeling and Simulation Verification, Validation, and Accreditation , 1993, Proceedings of 1993 Winter Simulation Conference - (WSC '93).

[34]  Saul I. Gass,et al.  Concepts of model confidence , 1981, Comput. Oper. Res..

[35]  Zicai Wang,et al.  The HITVICE VV&A environment , 2005, Proceedings of the Winter Simulation Conference, 2005..

[36]  Saul I. Gass,et al.  Evaluation of complex models , 1977, Comput. Oper. Res..

[37]  Heston R. Hicks,et al.  Comparison of models: ex post facto validation/acceptance? , 1992, WSC '92.

[38]  Osman Balci,et al.  Quality assessment, verification, and validation of modeling and simulation applications , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[39]  James D. Arthur,et al.  Expanding our horizons in verification, validation, and accreditation research and practice , 2002, Proceedings of the Winter Simulation Conference.

[40]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .