Towards a framework for more robust validation and verification of simulation models for systems of systems

We present a framework for verification and validation of simulation models of System of Systems that is based on an existing framework for modeling and simulation. The framework addresses problems arising especially in recently emerging Systems of Systems such as cyber-physical autonomous cooperative systems. The design of such systems presents challenges to the currently employed independent use of simplified models for formal verification or brute-force simulations which are severely limited in the range of conditions they can test. The proposed framework is applied to integration of formal analytic and simulation verification methods where there is a need to have confidence that the properties proved for idealized abstract models also hold in more realistic models which gave rise to the abstractions. Taking both logical and probabilistic perspectives clarifies the situation and suggests where more research is needed.

[1]  James J. Nutaro,et al.  Building Software for Simulation: Theory and Algorithms, with Applications in C++ , 2010 .

[2]  Andreas Tolk,et al.  Challenges of Combat Modeling and Distributed Simulation , 2012 .

[3]  D. Küçükkeçeci Çetinkaya.  Model Driven Development of Simulation Models : Defining and Transforming Conceptual Models into Simulation Models by Using Metamodels and Model Transformations , 2013 .

[4]  Bernard P. Zeigler.  STRUCTURING THE ORGANIZATION OF PARTIAL MODELS , 1978 .

[5]  Osman Balci.  Golden Rules of Verification, Validation, Testing, and Certification of Modeling and Simulation Applications , 2010 .

[6]  Andreas Tolk,et al.  Engineering Principles of Combat Modeling and Distributed Simulation: Tolk/Engineering Principles Combat Modeling , 2012 .

[7]  Bernard P. Zeigler,et al.  Combining DEVS and model-checking: using systems morphisms for integrating simulation and analysis in model engineering , 2014 .

[8]  J. M. Voogd,et al.  The Generic Methodology for Verification and Validation to support acceptance of models, simulations and data , 2013 .

[9]  Bernard P. Zeigler,et al.  Theory of modeling and simulation , 1976 .

[10]  Eric W. Weisel,et al.  A Decision-Theoretic Approach to Defining Use for Computer Simulation , 2012 .

[11]  Osman Balci,et al.  Verification, validation, and accreditation , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[12]  Andreas Tolk,et al.  Verification and Validation , 2012 .

[13]  Klaus Aachen.  Fundamentals Of Performance Evaluation Of Computer And Telecommunications Systems , 2016 .

[14]  Joachim Denil,et al.  Design, verification and deployment of software-intensive systems : a multi-paradigm modelling approach , 2013 .

[15]  Bernard P. Zeigler,et al.  Guide to Modeling and Simulation of Systems of Systems , 2012, SpringerBriefs in Computer Science.

[16]  Liu Bao-hong,et al.  A Formal Description Specification for Multi-Resolution Modeling Based on DEVS Formalism and its Applications , 2007 .

[17]  Sudhir Aggarwal,et al.  Ergodic machines--probabilistic and approximate homomorphic simplifications. , 1975 .

[18]  Laura R. Humphrey.  Model Checking for Verification in UAV Cooperative Control Applications , 2013 .

[19]  José Luis Risco-Martín,et al.  Model-driven systems engineering for netcentric system of systems with DEVS unified process , 2013, 2013 Winter Simulations Conference (WSC).

[20]  Mikel D. Petty.  Verification and Validation , 2008 .

[21]  Alexander Verbraeck,et al.  MDD4MS: a model driven development framework for modeling and simulation , 2011, SCSC 2011.

[22]  Gabriel A. Wainer,et al.  Principles of Discrete Event System Specification model verification , 2013, Simul..

[23]  Celso Leandro Palma,et al.  Principles of Modeling and Simulation: A Multidisciplinary Approach , 2009 .

[24]  Zhang Li,et al.  Model Engineering for Complex System Simulation , 2013 .

[25]  Christel Baier,et al.  Principles of model checking , 2008 .

[26]  Osman Balci,et al.  A life cycle for modeling and simulation , 2012, Simul..

[27]  Kangsun Lee,et al.  Dynamic model abstraction , 1996, Proceedings Winter Simulation Conference.

[28]  D. Caughlin,et al.  Verification, validation, and accreditation (VV&A) of models and simulations through reduced order metamodels , 1995, Winter Simulation Conference Proceedings, 1995..

[29]  D. Pace.  Modeling and simulation verification and validation challenges , 2004 .

[30]  A. Wayne Wymore,et al.  Model-based systems engineering : an introduction to the mathematical theory of discrete systems and to the tricotyledon theory of system design , 1993 .

[31]  Mikel D. Petty,et al.  Validity of Models and Classes of Models in Semantic Composability , 2003 .

[32]  Ireneusz Sierocki,et al.  A NOTE ON STRUCTURAL INFERENCE IN SYSTEMS THEORY , 1986 .

[33]  Dale K Pace,et al.  Comprehensive consideration of uncertainty in simulation use , 2013 .

[34]  Russell R. Barton,et al.  Proceedings of the 2000 winter simulation conference , 2000 .

[35]  Mohammad Jamshidi,et al.  System of systems engineering : innovations for the 21st century , 2008 .

[36]  Gabriel Wainer,et al.  Principles of DEVS Model Verification for Real-Time Embedded Applications , 2012 .

[37]  Paul K. Davis,et al.  Experiments In Multiresolution Modeling (MRM) , 1998 .

[38]  A. Wayne Wymore,et al.  A mathematical theory of systems engineering--the elements , 1967 .

[39]  Saurabh Mittal,et al.  Netcentric System of Systems Engineering with DEVS Unified Process , 2013 .

[40]  Hans Vangheluwe,et al.  Foundations of Modelling and Simulation of Complex Systems , 2008, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[41]  Mamadou Kaba Traoré,et al.  Capturing the dual relationship between simulation models and their context , 2006, Simul. Model. Pract. Theory.

[42]  Osman Balci,et al.  A methodology for cost-risk analysis in the statistical validation of simulation models , 1981, CACM.

[43]  Bernard P. Zeigler,et al.  DEVS representation and aggregation of spatially distributed systems: speed-versus-error tradeoffs , 1996 .

[44]  Osman Balci,et al.  Well-defined intended uses: an explicit requirement for accreditation of modeling and simulation applications , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[45]  Bernard P. Zeigler,et al.  DEVS Framework for Modelling, Simulation, Analysis, and Design of Hybrid Systems , 1994, Hybrid Systems.

[46]  M. Janishidi.  System of Systems - Innovations for 21st Century , 2008, 2008 IEEE Region 10 and the Third international Conference on Industrial and Information Systems.

[47]  Bernard P. Zeigler,et al.  System theoretic foundations of modeling and simulation: a historic perspective and the legacy of A Wayne Wymore , 2012, Simul..

[48]  Z. C. Roza,et al.  Simulation Fidelity Theory and Practice , 2005 .

[49]  Gabriel A. Wainer,et al.  A Formal Framework for Stochastic Discrete Event System Specification Modeling and Simulation , 2010, Simul..