Validation and verification of social processes within agent-based computational organization models

The use of simulation modeling in computational analysis of organizations is becoming a prominent approach in social science research. However, relying on simulations to gain intuition about social phenomena has significant implications. While simulations may give rise to interesting macro-level phenomena, and sometimes even mimic empirical data, the underlying micro and macro level processes may be far from realistic. Yet, this realism may be important to infer results that are relevant to existing theories of social systems and to policy making. Therefore, it is important to assess not only predictive capability but also explanation accuracy of formal models in terms of the degree of realism reflected by the embedded processes. This paper presents a process-centric perspective for the validation and verification (V&V) of agent-based computational organization models. Following an overview of the role of V&V within the life cycle of a simulation study, emergent issues in agent-based organization model V&V are outlined. The notion of social contract that facilitates capturing micro level processes among agents is introduced to enable reasoning about the integrity and consistency of agent-based organization designs. Social contracts are shown to enable modular compositional verification of interaction dynamics among peer agents. Two types of consistency are introduced: horizontal and vertical consistency. It is argued that such local consistency analysis is necessary, but insufficient to validate emergent macro processes within multi-agent organizations. As such, new formal validation metrics are introduced to substantiate the operational validity of emergent macro-level behavior.

[1]  Derek L. Nazareth,et al.  Issues in the Verification of Knowledge in Rule-Based Systems , 1989, Int. J. Man Mach. Stud..

[2]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[3]  William J. Kolarik,et al.  Toward an Integrated Framework for Modeling , 2004 .

[4]  Kathleen M. Carley Computational organizational science and organizational engineering , 2002, Simul. Model. Pract. Theory.

[5]  Jeannette M. Wing,et al.  A behavioral notion of subtyping , 1994, TOPL.

[6]  Jacques Ferber,et al.  Multi-agent systems - an introduction to distributed artificial intelligence , 1999 .

[7]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[8]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[9]  Timothy G. Trucano,et al.  Verification and validation. , 2005 .

[10]  Jan van Leeuwen,et al.  Handbook of Theoretical Computer Science, Vol. A: Algorithms and Complexity , 1994 .

[11]  Michael J. Prietula,et al.  Computational organization theory , 1994 .

[12]  Heinz-Otto Peitgen,et al.  The science of fractal images , 2011 .

[13]  David Sankoff,et al.  Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison , 1983 .

[14]  Jeannette M. Wing,et al.  Model checking software systems: a case study , 1995, SIGSOFT FSE.

[15]  R. Hegselmann,et al.  Simulating Social Phenomena , 1997 .

[16]  Maurice Landry,et al.  Revisiting the issue of model validation in OR: An epistemological view , 1993 .

[17]  Muhittin Oral,et al.  The facets of the modeling and validation process in operations research , 1993 .

[18]  William J. Kolarik,et al.  Toward an integrated framework for modeling enterprise processes , 2004, CACM.

[19]  Gerhard Weiss,et al.  Computational Organization Theory , 2000 .

[20]  Yoav Shoham,et al.  Agent-Oriented Programming , 1992, Artif. Intell..

[21]  Alan H. Bond,et al.  Distributed Artificial Intelligence , 1988 .

[22]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[23]  Richard E. Nance The Conical Methodology: A Framework for Simulation Model Development , 1987 .

[24]  M WingJeannette,et al.  Model checking software systems , 1995 .

[25]  Alfred V. Aho,et al.  Algorithms for Finding Patterns in Strings , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[26]  Kathleen M. Carley,et al.  An Algorithmic Approach to the Comparison of Partially Labeled Graphs , 2002 .

[27]  Ghislain Vansteenkiste,et al.  Computer-aided modelling and simulation , 1982 .

[28]  R. D. Smet Simulation modeling and analysis (2nd edition): Averill M. Law and W. David Kelton McGraw-Hill, Inc., New York, 1991, xxii + 759 pages, £31.05, ISBN 0 07 036698 5 , 1993 .

[29]  Garth R. MacKenzie,et al.  Verification Technology Potential with Different Modeling and Simulation Development and Implementation Paradigms , 2002 .

[30]  Kathleen M. Carley,et al.  The nature of the social agent , 1994 .

[31]  Robert Axelrod,et al.  Advancing the art of simulation in the social sciences , 1997, Complex..

[32]  Robert G. Sargent,et al.  Validation and verification of simulation models , 1999, Proceedings of the 2004 Winter Simulation Conference, 2004..

[33]  Daniel E. O'Leary,et al.  Expert system verification and validation: a survey and tutorial , 1993, Artificial Intelligence Review.

[34]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[35]  Osman Balci,et al.  Credibility Assessment of Simulation Results: The State of the Art , 1986 .

[36]  Osman Balci,et al.  Formulated problem verification as an explicit requirement of model credibility , 1985 .

[37]  Robert L. Axtell,et al.  The New Coevolution of Information Science and Social Science : From Software Agents to Artificial Societies and Back or How More Computing Became Different Computing , 2003 .

[38]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .

[39]  Les Gasser,et al.  Organizations as Complex, Dynamic Design Problems , 1993, EPIA.

[40]  Edmund M. Clarke,et al.  Model checking and abstraction , 1994, TOPL.

[41]  Graham A. Stephen String Searching Algorithms , 1994, Lecture Notes Series on Computing.

[42]  Shane S. Sturrock,et al.  Time Warps, String Edits, and Macromolecules – The Theory and Practice of Sequence Comparison . David Sankoff and Joseph Kruskal. ISBN 1-57586-217-4. Price £13.95 (US$22·95). , 2000 .

[43]  Thomas H. Naylor,et al.  Verification of Computer Simulation Models , 1967 .

[44]  M. Mitchell Waldrop,et al.  Complexity : the emerging science and the edge of order and chaos , 1992 .

[45]  S. Kauffman At Home in the Universe: The Search for the Laws of Self-Organization and Complexity , 1995 .