A systematic map on verification and validation of emergent behavior in software engineering research

Abstract Context: Emergent behavior cannot be attributed to one individual system alone but arises in the interplay of various systems, components etc. Ensuring the correctness of emergent behavior is a well-known challenge that has been addressed by research in various subfields of software engineering. Objective: This paper aims at providing a unified view on the research activities conducted and research contributions made on verification and validation of emergent behavior. Methods: We have conducted a systematic mapping study on the topic of verification and validation of emergent behavior. We applied a combined search strategy using manual, database, and snowball search. In total we investigated 7211 papers, from these 168 relevant papers have been included and classified. Results: Results show an increasing interest in the topic of verification and validation of emergent behavior. As only little validation and evaluation research has been conducted, the field can be considered still immature. There exist different verification and validation techniques used in the various solution approaches such as model checking, simulation, or runtime monitoring. It stands out that even though research is published in different software engineering fields and subfields no verification or validation technique can be attributed solely to a single field.

[1]  NavimipourNima Jafari,et al.  Formal verification approaches and standards in the cloud computing , 2018 .

[2]  Chao Yang,et al.  A Multi-Perspective Method for Analysis of Cooperative Behaviors Among Industrial Devices of Smart Factory , 2017, IEEE Access.

[3]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..

[4]  Jiafu Wan,et al.  A time-recordable cross-layer communication protocol for the positioning of Vehicular Cyber-Physical Systems , 2016, Future Gener. Comput. Syst..

[5]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[6]  Iván García-Magariño,et al.  Robust design of multi-agent system interactions: A testing approach based on pattern matching , 2013, Eng. Appl. Artif. Intell..

[7]  Logesh Ravi,et al.  A hybrid quantum-induced swarm intelligence clustering for the urban trip recommendation in smart city , 2017, Future Gener. Comput. Syst..

[8]  Ali Shahrokni,et al.  A systematic review of software robustness , 2013, Inf. Softw. Technol..

[9]  Vahid Garousi,et al.  A systematic mapping study of web application testing , 2013, Inf. Softw. Technol..

[10]  Tim Menzies,et al.  Verification and Validation and Artificial Intelligence , 2005, Adv. Comput..

[11]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[12]  Catholijn M. Jonker,et al.  Compositional Verification of Multi-Agent Systems: A Formal Analysis of Pro-activeness and Reactiveness , 2002, Int. J. Cooperative Inf. Syst..

[13]  Jemal H. Abawajy,et al.  Intelligent, smart and scalable cyber-physical systems , 2019, J. Intell. Fuzzy Syst..

[14]  Giancarlo Fortino,et al.  Cyber-physical systems, internet of things and big data , 2017, Future Gener. Comput. Syst..

[15]  Valdemar Vicente Graciano Neto.  Validating Emergent Behaviours in Systems-of-Systems through Model Transformations , 2016, SRC@MoDELS.

[16]  Lei Pan,et al.  Automatic extraction and integration of behavioural indicators of malware for protection of cyber-physical networks , 2019, Future Gener. Comput. Syst..

[17]  Keqin Li,et al.  A message efficient intersection control algorithm for intelligent transportation in smart cities , 2017, Future Gener. Comput. Syst..

[18]  Kai Petersen,et al.  Worldviews, Research Methods, and their Relationship to Validity in Empirical Software Engineering Research , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[19]  H. Arksey,et al.  Scoping studies: towards a methodological framework , 2005 .

[20]  Danny Weyns,et al.  A survey of formal methods in self-adaptive systems , 2012, C3S2E '12.

[21]  Armin Biere,et al.  Bounded Model Checking Using Satisfiability Solving , 2001, Formal Methods Syst. Des..

[22]  Yong Meng Teo,et al.  Post-mortem analysis of emergent behavior in complex simulation models , 2013, SIGSIM PADS '13.

[23]  Jeffrey C. Mogul,et al.  Emergent (mis)behavior vs. complex software systems , 2006, EuroSys.

[24]  Ali Selamat,et al.  Agent systems verification : systematic literature review and mapping , 2018, Applied intelligence (Boston).

[25]  Behrouz H. Far,et al.  Detection and verification of a new type of emergent behavior in multiagent systems , 2013, 2013 IEEE 17th International Conference on Intelligent Engineering Systems (INES).

[26]  Michael Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[27]  Ross Gore,et al.  Applying causal inference to understand emergent behavior , 2008, 2008 Winter Simulation Conference.

[28]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[29]  Kunikazu Kobayashi,et al.  Cooperative Behavior Acquisition in Multi-agent Reinforcement Learning System Using Attention Degree , 2012, ICONIP.

[30]  Nima Jafari Navimipour,et al.  Formal verification approaches and standards in the cloud computing: A comprehensive and systematic review , 2018, Comput. Stand. Interfaces.

[31]  H. Dieter Rombach,et al.  Research Collaborations between Academia and Industry , 2007, Future of Software Engineering (FOSE '07).

[32]  Michael Fisher,et al.  Verifying autonomous systems , 2013, CACM.

[33]  Chao Yang,et al.  A 3-Layer Method for Analysis of Cooperative Behaviors of Physical Devices in Cyber-Physical Systems , 2017, WASA.

[34]  Behrouz Homayoun Far,et al.  A Technique and a Tool to Detect Emergent Behavior of Distributed Systems Using Scenario-Based Specifications , 2010, 2010 22nd IEEE International Conference on Tools with Artificial Intelligence.

[35]  Fatemeh Hendijani Fard.  Detecting and fixing emergent behaviors in Distributed Software Systems using a message content independent method , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[36]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[37]  Sandro Morasca,et al.  systematic review on the functional testing of semantic web services , 2013 .

[38]  Jiang Lu,et al.  Robust Cyber-Physical Systems: Concept, models, and implementation , 2016, Future Gener. Comput. Syst..

[39]  Hareton K. N. Leung,et al.  Regression Testing of Web Service: A Systematic Mapping Study , 2014, ACM Comput. Surv..

[40]  INGOLF H. KRÜGER,et al.  Interaction-based Runtime Verification for Systems of Systems Integration , 2010, J. Log. Comput..

[41]  Marian Daun,et al.  On Different Search Methods for Systematic Literature Reviews and Maps: Experiences from a Literature Search on Validation and Verification of Emergent Behavior , 2018, EASE.

[42]  Pierre N. Robillard,et al.  Performing Systematic Literature Reviews With Novices: An Iterative Approach , 2014, IEEE Transactions on Education.

[43]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[44]  G. R. Gangadharan,et al.  Model Checking Based Web Service Verification: A Systematic Literature Review , 2018 .

[45]  Zhengping Li,et al.  A Survey of Emergent Behavior and Its Impacts in Agent-based Systems , 2006, 2006 4th IEEE International Conference on Industrial Informatics.

[46]  Vahid Garousi,et al.  Web application testing: A systematic literature review , 2014, J. Syst. Softw..

[47]  Marian Daun,et al.  On the Benefits of using Dedicated Models in Validation Processes for Behavioral Specifications , 2019, 2019 IEEE/ACM International Conference on Software and System Processes (ICSSP).

[48]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[49]  Pearl Brereton,et al.  A systematic review of systematic review process research in software engineering , 2013, Inf. Softw. Technol..

[50]  Behrouz Homayoun Far,et al.  Visualizing the network of software agents for verification of Multiagent Systems , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[51]  Tamio Arai,et al.  Cooperative behavior of multiple robots by chain of monolithic policies for two robots , 2008 .

[52]  Michael Rovatsos,et al.  Data mining agent conversations: A qualitative approach to multiagent systems analysis , 2013, Inf. Sci..

[53]  V VasilakosAthanasios,et al.  Robust Cyber-Physical Systems , 2016 .

[54]  Pierre-Yves Schobbens,et al.  Feature interaction in software product line engineering: A systematic mapping study , 2018, Inf. Softw. Technol..

[55]  Flávio Oquendo,et al.  Formal Verification of Software-Intensive Systems Architectures Described with Piping and Instrumentation Diagrams , 2016, ECSA.

[56]  Juan Julián Merelo Guervós,et al.  Studying real traffic and mobility scenarios for a Smart City using a new monitoring and tracking system , 2017, Future Gener. Comput. Syst..

[57]  Jonas Johansson,et al.  An approach for modelling interdependent infrastructures in the context of vulnerability analysis , 2010, Reliab. Eng. Syst. Saf..