Model checking intelligent avionics systems for test cases generation using multi-agent systems

Abstract The paper contributes by introducing a novel, formal and operational approach that addresses the open challenging issues of modeling, verifying, and testing intelligent critical avionics systems. We advance the state-of-the-art by unifying the three challenges and considering the intelligence, autonomy, and accountability of the components as first citizen concepts. The proposed methodology is effectively applied to a real, practical and complex case study of intelligent avionics systems, namely the landing gear system and uses multi-agent systems to model each main component in the system as an intelligent agent. We also introduce the formalism of extended interpreted systems that supports intelligence, autonomy, communication, input and output actions, predicate conditions and post-conditions. The paper adopts the computation tree logic of conditional commitments to model communication among autonomous agents and trace its progress. The symbolic model checker of this logic is used to run the verification of the system model, encoded in an extended input language, against coverage criteria and properties. Furthermore, we introduce a new testing methodology that: 1) Follows a test-driven development approach; 2) performs unit testing, component testing, and system testing in each increment; and 3) uses model checking to generate automatically counterexamples and witness traces interpreted into concrete test suites that achieve new coverage criteria. The experimental results showed the efficiency and scalability of the developed approach against a transformation-based technique. Finally, the computational complexity of the developed approach is analysed.

[1]  Felix Kossak Landing Gear System: An ASM-Based Solution for the ABZ Case Study , 2014, ABZ.

[2]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[3]  Paolo Arcaini,et al.  Offline Model-Based Testing and Runtime Monitoring of the Sensor Voting Module , 2014, ABZ.

[4]  Leonardo de Moura,et al.  Automated Test Generation with SAL , 2005 .

[5]  John M. Rushby,et al.  New challenges in certification for aircraft software , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[6]  Michael Leuschel,et al.  Validation of the ABZ Landing Gear System Using ProB , 2014, ABZ.

[7]  Loe M. G. Feijs,et al.  Test Generation for Intelligent Networks Using Model Checking , 1997, TACAS.

[8]  Richard Banach The Landing Gear Case Study in Hybrid Event-B , 2014, ABZ.

[9]  Philippe Dhaussy,et al.  Context-Aware Verification of a Landing Gear System , 2014, ABZ.

[10]  Michael Winikoff,et al.  Developing intelligent agent systems - a practical guide , 2004, Wiley series in agent technology.

[11]  Frédéric Boniol,et al.  The Landing Gear System Case Study , 2014, ABZ.

[12]  Antonio F. Gómez-Skarmeta,et al.  ACLAnalyser: A Tool for Debugging Multi-Agent Systems , 2004, ECAI.

[13]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[14]  Javier Tuya,et al.  Generating Test Cases Specifications for BPEL Compositions of Web Services Using SPIN , 2006 .

[15]  Jamal Bentahar,et al.  Formal Specification and Automatic Verification of Conditional Commitments , 2015, IEEE Intelligent Systems.

[16]  Michael J. Butler,et al.  Co-simulation Environment for Rodin: Landing Gear Case Study , 2014, ABZ.

[17]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[18]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[19]  Jamal Bentahar,et al.  Specifying and verifying contract-driven service compositions using commitments and model checking , 2017, Expert Syst. Appl..

[20]  Paolo Arcaini,et al.  AsmetaF: A Flattener for the ASMETA Framework , 2018, F-IDE@FLoC.

[21]  Jamal Bentahar,et al.  On the soundness, completeness and applicability of the logic of knowledge and communicative commitments in multi-agent systems , 2016, Expert Syst. Appl..

[22]  Amel Mammar,et al.  Modeling a Landing Gear System in Event-B , 2014, ABZ.

[23]  Raymond A. Paul,et al.  Automated model checking and testing for composite Web services , 2005, Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC'05).

[24]  Jorge J. Gómez-Sanz,et al.  Testing and Debugging of MAS Interactions with INGENIAS , 2009, AOSE.

[25]  Jamal Bentahar,et al.  Reducing model checking commitments for agent communication to model checking ARCTL and GCTL* , 2013, Autonomous Agents and Multi-Agent Systems.

[26]  Jamal Bentahar,et al.  Communicative commitments: Model checking and complexity analysis , 2012, Knowl. Based Syst..

[27]  Jamal Bentahar,et al.  Computationally Grounded Quantitative Trust with Time , 2020, AAMAS.

[28]  Jamal Bentahar,et al.  Model checking real-time conditional commitment logic using transformation , 2018, J. Syst. Softw..

[29]  Jean-Raymond Abrial,et al.  Aircraft Landing Gear System: Approaches with Event-B to the Modeling of an Industrial System , 2014, ABZ.

[30]  Insup Lee,et al.  Data flow testing as model checking , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[31]  Kim G. Larsen,et al.  Time-Optimal Real-Time Test Case Generation Using Uppaal , 2003, FATES.

[32]  Oguz Dikenelli,et al.  Goal-Oriented Agent Testing Revisited , 2009, AOSE.

[33]  Thomas A. Henzinger,et al.  Software Verification with BLAST , 2003, SPIN.

[34]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[35]  Paolo Arcaini,et al.  Modeling and Analyzing Using ASMs: The Landing Gear System Case Study , 2014, ABZ.

[36]  Leonardo Mendonça de Moura,et al.  Generating efficient test sets with a model checker , 2004, Proceedings of the Second International Conference on Software Engineering and Formal Methods, 2004. SEFM 2004..

[37]  Jamal Bentahar,et al.  Specification and automatic verification of trust-based multi-agent systems , 2018, Future Gener. Comput. Syst..

[38]  Lisa Crispin,et al.  Agile Testing: A Practical Guide for Testers and Agile Teams , 2008 .

[39]  Gregory Zoughbi,et al.  Modeling safety and airworthiness (RTCA DO-178B) information: conceptual model and UML profile , 2011, Software & Systems Modeling.

[40]  Gordon Fraser,et al.  Testing with model checkers: a survey , 2009 .

[41]  Silvano Dal-Zilio,et al.  Model-Checking Real-Time Properties of an Aircraft Landing Gear System Using Fiacre , 2014, ABZ.

[42]  Barry W. Boehm Software Engineering , 1976, IEEE Transactions on Computers.

[43]  Lin Padgham,et al.  Automated Unit Testing for Agent Systems , 2007, ENASE.

[44]  Marco Pistore,et al.  NuSMV 2: An OpenSource Tool for Symbolic Model Checking , 2002, CAV.

[45]  Paolo Arcaini,et al.  Closing the gap between the specification and the implementation: the ASMETA way , 2018, Models: Concepts, Theory, Logic, Reasoning and Semantics.

[46]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[47]  Paolo Arcaini,et al.  AsmetaSMV: A Way to Link High-Level ASM Models to Low-Level NuSMV Specifications , 2010, ASM.

[48]  Jamal Bentahar,et al.  Conditional Commitments , 2014, ACM Trans. Softw. Eng. Methodol..

[49]  Yannick Moy,et al.  Testing or Formal Verification: DO-178C Alternatives and Industrial Experience , 2013, IEEE Software.

[50]  Alex Groce,et al.  Adaptive Model Checking , 2002, Log. J. IGPL.

[51]  Dominique Méry,et al.  Modeling an Aircraft Landing System in Event-B , 2014, ABZ.

[52]  Ioan Alfred Letia,et al.  Assuring safety in air traffic control systems with argumentation and model checking , 2016, Expert Syst. Appl..