Verifying and Validating Autonomous Systems: Towards an Integrated Approach

When applying formal verification to a system that interacts with the real world we must use a model of the environment. This model represents an abstraction of the actual environment, but is necessarily incomplete and hence presents an issue for system verification. If the actual environment matches the model, then the verification is correct; however, if the environment falls outside the abstraction captured by the model, then we cannot guarantee that the system is well-behaved. A solution to this problem consists in exploiting the model of the environment for statically verifying the system’s behaviour and, if the verification succeeds, using it also for validating the model against the real environment via runtime verification. The paper discusses this approach and demonstrates its feasibility by presenting its implementation on top of a framework integrating the Agent Java PathFinder model checker. Trace expressions are used to model the environment for both static formal verification and runtime verification.

[1]  Angelo Ferrando,et al.  Parametric Trace Expressions for Runtime Verification of Java-Like Programs , 2017, FTfJP@ECOOP.

[2]  Michael Fisher,et al.  Declarative Abstractions for Agent Based Hybrid Control Systems , 2010, DALT.

[3]  Rafael H. Bordini,et al.  Model checking agent programming languages , 2012, Automated Software Engineering.

[4]  Angelo Ferrando,et al.  Managing Bad AIPs with RIVERtools , 2018, PAAMS.

[5]  Michael Wooldridge,et al.  Programming Multi-Agent Systems in AgentSpeak using Jason (Wiley Series in Agent Technology) , 2007 .

[6]  Angelo Ferrando,et al.  Coping with Bad Agent Interaction Protocols When Monitoring Partially Observable Multiagent Systems , 2018, PAAMS.

[7]  Brink van der Merwe,et al.  Verifying android applications using Java PathFinder , 2012, ACM SIGSOFT Softw. Eng. Notes.

[8]  Rafael Heitor Bordini,et al.  Verifying Multi-agent Programs by Model Checking , 2006, Autonomous Agents and Multi-Agent Systems.

[9]  Angelo Ferrando,et al.  Comparing Trace Expressions and Linear Temporal Logic for Runtime Verification , 2016, Theory and Practice of Formal Methods.

[10]  George J. Pappas,et al.  Discrete abstractions of hybrid systems , 2000, Proceedings of the IEEE.

[11]  Affan Shaukat,et al.  Agent-Based Autonomous Systems and Abstraction Engines: Theory Meets Practice , 2016, TAROS.

[12]  Damien Zufferey,et al.  P: safe asynchronous event-driven programming , 2013, PLDI.

[13]  Angelo Ferrando,et al.  Parametric Runtime Verification of Multiagent Systems , 2017, AAMAS.

[14]  Paola Mello,et al.  Commitment Tracking via the Reactive Event Calculus , 2009, IJCAI.

[15]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[16]  Louise A. Dennis,et al.  The MCAPL Framework including the Agent Infrastructure Layer an Agent Java Pathfinder , 2018, J. Open Source Softw..

[17]  Angelo Ferrando,et al.  Global Protocols as First Class Entities for Self-Adaptive Agents , 2015, AAMAS.

[18]  Amir Pnueli,et al.  The temporal logic of programs , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[19]  Sanjit A. Seshia,et al.  Combining Model Checking and Runtime Verification for Safe Robotics , 2017, RV.

[20]  Michael Fisher,et al.  Practical verification of decision-making in agent-based autonomous systems , 2013, Automated Software Engineering.

[21]  John Penix,et al.  Verification of time partitioning in the DEOS scheduler kernel , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[22]  Angelo Ferrando,et al.  Recognising Assumption Violations in Autonomous Systems Verification , 2018, AAMAS.

[23]  Davide Ancona,et al.  Constrained global types for dynamic checking of protocol conformance in multi-agent systems , 2013, SAC '13.

[24]  Michael Fisher,et al.  Formal verification of autonomous vehicle platooning , 2016, Sci. Comput. Program..

[25]  Davide Ancona,et al.  Automatic Generation of Self-monitoring MASs from Multiparty Global Session Types in Jason , 2012, DALT.

[26]  Angelo Ferrando,et al.  Runtime verification of fail-uncontrolled and ambient intelligence systems: A uniform approach , 2015, Intelligenza Artificiale.

[27]  A. Prasad Sistla,et al.  Runtime Monitoring of Stochastic Cyber-Physical Systems with Hybrid State , 2011, RV.

[28]  Angelo Ferrando,et al.  The early bird catches the worm: First verify, then monitor! , 2019, Sci. Comput. Program..

[29]  Winfried Lamersdorf,et al.  Jadex: A BDI Reasoning Engine , 2005, Multi-Agent Programming.

[30]  Sergiy Bogomolov,et al.  Runtime Verification for Hybrid Analysis Tools , 2015, RV.

[31]  Davide Ancona,et al.  Towards Runtime Monitoring of Node.js and Its Application to the Internet of Things , 2017, ALP4IoT@iFM.

[32]  T. Henzinger.  The theory of hybrid automata , 1996, LICS 1996.

[33]  Evelina Lamma,et al.  The SCIFF Abductive Proof-Procedure , 2005, AI*IA.

[34]  Frédéric Boniol,et al.  Reducing State Explosion with Context Modeling for Model-Checking , 2011, 2011 IEEE 13th International Symposium on High-Assurance Systems Engineering.

[35]  Fausto Giunchiglia,et al.  Model Checking Multiagent Systems , 1998, J. Log. Comput..

[36]  Nelma Moreira,et al.  Automata for regular expressions with shuffle , 2018, Inf. Comput..

[37]  D. Ancona,et al.  Monitoring Patients with Hypoglycemia Using Self-adaptive Protocol-Driven Agents: A Case Study , 2016, EMAS@AAMAS.

[38]  Evelina Lamma,et al.  Modelling Interactions via Commitments and Expectations , 2009, Handbook of Research on Multi-Agent Systems.

[39]  Alessio Lomuscio,et al.  Automatic verification of multi-agent systems by model checking via ordered binary decision diagrams , 2007, J. Appl. Log..

[40]  Alessio Lomuscio,et al.  MCMAS: A Model Checker for Multi-agent Systems , 2006, TACAS.