Experiences from Large-Scale Model Checking: Verifying a Vehicle Control System with NuSMV

In the age of autonomously driving vehicles, functionality and complexity of embedded systems are increasing tremendously. Safety aspects become more important and require such systems to operate with the highest possible level of fault tolerance. Simulation and systematic testing techniques have reached their limits in this regard. Here, formal verification as a long established technique can be an appropriate complement. However, the necessary preparatory work like adequately modeling a system and specifying properties in temporal logic are anything but trivial. In this paper, we report on our experiences applying model checking to verify the arbitration logic of a Vehicle Control System. We balance pros and cons of different model checking techniques and tools, and reason about our choice of the symbolic model checker NuSMV. We describe the process of modeling the architecture, resulting in ~1500 LOC, 69 state variables and 38 LTL constraints. To handle this large-scale model, we automate and optimize the model checking procedure for use on multi-core CPUs and employ Bounded Model Checking to avoid the state explosion problem. We share our lessons learned and provide valuable insights for architects, developers, and test engineers involved in this highly present topic.

[1]  Eun-Young Kang,et al.  Model-based Verification and Validation of an Autonomous Vehicle System , 2018, ArXiv.

[2]  Dilian Gurov,et al.  Formal Verification in Automotive Industry: Enablers and Obstacles , 2018, ISoLA.

[3]  M. Petró‐Turza,et al.  The International Organization for Standardization. , 2003 .

[4]  Kim G. Larsen,et al.  A Tutorial on Uppaal , 2004, SFM.

[5]  Kristin Yvonne Rozier,et al.  Linear Temporal Logic Symbolic Model Checking , 2011, Comput. Sci. Rev..

[6]  Marco Pistore,et al.  NuSMV 2: An OpenSource Tool for Symbolic Model Checking , 2002, CAV.

[7]  Stefan Wagner,et al.  A Safety Argumentation for Fail-Operational Automotive Systems in Compliance with ISO 26262 , 2019, 2019 4th International Conference on System Reliability and Safety (ICSRS).

[8]  Edmund M. Clarke,et al.  Model Checking and the State Explosion Problem , 2011, LASER Summer School.

[9]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[10]  Benoît Fraikin,et al.  Comparison of Model Checking Tools for Information Systems , 2010, ICFEM.

[11]  Virginie Wiels,et al.  Experiences in using model checking to verify real time properties of a landing gear control system , 2005 .

[12]  Chao Wang,et al.  Abstraction Refinement for Large Scale Model Checking (Series on Integrated Circuits and Systems) , 2006 .

[13]  B. R. Mehta,et al.  Functional safety and safety instrumented systems , 2015 .

[14]  Rong Gu,et al.  Formal Verification of an Autonomous Wheel Loader by Model Checking , 2018, 2018 IEEE/ACM 6th International FME Workshop on Formal Methods in Software Engineering (FormaliSE).

[15]  Yunja Choi,et al.  From NuSMV to SPIN: Experiences with model checking flight guidance systems , 2007, Formal Methods Syst. Des..

[16]  Stefan Leue,et al.  Automated Functional Safety Analysis of Automated Driving Systems , 2018, FMICS.

[17]  Stefan Wagner,et al.  An Approach for Structuring a Highly Automated Driving Multiple Channel Vehicle System for Safety Analysis , 2018, 2018 3rd International Conference on System Reliability and Safety (ICSRS).

[18]  Gerard J. Holzmann,et al.  The SPIN Model Checker - primer and reference manual , 2003 .

[19]  Armin Biere,et al.  Bounded model checking , 2003, Adv. Comput..

[20]  Stefan Wagner,et al.  Integrated Safety Analysis Using Systems-Theoretic Process Analysis and Software Model Checking , 2015, SAFECOMP.

[21]  Ana Maria Ambrosio,et al.  Integrating model checking and model based testing for industrial software development , 2019, Comput. Ind..

[22]  Alexander Kron,et al.  Motion control solutions for automated driving systems at BMW , 2019 .

[23]  Alexandre M. Bayen,et al.  VERIFICATION OF HYBRID SYSTEMS , 2004 .

[24]  Stefan Wagner,et al.  Formal Verification of a Fail-Operational Automotive Driving System , 2021, ArXiv.

[25]  Niklas Eén,et al.  SAT Based Model Checking , 2005 .

[26]  Andreas Herkersdorf,et al.  Fail-operational in safety-related automotive multi-core systems , 2015, 10th IEEE International Symposium on Industrial Embedded Systems (SIES).

[27]  Jussi Lahtinen Verification of Fault-Tolerant System Architectures Using Model Checking , 2014, SAFECOMP Workshops.