Verification of Continuous and Hybrid Systems ARCH-COMP 19 Repeatability Evaluation Report

This report presents the results of the repeatability evaluation for the 3rd International Competition on Verifying Continuous and Hybrid Systems (ARCH-COMP’19). The competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2019, affiliated with the Cyber-Physical Systems and Internet of Things (CPS-IoT Week’19). In its third edition, twenty-five tools submitted artifacts through a Git repository for the repeatability evaluation, applied to solve benchmark problems for eight competition categories. The majority of participants adhered to new requirements for this year’s repeatability evaluation, namely to submit scripts to automatically install and execute tools in containerized virtual environments (specifically Dockerfiles to execute within Docker). The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable.

[1]  Tobias Nipkow,et al.  A Proof Assistant for Higher-Order Logic , 2002 .

[2]  Tiziano Villa,et al.  Ariadne: a framework for reachability analysis of hybrid automata , 2006 .

[3]  André Platzer,et al.  KeYmaera: A Hybrid Theorem Prover for Hybrid Systems (System Description) , 2008, IJCAR.

[4]  You Li,et al.  BACH : Bounded ReAchability CHecker for Linear Hybrid Automata , 2008, 2008 Formal Methods in Computer-Aided Design.

[5]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[6]  Davide Bresolin,et al.  Assume–guarantee verification of nonlinear hybrid systems with Ariadne , 2014 .

[7]  Sergiy Bogomolov,et al.  HYST: a source transformation and translation tool for hybrid automaton models , 2015, HSCC.

[8]  Nathan Fulton,et al.  KeYmaera X: An Axiomatic Tactical Theorem Prover for Hybrid Systems , 2015, CADE.

[9]  Fabian Immler,et al.  Verified Reachability Analysis of Continuous Systems , 2015, TACAS.

[10]  Matthias Althoff,et al.  Implementation of Interval Arithmetic in CORA 2016 , 2016, ARCH@CPSWeek.

[11]  Stefan Kowalewski,et al.  HyPro: A C++ Library of State Set Representations for Hybrid Systems Reachability Analysis , 2017, NFM.

[12]  Anne Remke,et al.  HYPEG: Statistical Model Checking for hybrid Petri nets: Tool Paper , 2017, VALUETOOLS.

[13]  Stanley Bak,et al.  HyLAA: A Tool for Computing Simulation-Equivalent Reachability for Linear Systems , 2017, HSCC.

[14]  Sergiy Bogomolov,et al.  Reach Set Approximation through Decomposition with Low-dimensional Sets and High-dimensional Matrices , 2018, HSCC.

[15]  Weiming Xiang,et al.  Output Reachable Set Estimation and Verification for Multilayer Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Gidon Ernst,et al.  Two-Layered Falsification of Hybrid Systems Guided by Monte Carlo Tree Search , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[17]  Rupak Majumdar,et al.  Approximate Time Bounded Reachability for CTMCs and CTMDPs: A Lyapunov Approach , 2018, QEST.

[18]  Hao Ma,et al.  Interacting Particle System-based Estimation of Reach Probability for a Generalized Stochastic Hybrid System , 2018, ADHS.

[19]  Weiming Xiang,et al.  Reachability Analysis and Safety Verification for Neural Network Control Systems , 2018, ArXiv.

[20]  Alessandro Abate,et al.  StocHy: automated verification and synthesis of stochastic processes , 2019, ArXiv.

[21]  Sergiy Bogomolov,et al.  JuliaReach: a toolbox for set-based reachability , 2019, HSCC.

[22]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.