Model Conformance for Cyber-Physical Systems

Model-based development is an important paradigm for developing cyber-physical systems (CPS). The underlying assumption is that the functional behavior of a model is related to the behavior of a more concretized model or the real system. A formal definition of such a relation is called conformance relation. There are a variety of conformance relations, and the question arises of how to select a conformance relation for the development of CPS. The contribution of this article is a survey of the definitions and algorithms of conformance relations for CPS. Additionally, the article compares several conformance relations and provides guidance on which relation to select for specific problems. Finally, we discuss how to select inputs for testing conformance.

[1]  Matthias Althoff,et al.  Reachability computation of low-order models for the safety verification of high-order road vehicle models , 2012, 2012 American Control Conference (ACC).

[2]  Paulo Tabuada,et al.  Compositional Abstractions of Hybrid Control Systems , 2004, Discret. Event Dyn. Syst..

[3]  Alexandre Donzé,et al.  Trajectory-based Verification and Controller Synthesis for Continuous and Hybrid Systems. (Trajectoires pour la Vérification et la Commande de Systèmes Dynamiques Continus et Hybrides) , 2007 .

[4]  Morteza Mohaqeqi,et al.  Sound Test-Suites for Cyber-Physical Systems , 2016, 2016 10th International Symposium on Theoretical Aspects of Software Engineering (TASE).

[5]  Kevin A. Grasse,et al.  Simulation Relations and Controllability Properties of Linear and Nonlinear Control Systems , 2015, SIAM J. Control. Optim..

[6]  Matthias Althoff,et al.  Reachability Analysis of Nonlinear Differential-Algebraic Systems , 2014, IEEE Transactions on Automatic Control.

[7]  Sriram Sankaranarayanan,et al.  Monte-carlo techniques for falsification of temporal properties of non-linear hybrid systems , 2010, HSCC '10.

[8]  Mohammad Reza Mousavi,et al.  A Tool Prototype for Model-Based Testing of Cyber-Physical Systems , 2015, ICTAC.

[9]  Stavros Tripakis,et al.  Towards Compositional Feedback in Non-Deterministic and Non-Input-Receptive Systems* , 2015, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[10]  Sergiy Bogomolov,et al.  Reach Set Approximation through Decomposition with Low-dimensional Sets and High-dimensional Matrices , 2018, HSCC.

[11]  Matthias Althoff,et al.  ARCH-COMP17 Category Report: Continuous Systems with Nonlinear Dynamics , 2017, ARCH@CPSWeek.

[12]  Matthias Althoff,et al.  Avoiding geometric intersection operations in reachability analysis of hybrid systems , 2012, HSCC '12.

[13]  Houssam Abbas,et al.  Formal property verification in a conformance testing framework , 2014, 2014 Twelfth ACM/IEEE Conference on Formal Methods and Models for Codesign (MEMOCODE).

[14]  Gordon Fraser,et al.  Coverage-based Testing Using Qualitative Reasoning Models , 2008, SEKE.

[15]  G. Smirnov.  Introduction to the Theory of Differential Inclusions , 2002 .

[16]  Björn Rüffer,et al.  Integral input-to-state stability of interconnected iISS systems by means of a lower-dimensional comparison system , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[17]  Paulo Tabuada,et al.  Bisimilar control affine systems , 2004, Syst. Control. Lett..

[18]  Stanley Bak,et al.  Simulation-Equivalent Reachability of Large Linear Systems with Inputs , 2017, CAV.

[19]  John Lygeros,et al.  Bisimulation for General Stochastic Hybrid Systems , 2005, HSCC.

[20]  Radu Grosu,et al.  Computing compositional proofs of Input-to-Output Stability using SOS optimization and δ-decidability , 2017 .

[21]  Maria Domenica Di Benedetto,et al.  Approximate equivalence and synchronization of metric transition systems , 2009, Syst. Control. Lett..

[22]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[23]  André Platzer,et al.  Differential Refinement Logic* , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[24]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[25]  Hongyan Tan,et al.  Approximate reachability and bisimulation equivalences for transition systems , 2016 .

[26]  Andrija T. Saric,et al.  Approximate Bisimulation-Based Reduction of Power System Dynamic Models , 2015, IEEE Transactions on Power Systems.

[27]  Yixiang Chen,et al.  Approximate Simulation for Metric Hybrid Input/Output Automata , 2011, 2011 Fifth International Conference on Secure Software Integration and Reliability Improvement - Companion.

[28]  G. Frehse,et al.  Assume-guarantee reasoning for hybrid I/O-automata by over-approximation of continuous interaction , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[29]  Yashwanth Singh Rahul Annapureddy,et al.  Ant colonies for Temporal Logic falsification of hybrid systems , 2010, IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society.

[30]  Augusto Sampaio,et al.  A Process for Sound Conformance Testing of Cyber-Physical Systems , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[31]  Alexandre Donzé,et al.  Breach, A Toolbox for Verification and Parameter Synthesis of Hybrid Systems , 2010, CAV.

[32]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems: Advanced Lectures (Lecture Notes in Computer Science) , 2005 .

[33]  A. Agung Julius,et al.  Approximate Abstraction of Stochastic Hybrid Automata , 2006, HSCC.

[34]  Matthias Althoff,et al.  Provably safe motion of mobile robots in human environments , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Thomas A. Henzinger,et al.  Quantifying Similarities Between Timed Systems , 2005, FORMATS.

[36]  Matthias Althoff,et al.  Reachset Conformance Testing of Hybrid Automata , 2016, HSCC.

[37]  Antoine Girard,et al.  Approximate bisimulation relations for constrained linear systems , 2007, Autom..

[38]  Matthias Althoff,et al.  STL Model Checking of Continuous and Hybrid Systems , 2016, ATVA.

[39]  Anthony G. Pipe,et al.  Systematic and Realistic Testing in Simulation of Control Code for Robots in Collaborative Human-Robot Interactions , 2015, TAROS.

[40]  Thomas A. Henzinger,et al.  Assume-Guarantee Reasoning for Hierarchical Hybrid Systems , 2001, HSCC.

[41]  Pieter J. L. Cuijpers,et al.  On Bicontinuous Bisimulation and the Preservation of Stability , 2007, HSCC.

[42]  Maria Domenica Di Benedetto,et al.  Bisimulation theory for switching linear systems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[43]  Antoine Girard,et al.  Approximate Simulation Relations for Hybrid Systems , 2008, Discret. Event Dyn. Syst..

[44]  Paulo Tabuada,et al.  Verification and Control of Hybrid Systems , 2009 .

[45]  Markus Roggenbach,et al.  Towards a unified view of bisimulation: a comparative study , 2000, Theor. Comput. Sci..

[46]  Michiel van Osch.  Hybrid Input-Output Conformance and Test Generation , 2006, FATES/RV.

[47]  Houssam Abbas,et al.  Model Validation of PWM DC–DC Converters , 2017, IEEE Transactions on Industrial Electronics.

[48]  Mohammad Reza Mousavi,et al.  Notions of Conformance Testing for Cyber-Physical Systems: Overview and Roadmap (Invited Paper) , 2015, CONCUR.

[49]  Goran Frehse,et al.  Flowpipe-Guard Intersection for Reachability Computations with Support Functions , 2012, ADHS.

[50]  Ralph-Johan Back,et al.  Refinement Calculus: A Systematic Introduction , 1998 .

[51]  Julien Schmaltz,et al.  On Conformance Testing for Timed Systems , 2008, FORMATS.

[52]  George J. Pappas,et al.  Approximations of Stochastic Hybrid Systems , 2009, IEEE Transactions on Automatic Control.

[53]  Kevin A. Grasse,et al.  Constructing simulation relations for IDO systems affine in inputs and disturbances , 2015, Math. Control. Signals Syst..

[54]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[55]  Thomas A. Henzinger,et al.  The benefits of relaxing punctuality , 1991, PODC '91.

[56]  Morteza Mohaqeqi,et al.  Towards an Approximate Conformance Relation for Hybrid I/O Automata , 2016, V2CPS@IFM.

[57]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[58]  Goran Frehse,et al.  PHAVer: algorithmic verification of hybrid systems past HyTech , 2005, International Journal on Software Tools for Technology Transfer.

[59]  Paulo Tabuada,et al.  Bisimulation relations for dynamical, control, and hybrid systems , 2005, Theor. Comput. Sci..

[60]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[61]  Rupak Majumdar,et al.  Computing Distances between Reach Flowpipes , 2016, HSCC.

[62]  George J. Pappas.  Bisimilar linear systems , 2003, Autom..

[63]  Hendrik Roehm,et al.  STLInspector: STL Validation with Guarantees , 2017, CAV.

[64]  Insup Lee,et al.  Compositional Refinement for Hierarchical Hybrid Systems , 2001, HSCC.

[65]  Antoine Girard,et al.  Approximation Metrics for Discrete and Continuous Systems , 2006, IEEE Transactions on Automatic Control.

[66]  Nancy A. Lynch,et al.  Hybrid I/O Automata Revisited , 2001, HSCC.

[67]  Rupak Majumdar,et al.  Quantifying Conformance Using the Skorokhod Metric , 2015, CAV.

[68]  Arjan van der Schaft,et al.  Bisimulation of Dynamical Systems , 2004, HSCC.

[69]  George J. Pappas,et al.  Hierarchical control system design using approximate simulation , 2001 .

[70]  Haibo Ji,et al.  Hierarchical analysis of large-scale control systems via vector simulation function , 2017, Syst. Control. Lett..

[71]  Rupak Majumdar,et al.  Computing the Skorokhod distance between polygonal traces , 2015, HSCC.

[72]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[73]  Paulo Tabuada,et al.  Approximately Bisimilar Symbolic Models for Incrementally Stable Switched Systems , 2008, IEEE Transactions on Automatic Control.

[74]  Ezio Bartocci,et al.  Approximate Bisimulations for Sodium Channel Dynamics , 2012, CMSB.

[75]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[76]  Mahesh Viswanathan,et al.  Stability Preserving Simulations and Bisimulations for Hybrid Systems , 2015, IEEE Transactions on Automatic Control.

[77]  George J. Pappas,et al.  Approximate bisimulation for a class of stochastic hybrid systems , 2006, 2006 American Control Conference.

[78]  George J. Pappas,et al.  Abstractions of constrained linear systems , 2003, Proceedings of the 2003 American Control Conference, 2003..

[79]  Radu Grosu,et al.  Computing bisimulation functions using SOS optimization and δ-decidability over the reals , 2015, HSCC.

[80]  André Platzer,et al.  ModelPlex: verified runtime validation of verified cyber-physical system models , 2014, Formal Methods in System Design.

[81]  Stavros Tripakis,et al.  Conformance testing for real-time systems , 2009, Formal Methods Syst. Des..

[82]  Thomas A. Henzinger,et al.  The Algorithmic Analysis of Hybrid Systems , 1995, Theor. Comput. Sci..

[83]  Tarik Nahhal,et al.  Coverage-guided test generation for continuous and hybrid systems , 2009, Formal Methods Syst. Des..

[84]  Alessandro Abate,et al.  On the Relationship Between Bisimulation and Trace Equivalence in an Approximate Probabilistic Context , 2017, FoSSaCS.

[85]  Mohammad Reza Mousavi,et al.  Model-Based Testing of Cyber-Physical Systems , 2017 .

[86]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[87]  George J. Pappas,et al.  Discrete abstractions of hybrid systems , 2000, Proceedings of the IEEE.

[88]  Richard Banach,et al.  Continuous ASM, and a Pacemaker Sensing Fragment , 2012, ABZ.

[89]  Matthias Althoff,et al.  Combining zonotopes and support functions for efficient reachability analysis of linear systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[90]  Joseph Sifakis,et al.  Property Preserving Simulations , 1992, CAV.

[91]  Matthias Althoff,et al.  An Introduction to CORA 2015 , 2015, ARCH@CPSWeek.

[92]  Edmund M. Clarke,et al.  The Image Computation Problem in Hybrid Systems Model Checking , 2007, HSCC.

[93]  Jens Oehlerking,et al.  Verifying Properties of an Electro-Mechanical Braking System , 2015, ARCH@CPSWeek.

[94]  Houssam Abbas,et al.  WiP abstract: Conformance testing as falsification for cyber-physical systems , 2014, 2014 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS).

[95]  Bernhard K. Aichernig,et al.  Automated Conformance Verification of Hybrid Systems , 2010, 2010 10th International Conference on Quality Software.

[96]  Paulo Tabuada,et al.  Verification and Control of Hybrid Systems - A Symbolic Approach , 2009 .

[97]  Antoine Girard.  A composition theorem for bisimulation functions , 2013, ArXiv.

[98]  Jun Liu,et al.  Bisimulations for input-output stability of hybrid systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[99]  A. Girard,et al.  Efficient reachability analysis for linear systems using support functions , 2008 .

[100]  Paulo Tabuada.  Approximate Simulation Relations and Finite Abstractions of Quantized Control Systems , 2007, HSCC.

[101]  A. J. van der Schaft,et al.  Equivalence of dynamical systems by bisimulation , 2004, IEEE Transactions on Automatic Control.

[102]  Alessandro Abate,et al.  Approximation Metrics Based on Probabilistic Bisimulations for General State-Space Markov Processes: A Survey , 2013, Hybrid Autonomous Systems@ETAPS.

[103]  Houssam Abbas,et al.  Towards composition of conformant systems , 2015, ArXiv.

[104]  Naijun Zhan,et al.  Approximate Bisimulation and Discretization of Hybrid CSP , 2016, FM.

[105]  George J. Pappas,et al.  SIMULATION RELATIONS FOR DISCRETE-TIME LINEAR SYSTEMS , 2002 .

[106]  Bernhard K. Aichernig,et al.  Conformance Testing of Hybrid Systems with Qualitative Reasoning Models , 2009, MBT@ETAPS.

[107]  Ian M. Mitchell.  Comparing Forward and Backward Reachability as Tools for Safety Analysis , 2007, HSCC.

[108]  Morteza Mohaqeqi,et al.  Conformance Testing of Cyber-Physical Systems: A Comparative Study , 2014, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[109]  Houssam Abbas,et al.  Conformance Testing as Falsification for Cyber-Physical Systems , 2014, ArXiv.

[110]  André Platzer,et al.  KeYmaera: A Hybrid Theorem Prover for Hybrid Systems (System Description) , 2008, IJCAR.

[111]  Thao Dang.  Model-Based Testing of Hybrid Systems , 2011, Model-Based Testing for Embedded Systems.

[112]  Antoine Girard,et al.  Computational Approaches to Analysis and Control of Hybrid Systems. (Approches Computationnelles pour l'Analyse et le Contrôle des Systèmes Hybrides) , 2013 .

[113]  van Mpwj Michiel Osch.  Model-based testing of hybrid systems , 2007 .

[114]  Javier Esparza,et al.  Reachability Analysis of Pushdown Automata: Application to Model-Checking , 1997, CONCUR.

[115]  George J. Pappas,et al.  Approximate Bisimulations for Nonlinear Dynamical Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[116]  Thomas A. Henzinger,et al.  Alternating Refinement Relations , 1998, CONCUR.

[117]  Jan-David Quesel,et al.  Similarity, Logic, and Games - Bridging Modeling Layers of Hybrid Systems , 2013, Berichte aus dem Department für Informatik / Universität Oldenburg / Fachbereich Informatik.

[118]  Olaf Stursberg,et al.  On Systematic Simulation of Open Continuous Systems , 2003, HSCC.

[119]  Goran Frehse,et al.  Compositional verification of hybrid systems using simulation relations , 2005 .

[120]  Goran Frehse,et al.  On Timed Simulation Relations for Hybrid Systems and Compositionality , 2006, FORMATS.

[121]  Jan Tretmans,et al.  A Formal Approach to Conformance Testing , 1993, Protocol Test Systems.

[122]  Bernhard K. Aichernig,et al.  Model-Based Mutation Testing of Hybrid Systems , 2009, FMCO.

[123]  George J. Pappas,et al.  Approximate Bisimulations for Constrained Linear Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[124]  Stanley Bak,et al.  HyLAA: A Tool for Computing Simulation-Equivalent Reachability for Linear Systems , 2017, HSCC.

[125]  Thao Dang,et al.  Test Coverage Estimation Using Threshold Accepting , 2014, ATVA.

[126]  Pieter J. Mosterman,et al.  Model-Based Testing for Embedded Systems , 2011, Computational Analysis, Synthesis, & Design Dynamic Systems.

[127]  Oded Maler,et al.  Accurate hybridization of nonlinear systems , 2010, HSCC '10.

[128]  D. Siljak,et al.  An Inclusion Principle for Dynamic Systems , 1982, 1982 American Control Conference.

[129]  Delin Chu,et al.  A Canonical Form for The Inclusion Principle of Dynamic Systems , 2004, 2007 IEEE International Conference on Control and Automation.

[130]  Insup Lee,et al.  Compositional modeling and refinement for hierarchical hybrid systems , 2006, J. Log. Algebraic Methods Program..

[131]  Kevin A. Grasse,et al.  Simulation and Bisimulation of Nonlinear Control Systems with Admissible Classes of Inputs and Disturbances , 2007, SIAM J. Control. Optim..

[132]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures , 2005 .

[133]  Linlin Qin,et al.  Approximate bisimulations for constrained discrete-time linear systems (ICCAS 2015) , 2015, 2015 15th International Conference on Control, Automation and Systems (ICCAS).

[134]  Paulo Tabuada,et al.  Approximately bisimilar symbolic models for nonlinear control systems , 2007, Autom..

[135]  Robert K. Brayton,et al.  Compositional and hierarchical techniques for the formal verification of real-time systems , 1998 .

[136]  Florian Lorber,et al.  Time for Mutants - Model-Based Mutation Testing with Timed Automata , 2013, TAP@STAF.

[137]  Mahesh Viswanathan,et al.  Pre-orders for reasoning about stability , 2012, HSCC '12.