A Process for Sound Conformance Testing of Cyber-Physical Systems

We present a process for sound conformance testing of cyber-physical systems, which involves functional but also non-functional aspects. The process starts with a hybrid model of cyber-physical systems in which the correct behavior of the system (at its interface level) is specified. Such a model captures both discrete behavior and evolution of continuous dynamics of the system in time. Since conformance testing inherently involves comparing continuous dynamics, the key parameters of the process are (1) the conformance bounds defining when two signals are sufficiently close to each other, and (2) the permitted error margin in the conformance analysis introduced by sampling of continuous signals. The final parameter of this process is (3) finding (and adjusting) the sampling rate of the dynamic behavior. In the specified process, we provide different alternatives for fixing the error margin of the conformance testing if the sampling rate is fixed, establishing the sampling rate if the error margin is fixed and finding conformance bounds once the sampling rate and the error margin are fixed.

[1]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[2]  Jan Tretmans,et al.  Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation , 1996, Comput. Networks ISDN Syst..

[3]  Thomas A. Henzinger,et al.  The Algorithmic Analysis of Hybrid Systems , 1995, Theor. Comput. Sci..

[4]  Augusto Sampaio,et al.  Test case generation from natural language requirements based on SCR specifications , 2013, SAC '13.

[5]  Mohammad Reza Mousavi,et al.  A Tool Prototype for Model-Based Testing of Cyber-Physical Systems , 2015, ICTAC.

[6]  Teck Ping Khoo.  Model Based Testing of Cyber-Physical Systems , 2018, ICFEM.

[7]  Georgios Fainekos,et al.  Towards Formal Specification Visualization for Testing and Monitoring of Cyber-Physical Systems , 2014 .

[8]  R. Sanfelice,et al.  Hybrid dynamical systems , 2009, IEEE Control Systems.

[9]  Albert Benveniste,et al.  Toward an Approximation Theory for Computerised Control , 2002, EMSOFT.

[10]  Matthias Althoff,et al.  Zonotope bundles for the efficient computation of reachable sets , 2011, IEEE Conference on Decision and Control and European Control Conference.

[11]  Olaf Stursberg,et al.  Efficient Representation and Computation of Reachable Sets for Hybrid Systems , 2003, HSCC.

[12]  Matthias Althoff,et al.  An Introduction to CORA 2015 , 2015, ARCH@CPSWeek.

[13]  Matthias Althoff,et al.  Reachability Analysis and its Application to the Safety Assessment of Autonomous Cars , 2010 .

[14]  Augusto Sampaio,et al.  NAT2TEST Tool: From Natural Language Requirements to Test Cases Based on CSP , 2015, SEFM.

[15]  Matthias Althoff,et al.  Reachset Conformance Testing of Hybrid Automata , 2016, HSCC.

[16]  Tarik Nahhal,et al.  Coverage-guided test generation for continuous and hybrid systems , 2009, Formal Methods Syst. Des..

[17]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[18]  Augusto Sampaio,et al.  Simulation of hybrid systems from natural-language requirements , 2017, 2017 13th IEEE Conference on Automation Science and Engineering (CASE).

[19]  Ilya Kolmanovsky,et al.  Automotive Powertrain Control — A Survey , 2006 .

[20]  Hans A. Hansson,et al.  A Survey on Testing for Cyber Physical System , 2015, ICTSS.

[21]  Antoine Girard,et al.  Approximate Bisimulation: A Bridge Between Computer Science and Control Theory , 2011, Eur. J. Control.

[22]  O. Stursberg,et al.  Computing Reachable Sets of Hybrid Systems Using a Combination of Zonotopes and Polytopes , 2010 .

[23]  Antoine Girard,et al.  Approximate Simulation Relations for Hybrid Systems , 2008, Discret. Event Dyn. Syst..

[24]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[25]  Pieter J. Mosterman,et al.  Model-Based Testing for Embedded Systems , 2011, Computational Analysis, Synthesis, & Design Dynamic Systems.

[26]  Morteza Mohaqeqi,et al.  Sound Test-Suites for Cyber-Physical Systems , 2016, 2016 10th International Symposium on Theoretical Aspects of Software Engineering (TASE).

[27]  Wpmh Maurice Heemels,et al.  Survey of modeling, analysis, and control of hybrid systems , 2009 .

[28]  Houssam Abbas,et al.  Formal property verification in a conformance testing framework , 2014, 2014 Twelfth ACM/IEEE Conference on Formal Methods and Models for Codesign (MEMOCODE).

[29]  Rupak Majumdar,et al.  Quantifying Conformance Using the Skorokhod Metric , 2015, CAV.

[30]  Augusto Sampaio,et al.  A CSP Timed Input-Output Relation and a Strategy for Mechanised Conformance Verification , 2013, ICFEM.

[31]  David Lee,et al.  Testing for Finite State Systems , 1998, CSL.

[32]  Thomas Stauner,et al.  Modelling and Verification using Linear Hybrid Automata -- a Case Study , 2000 .

[33]  Pieter J. Mosterman,et al.  Cyber-physical systems challenges: a needs analysis for collaborating embedded software systems , 2016, Software & Systems Modeling.

[34]  Houssam Abbas,et al.  Conformance Testing as Falsification for Cyber-Physical Systems , 2014, ArXiv.

[35]  Thao Dang.  Model-Based Testing of Hybrid Systems , 2011, Model-Based Testing for Embedded Systems.

[36]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[37]  van Mpwj Michiel Osch.  Model-based testing of hybrid systems , 2007 .

[38]  Kenneth R. Butts,et al.  Powertrain control verification benchmark , 2014, HSCC.

[39]  Houssam Abbas.  Test-Based Falsification and Conformance Testing for Cyber-Physical Systems , 2015 .

[40]  Thomas A. Henzinger,et al.  The theory of hybrid automata , 1996, Proceedings 11th Annual IEEE Symposium on Logic in Computer Science.

[41]  Augusto Sampaio,et al.  A Process Algebra Based Strategy for Generating Test Vectors from SCR Specifications , 2012, SBMF.

[42]  George J. Pappas,et al.  Robustness of temporal logic specifications for continuous-time signals , 2009, Theor. Comput. Sci..

[43]  W. Marsden.  I and J , 2012 .

[44]  James Kapinski,et al.  Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems , 2015, NFM.

[45]  Xin Chen,et al.  Reachability analysis of non-linear hybrid systems using Taylor Models , 2015 .

[46]  Gerardo Lafferriere,et al.  Symbolic Reachability Computation for Families of Linear Vector Fields , 2001, J. Symb. Comput..

[47]  R. K. Rao Yarlagadda,et al.  Analog and Digital Signals and Systems , 2009 .

[48]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[49]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[50]  Augusto Sampaio,et al.  Sound conformance testing for cyber-physical systems: Theory and implementation , 2017, Sci. Comput. Program..

[51]  Amir Pnueli,et al.  Orthogonal Polyhedra: Representation and Computation , 1999, HSCC.

[52]  Michiel van Osch.  Hybrid Input-Output Conformance and Test Generation , 2006, FATES/RV.

[53]  Mohammad Reza Mousavi,et al.  Notions of Conformance Testing for Cyber-Physical Systems: Overview and Roadmap (Invited Paper) , 2015, CONCUR.

[54]  Thomas A. Henzinger,et al.  Hybrid Automata: An Algorithmic Approach to the Specification and Verification of Hybrid Systems , 1992, Hybrid Systems.

[55]  Rajeev Alur,et al.  Formal verification of hybrid systems , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[56]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[57]  HarmanMark,et al.  Using formal specifications to support testing , 2009 .

[58]  Antoine Girard,et al.  Zonotope/Hyperplane Intersection for Hybrid Systems Reachability Analysis , 2008, HSCC.