Compositional Verification of Input-Output Conformance via CSP Refinement Checking

This paper contributes to a testing theory, based on the CSP process algebra, whose conformance relation (cspio) distinguishes input and output events. Although cspio has been defined in terms of the standard CSP traces model, we show that our theory can be immediately extended to address deadlock, outputlock and livelock situations if a special output event is used to represent quiescence. This is formally established by showing that this broader view of cspio is equivalent to Tretmans' ioco relation. Furthermore, we address compositional conformance verification, establishing compositionality properties for cspio with respect to process composition operators. Our testing theory has been adopted in an industrial context involving a collaboration with Motorola, whose focus is on the testing of mobile applications. Some examples are presented to illustrate the overall approach.

[1]  C. A. R. Hoare,et al.  Unifying theories of programming , 1998, RelMiCS.

[2]  Ajitha Rajan,et al.  Requirements Coverage as an Adequacy Measure for Conformance Testing , 2008, ICFEM.

[3]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[4]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[5]  Thierry Jéron,et al.  A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2005 .

[6]  Fabien Peureux,et al.  I/O-automata Based Testing , 2004, Model-Based Testing of Reactive Systems.

[7]  S. Anderson,et al.  Secure Synthesis of Code: A Process Improvement Experiment , 1999, World Congress on Formal Methods.

[8]  Bernhard K. Aichernig,et al.  From Faults Via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems , 2006, FASE.

[9]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[10]  Bernhard K. Aichernig,et al.  Unifying Input Output Conformance , 2008, UTP.

[11]  Arend Rensink,et al.  Compositional Testing with ioco , 2003, FATES.

[12]  Mogens Nielsen,et al.  TAPSOFT '95: Theory and Practice of Software Development , 1995, Lecture Notes in Computer Science.

[13]  Thomas Ball Automated Abstraction of Software , 2006, ATVA.

[14]  Franz Wotawa,et al.  On the fly input output conformance verification , 2008, ICSE 2008.

[15]  Augusto Sampaio,et al.  Guided Test Generation from CSP Models , 2008, ICTAC.

[16]  Katalin Tarnay,et al.  Testing of Communicating Systems , 1999, IFIP — The International Federation for Information Processing.

[17]  Jean-Raymond Abrial,et al.  The B-book - assigning programs to meanings , 1996 .

[18]  Yoshinao Isobe,et al.  A Generic Theorem Prover of CSP Refinement , 2005, TACAS.

[19]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.

[20]  Jan Peleska,et al.  Test automation of safety-critical reactive systems , 1997 .

[21]  Cliff B. Jones,et al.  Systematic software development using VDM (2. ed.) , 1990, Prentice Hall International Series in Computer Science.

[22]  Flávia de Almeida Barros,et al.  Motorola SpecNL: A Hybrid System to Generate NL Descriptions from Test Case Specifications , 2006, 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06).

[23]  Steve A. Schneider Abstraction and Testing , 1999, World Congress on Formal Methods.

[24]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[25]  Bernhard K. Aichernig,et al.  Improving Fault-based Conformance Testing , 2008, Electron. Notes Theor. Comput. Sci..

[26]  Ana Cavalcanti,et al.  Testing for Refinement in CSP , 2007, ICFEM.

[27]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[28]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[29]  Brian Nielsen,et al.  Formal Approaches to Software Testing, 4th International Workshop, FATES 2004, Linz, Austria, September 21, 2004, Revised Selected Papers , 2005, FATES.

[30]  Cliff B. Jones,et al.  Systematic software development using VDM , 1986, Prentice Hall International Series in Computer Science.

[31]  Gerhard Goos,et al.  Automated Technology for Verification and Analysis , 2004, Lecture Notes in Computer Science.

[32]  Anne Elisabeth Haxthausen,et al.  Theoretical Aspects of Computing - ICTAC 2008, 5th International Colloquium, Istanbul, Turkey, September 1-3, 2008. Proceedings , 2008, ICTAC.

[33]  Perdita Stevens,et al.  Modelling Recursive Calls with UML State Diagrams , 2003, FASE.

[34]  Ferhat Khendek,et al.  Compositional Testing of Communication Systems , 2006, TestCom.

[35]  Steve A. Schneider,et al.  Concurrent and Real-time Systems: The CSP Approach , 1999 .

[36]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[37]  C. A. Petri,et al.  Concurrency Theory , 1986, Advances in Petri Nets.

[38]  Andrew William Roscoe,et al.  The Theory and Practice of Concurrency , 1997 .

[39]  Jim Woodcock,et al.  Using Z - specification, refinement, and proof , 1996, Prentice Hall international series in computer science.

[40]  Augusto Sampaio,et al.  Formal Specification Generation from Requirement Documents , 2006, SBMF.