Use of sequencing constraints for specification-based testing of concurrent programs

This paper presents and evaluates a specification-based methodology for testing concurrent programs. This methodology requires sequencing constraints, which specify restrictions on the allowed sequences of synchronization events. Sequencing constraints for a concurrent program can be derived from the program's formal or informal specification. Details of the proposed testing methodology based on the use of Constraints on Succeeding and Preceding Events (CSPE) are given. How to achieve coverage and detect violations of CSPE constraints for a concurrent program, according to deterministic and nondeterministic testing of this program, are described. A coverage criterion for CSPE-based testing is defined and analyzed. The results of empirical studies of CSPE-based testing for four concurrent problems are reported. These results indicate that the use of sequencing constraints for specification-based testing of concurrent programs is a promising approach.

[1]  Gregor von Bochmann,et al.  Trace Analysis for Conformance and Arbitration Testing , 1989, IEEE Trans. Software Eng..

[2]  Richard H. Carver,et al.  Replay and testing for concurrent programs , 1991, IEEE Software.

[3]  Jari Arkko.  On the Existence and Production of State Identification Machines for Labelled Transition Systems , 1993, FORTE.

[4]  Rance Cleaveland,et al.  The concurrency workbench: a semantics-based tool for the verification of concurrent systems , 1993, TOPL.

[5]  David S. Rosenblum.  Specifying concurrent systems with TSL , 1991, IEEE Software.

[6]  Qing Yu,et al.  Oracles for checking temporal properties of concurrent systems , 1994, SIGSOFT '94.

[7]  Louise E. Moser,et al.  Graphical specifications for concurrent software systems , 1992, International Conference on Software Engineering.

[8]  David Luckham,et al.  Debugging Ada Tasking Programs , 1985, IEEE Software.

[9]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[10]  Colin Stirling,et al.  An Introduction to Modal and Temporal Logics for CCS , 1991, Concurrency: Theory, Language, And Architecture.

[11]  Richard N. Taylor,et al.  Structural Testing of Concurrent Programs , 1992, IEEE Trans. Software Eng..

[12]  Jack C. Wileden,et al.  High-level debugging of distributed systems: The behavioral abstraction approach , 1983, J. Syst. Softw..

[13]  R.-D. Yang,et al.  Path analysis testing of concurrent programs , 1992, Inf. Softw. Technol..

[14]  Kuo-Chung Tai,et al.  Observation inaccuracy in conformance testing with multiple testers , 1998, Proceedings. 1998 IEEE Workshop on Application-Specific Software Engineering and Technology. ASSET-98 (Cat. No.98EX183).

[15]  K. Tai,et al.  A Specification-Based Methology for Testing Concurrent Programs , 1995, ESEC.

[16]  Richard H. Carver,et al.  Static analysis of concurrent software for deriving synchronization constraints , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[17]  Thomas J. LeBlanc,et al.  Debugging Parallel Programs with Instant Replay , 1987, IEEE Transactions on Computers.

[18]  Glenn Bruns,et al.  Distributed systems analysis with CCS , 1997 .

[19]  K. C. Tai,et al.  Conformance testing of protocols specified as labeled transition systems , 1996 .

[20]  Kuo-Chung Tai,et al.  Theory of Fault-Based Predicate Testing for Computer Programs , 1996, IEEE Trans. Software Eng..

[21]  Kuo-Chung Tai,et al.  An incremental approach to structural testing of concurrent software , 1996, ISSTA '96.

[22]  Elaine J. Weyuker,et al.  Automatically Generating Test Data from a Boolean Specification , 1994, IEEE Trans. Software Eng..

[23]  Mladen A. Vouk,et al.  Automatic test-generation for predicates [software testing] , 1996, IEEE Trans. Reliab..

[24]  Gerard J. Holzmann,et al.  Design and validation of computer protocols , 1991 .

[25]  Kuo-Chung Tai,et al.  Reachability testing of asynchronous message-passing programs , 1997, Proceedings of PDSE '97: 2nd International Workshop on Software Engineering for Parallel and Distributed Systems.

[26]  R. H. Carver,et al.  Integrating formal methods and testing for concurrent programs , 1995, COMPASS '95 Proceedings of the Tenth Annual Conference on Computer Assurance Systems Integrity, Software Safety and Process Security'.

[27]  Mladen A. Vouk,et al.  Automatic test generation for predicates , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[28]  David C. Luckham,et al.  TSL: task sequencing language , 1985 .

[29]  Richard H. Carver,et al.  Testing abstract distributed programs and their implementations: A constraint-based approach , 1996, J. Syst. Softw..

[30]  Kuo-Chung Tai,et al.  Synchronizable Test Sequences of Finite State Machines , 1995, Comput. Networks.

[31]  Sagar Naik,et al.  Generation of Adaptive Test Cases from Nondeterministic Finite State Models , 1992, Protocol Test Systems.

[32]  Gregg Rothermel,et al.  An experimental evaluation of selective mutation , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[33]  Kenneth J. Turner,et al.  Using Formal Description Techniques: An Introduction to Estelle, Lotos, and SDL , 1993 .

[34]  David F. McAllister,et al.  An experimental evaluation of the effectiveness of random testing of fault-tolerant software , 1986 .

[35]  Joan M. Francioni,et al.  Nondeterminancy: testing and debugging in message passing parallel programs , 1993, PADD '93.

[36]  Gwan-Hwan Hwang,et al.  Reachability Testing: an Approach to Testing Concurrent Software , 1995, Int. J. Softw. Eng. Knowl. Eng..

[37]  Kuo-Chung Tai.  A graphical notation for describing executions of concurrent Ada programs , 1986, ALET.

[38]  Kuo-Chung Tai,et al.  Test generation for Boolean expressions , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[39]  Alexandre Petrenko,et al.  Protocol testing: review of methods and relevance for software testing , 1994, ISSTA '94.

[40]  Richard H. Carver,et al.  Debugging Concurrent Ada Programs by Deterministic Execution , 1991, IEEE Trans. Software Eng..