A Method for Systematically Generating Tests from Object-Oriented Class Interfaces

(Abstract) This thesis describes the development and evaluation of a manual black-box testing method inspired by Zweben's test adequacy criteria, which apply white-box analogues of all-DU-pairs and all-nodes to a flow graph generated from the black-box specification. The approach described herein generates tests from a matrix representation of a class interface based on the flow graph concept. In this process, separate matrices for all-DU-pairs and all-nodes guide the generation of the required tests. The primary goal of the research is not to optimize the number of tests generated but to describe the process in a user-friendly manner so that practitioners can utilize it directly, quickly, and efficiently for real-world testing purposes. The approach has been evaluated to assess its effectiveness at detecting bugs. Both strategies—all-DU-pairs and all-nodes—were compared against three other testing methods: the commercial white-box testing tool Jtest, Orthogonal Array Testing Strategy (OATS), and test cases generated at random. The five approaches were applied across a sample of eleven java classes selected from java.util.*. Experimental results indicate that the two versions resulting from this research performed on par with or better than their respective equivalent approaches. The all-DU-pairs method performed better than all other approaches except for the random approach, with which it compared equally. Experimental evaluation results thus indicate that an automated approach based on the manual method is worth exploring. iii Acknowledgments

[1]  Sriram Sankar,et al.  Structural specification-based testing with ADL , 1996, ISSTA '96.

[2]  Robin W. Whitty An exercise in weakest preconditions , 1990, Bull. EATCS.

[3]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[4]  Wei-Tek Tsai,et al.  Automated Test Case Generation for Programs Specified by Relational Algebra Queries , 1990, IEEE Trans. Software Eng..

[5]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[6]  Laurent Fribourg,et al.  Test sets generation from algebraic specifications using logic programming , 1986, J. Syst. Softw..

[7]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[8]  Paul Ammann,et al.  Using formal methods to derive test frames in category-partition testing , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[9]  Stephen H. Edwards,et al.  A framework for detecting interface violations in component-based software , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[10]  I. Spence,et al.  Generation Of Software Tests From Specifications , 1970 .

[11]  Robert M. Hierons Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[12]  Shaoying Liu,et al.  Criteria for generating specification-based tests , 1999, Proceedings Fifth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'99) (Cat. No.PR00434).

[13]  Jeannette M. Wing A specifier's introduction to formal methods , 1990, Computer.

[14]  Stephen H. Edwards,et al.  A framework for practical, automated black‐box testing of component‐based software † , 2001, Softw. Test. Verification Reliab..

[15]  Stuart H. Zweben,et al.  Systematic Testing of Data Abstractions Based on Software Specifications , 1992, Softw. Test. Verification Reliab..

[16]  Bertrand Meyer,et al.  Object-Oriented Software Construction, 2nd Edition , 1997 .

[17]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[18]  Richard G. Hamlet,et al.  Data Abstraction, Implementation, Specification, and Testing , 1981, TOPL.

[19]  A. Jefferson Offutt,et al.  Generating Tests from UML Specifications , 1999, UML.

[20]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[21]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[22]  S. N. Weiss,et al.  All-Uses versus Mutation Testing : An ExperimentalComparison of E ectiveness , 1996 .

[23]  W. Eric Wong,et al.  Comparing the Fault Detection Effectiveness of Mutation and Data Flow Testing: An Empirical Study , 1993 .

[24]  Ian J. Hayes,et al.  Specification directed module testing , 1986, IEEE Transactions on Software Engineering.

[25]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[26]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[27]  Phyllis G. Frankl,et al.  The ASTOOT approach to testing object-oriented programs , 1994, TSEM.