Generating Optimal Test Set for Neighbor Factors Combinatorial Testing

Combinatorial testing is a specification-based testing method, which can detect the faults triggered by interaction of factors. For one kind of software in which the interactions only exist between neighbor factors, this paper proposes the concept of neighbor factors combinatorial testing, presents the covering array generation algorithms for neighbor factors pair-wise (N=2) coverage, neighbor factors N-way (Nges2) coverage and variable strength neighbor factors coverage, and proves that the covering arrays generated by these three algorithms are optimal. Finally we analyze an application scenario, which shows that this approach is very practical

[1]  Robert L. Probert,et al.  A practical strategy for testing pair-wise coverage of network interfaces , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[2]  Siddhartha R. Dalal,et al.  The Automatic Efficient Test Generator (AETG) system , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[3]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.

[4]  Myra B. Cohen,et al.  A variable strength interaction testing of components , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[5]  Patrick J. Schroeder,et al.  Comparing the fault detection effectiveness of n-way and random test suites , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[6]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[7]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[8]  Tatsuhiro Tsuchiya,et al.  A new method for constructing pair-wise covering designs for software testing , 2002, Information Processing Letters.

[9]  Alan W. Williams,et al.  Software component interaction testing: coverage measurement and generation of configurations , 2002 .

[10]  Robert Mandl,et al.  Orthogonal Latin squares: an application of experiment design to compiler testing , 1985, CACM.

[11]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[12]  Dennis Shasha,et al.  Products of mixed covering arrays of strength two , 2006 .

[13]  Bogdan Korel,et al.  Black-box test reduction using input-output analysis , 2000, ISSTA '00.

[14]  Charles J. Colbourn,et al.  Covering arrays of higher strength from permutation vectors , 2006 .

[15]  M. J. Reilly,et al.  An investigation of the applicability of design of experiments to software testing , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..

[16]  James A. Whittaker,et al.  A software testing model: using design of experiments (doe) and logistic regression , 2001 .

[17]  Baowen Xu,et al.  Automatic Test Generation for N-Way Combinatorial Testing , 2005, QoSA/SOQUA.

[18]  Myra B. Cohen,et al.  A deterministic density algorithm for pairwise interaction coverage , 2004, IASTED Conf. on Software Engineering.

[19]  Bogdan Korel,et al.  Maintaining the Quality of Black-Box Testing , 2001 .

[20]  Robert Brownlie,et al.  Robust testing of AT&T PMX/StarMAIL using OATS , 1992, AT&T Technical Journal.

[21]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[22]  Soumen Maity,et al.  An Improved Test Generation Algorithm for PairWise Testing , 2003 .

[23]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[24]  Soumen Maity,et al.  Improved test generation algorithms for pair-wise testing , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[25]  Robert L. Probert,et al.  A measure for component interaction test coverage , 2001, Proceedings ACS/IEEE International Conference on Computer Systems and Applications.

[26]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..