Improved test generation algorithms for pair-wise testing

Software testing is expensive and time consuming. Given the different input parameters with multiple possible values for each parameter, performing exhaustive testing which tests all possible combinations is practically impossible. Generating an optimal test set which will effectively test the software system is therefore desired. Pair-wise testing is known for its effectiveness in different types of software testing. Pair-wise testing requires that for a given numbers of input parameters to the system, each possible combination of values for any pair of parameters be covered by at least one test case. Pair-wise testing is known for its effectiveness in different types of software testing. The problem of generating a minimum size test set for pair-wise testing is NP-complete. This paper presents new techniques for reducing the number of test cases for pair-wise testing. The paper shows an algorithm to generate test cases for 2-valued parameters and how orthogonal arrays and ordered designs may be used for deriving test cases for parameters with more than two values. Moreover, using mixed-level or asymmetric orthogonal array as tool, we present test set generation strategy for parameters with different number of values. A comparison of empirical results with previous test generation strategies "AETG" and "IPO" shows that the number of test cases generated with the proposed methodology is never higher and in some cases significantly lower than using AETG or IPO

[1]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[2]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[3]  Myra B. Cohen,et al.  Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces , 2006, IEEE Trans. Software Eng..

[4]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.

[5]  Madhan Shridhar Phadke,et al.  Quality Engineering Using Robust Design , 1989 .

[6]  Colin L. Mallows,et al.  Factor-covering designs for testing software , 1998 .

[7]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[8]  R. C. Bose,et al.  Introduction to Combinatorial Theory , 1984 .

[9]  Robert L. Probert,et al.  A practical strategy for testing pair-wise coverage of network interfaces , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[10]  A. A. Andrews.  Editortial: New EIC Introduction , 2002 .

[11]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[12]  Robert Mandl,et al.  Orthogonal Latin squares: an application of experiment design to compiler testing , 1985, CACM.

[13]  Charles J. Colbourn,et al.  The CRC handbook of combinatorial designs , edited by Charles J. Colbourn and Jeffrey H. Dinitz. Pp. 784. $89.95. 1996. ISBN 0-8493-8948-8 (CRC). , 1997, The Mathematical Gazette.

[14]  Myra B. Cohen,et al.  Covering arrays for efficient fault characterization in complex configuration spaces , 2004, IEEE Transactions on Software Engineering.

[15]  C. Colbourn,et al.  The CRC handbook of combinatorial designs , edited by Charles J. Colbourn and Jeffrey H. Dinitz. Pp. 784. $89.95. 1996. ISBN 0-8493-8948-8 (CRC). , 1997, The Mathematical Gazette.

[16]  C. R. Rao,et al.  Factorial Experiments Derivable from Combinatorial Arrangements of Arrays , 1947 .