Construction of Mixed Covering Arrays of strengths 2 through 6 Using a Tabu Search Approach

The development of a new software system involves extensive tests of the software functionality in order to identify possible failures. Also, a software system already built requires a fine tuning of its configurable options to give the best performance in the environment where it is going to work. Both cases require a finite set of tests that avoids testing all the possible combinations (which is time consuming); to this situation mixed covering arrays (MCAs) are a feasible alternative. MCAs are combinatorial structures having a case per row. MCAs are small, in comparison with exhaustive search, and guarantee a level of interaction among the involved parameters (a difference with random testing). We present a tabu search algorithm (TSA) for the construction of MCAs. Also, we report the fine tuning process used to identify the best parameter values for TSA. The analyzed TSA parameters were three different initialization functions, five different tabu list sizes and the mixture of four neighborhood functio...

[1]  Charles J. Colbourn,et al.  One-test-at-a-time heuristic search for interaction test suites , 2007, GECCO '07.

[2]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[3]  Vadim Okun,et al.  Testing with Model Checker: Insuring Fault Visibility , 2002 .

[4]  Yu Lei,et al.  Refining the In-Parameter-Order Strategy for Constructing Covering Arrays , 2008, Journal of research of the National Institute of Standards and Technology.

[5]  Jeff Yu Lei,et al.  IPOG: A General Strategy for T-Way Software Testing , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[6]  Jeff Yu Lei,et al.  A Survey of Binary Covering Arrays , 2011, Electron. J. Comb..

[7]  José Torres-Jiménez,et al.  Computation of Ternary Covering Arrays Using a Grid , 2004, AACC.

[8]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.

[9]  Charles J. Colbourn,et al.  The density algorithm for pairwise interaction testing: Research Articles , 2007 .

[10]  Myra B. Cohen,et al.  A framework of greedy methods for constructing interaction test suites , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[11]  José Torres-Jiménez,et al.  Construction of Mixed Covering Arrays of Variable Strength Using a Tabu Search Approach , 2010, COCOA.

[12]  Yu-Wen Tung,et al.  Automating test case generation for the new generation mission software system , 2000, 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484).

[13]  Charles J. Colbourn,et al.  The density algorithm for pairwise interaction testing , 2007, Softw. Test. Verification Reliab..

[14]  Myra B. Cohen,et al.  A deterministic density algorithm for pairwise interaction coverage , 2004, IASTED Conf. on Software Engineering.

[15]  Jun Yan,et al.  A backtracking search tool for constructing combinatorial test suites , 2008, J. Syst. Softw..

[16]  Robert L. Probert,et al.  A practical strategy for testing pair-wise coverage of network interfaces , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[17]  José Torres-Jiménez,et al.  A New Backtracking Algorithm for Constructing Binary Covering Arrays of Variable Strength , 2009, MICAI.

[18]  Charles J. Colbourn,et al.  Tabu search for covering arrays using permutation vectors , 2009 .

[19]  Victor V. Kuliamin,et al.  A survey of methods for constructing covering arrays , 2011, Programming and Computer Software.

[20]  A. Jefferson Offutt,et al.  Combination testing strategies: a survey , 2005, Softw. Test. Verification Reliab..

[21]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[22]  D. Richard Kuhn,et al.  Pseudo-Exhaustive Testing for Software , 2006, 2006 30th Annual IEEE/NASA Software Engineering Workshop.

[23]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[24]  Kari J. Nurmela,et al.  Upper bounds for covering arrays by tabu search , 2004, Discret. Appl. Math..

[25]  Steven David Prestwich,et al.  Constraint Models for the Covering Test Problem , 2006, Constraints.

[26]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[27]  José Torres-Jiménez,et al.  Strength Two Covering Arrays Construction Using a SAT Representation , 2008, MICAI.

[28]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[29]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).