General variable strength t-way strategy supporting flexible interactions

To ensure conformance and establish quality, software testing is an integral part in software engineering lifecycle. However, because of resource and time-to-market constraints, testing all exhaustive possibilities is impossible in nearly all practical testing problems. Considering the aforementioned constraints, much research now focuses on a sampling technique based on interaction testing (termed as t-way strategy). Although helpful, most t- way strategies (e.g. AETG, In-Parameter-Order General (IPOG), and GTWay) assume that all parameters have uniform interaction. In reality, the interaction among parameters is rarely uniform. Some parameters may not even interact, wasting the testing efforts. As a result, a number of newly developed t-way strategies that consider variable-strength interaction based on input-output relationships have been developed, e.g. Union, ParaOrder, and Density. Although useful, these strategies often suffer from lack of optimality in terms of the generated test size. Furthermore, no single strategy is dominant because the optimal generation of t-way interaction test suite is considered an Nondeterministic Polynomial (NP) hard problem. Motivated by the above-mentioned challenges, this paper proposes and implements a new strategy, called General Variable Strength (GVS). GVS has been demonstrated, in some cases, to produce better results than other competing strategies.

[1]  Kamal Zuhairi Zamli,et al.  A variable strength interaction test suites generation strategy using Particle Swarm Optimization , 2011, J. Syst. Softw..

[2]  Yu Lei,et al.  IPOG-IPOG-D: efficient test generation for multi-way combinatorial testing , 2008 .

[3]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[4]  Baowen Xu,et al.  Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite , 2008, 2008 The Eighth International Conference on Quality Software.

[5]  Myra B. Cohen,et al.  A variable strength interaction testing of components , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[6]  Xiang Chen,et al.  Variable Strength Interaction Testing with an Ant Colony System Approach , 2009, 2009 16th Asia-Pacific Software Engineering Conference.

[7]  Myra B. Cohen,et al.  Designing Test Suites for Software Interactions Testing , 2004 .

[8]  Bogdan Korel,et al.  Black-box test reduction using input-output analysis , 2000, ISSTA '00.

[9]  Jeff Yu Lei,et al.  IPOG: A General Strategy for T-Way Software Testing , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[10]  Charles J. Colbourn,et al.  The density algorithm for pairwise interaction testing , 2007, Softw. Test. Verification Reliab..

[11]  Xu Baowen,et al.  Generating combinatorial test suite for interaction relationship , 2007 .

[12]  Jacek Czerwonka,et al.  Pairwise Testing in Real World , 2006 .

[13]  Bogdan Korel,et al.  Generating expected results for automated black-box testing , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[14]  Charles J. Colbourn,et al.  A density-based greedy algorithm for higher strength covering arrays , 2009 .

[15]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[16]  Rusli Abdullah,et al.  Design and implementation of a t-way test data generation strategy with automated execution tool support , 2011, Inf. Sci..

[17]  Ron Patton,et al.  Software Testing , 2000 .

[18]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[19]  MOHAMMAD FADEL JAMIL DEVELOPMENT OF AN AUTOMATED TEST DATA GENERATION AND EXECUTION STRATEGY USING COMBINATORIAL APPROACH , 2009 .