Construction of Mixed Covering Arrays for Pair-wise Testing Using Probabilistic Approach in Genetic Algorithm

In a system with large number of input parameters, it is necessary to check for errors that can occur as a result of interactions between various input parameters. However, checking of all possible combinations of input parameters is often restricted due to time and budget constraints. In order to overcome the constraints of exhaustive testing, combinatorial testing has been employed to generate optimal and efficient test set that covers all t-way combinations of input parameters. Pair-wise testing, a combinatorial testing technique, tests all possible combinations of each pair of input parameter values. In this paper, we present an efficient algorithm pair-wise test set generator using genetic algorithm (PTSG-GA) for generating test set for pair-wise testing. PTSG-GA is an extension of our previous work that applies genetic algorithm (GA) to generate optimal test set for pair-wise testing. In this paper, combinatorial objects, namely covering array (CA) and mixed covering arrays (MCA), are used to represent test set. The major contribution of algorithm PTSG-GA is that it uses a probabilistic approach to generate initial population of CAs/MCAs to improve the performance of GA. The algorithm PTSG-GA is implemented using an open-source tool PWiseGen. We have reported experimental results that illustrate the effectiveness of PTSG-GA as compared to existing state-of-the-art algorithms.

[1]  Rusli Abdullah,et al.  G2Way A Backtracking Strategy for Pairwise Test Data Generation , 2008, 2008 15th Asia-Pacific Software Engineering Conference.

[2]  Alan Hartman,et al.  Problems and algorithms for covering arrays , 2004, Discret. Math..

[3]  K. Miettinen,et al.  Quasi-random initial population for genetic algorithms , 2004 .

[4]  Wenhua Wang,et al.  An Interaction-Based Test Sequence Generation Approach for Testing Web Applications , 2008, 2008 11th IEEE High Assurance Systems Engineering Symposium.

[5]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[6]  José Torres-Jiménez,et al.  Construction of Mixed Covering Arrays of Variable Strength Using a Tabu Search Approach , 2010, COCOA.

[7]  Xiang Chen,et al.  Applying Particle Swarm Optimization to Pairwise Testing , 2010, 2010 IEEE 34th Annual Computer Software and Applications Conference.

[8]  Zainal Hisham Che Soh,et al.  Variable-Strength Interaction for T-Way Test Generation Strategy , 2013, SOCO 2013.

[9]  Sangeeta Sabharwal,et al.  An Approach to Test Set Generation for Pair-Wise Testing Using Genetic Algorithms , 2013, SSBSE.

[10]  Paolo Tonella,et al.  Combining model-based and combinatorial testing for effective test case generation , 2012, ISSTA 2012.

[11]  John A. Clark,et al.  Formulating software engineering as a search problem , 2003, IEE Proc. Softw..

[12]  Jing Wang,et al.  A New Population Initialization Method Based on Space Transformation Search , 2009, 2009 Fifth International Conference on Natural Computation.

[13]  James D. McCaffrey,et al.  Generation of Pairwise Test Sets Using a Genetic Algorithm , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[14]  Kamal Z. Zamli,et al.  The Development of a Particle Swarm Based Optimization Strategy for Pairwise Testing , 2011 .

[15]  Baowen Xu,et al.  Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite , 2008, 2008 The Eighth International Conference on Quality Software.

[16]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[17]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[18]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[19]  Myra B. Cohen,et al.  Covering array sampling of input event sequences for automated gui testing , 2007, ASE.

[20]  Rusli Abdullah,et al.  Design and implementation of a t-way test data generation strategy with automated execution tool support , 2011, Inf. Sci..

[21]  Yuanyuan Zhang,et al.  Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications , 2009 .

[22]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[23]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[24]  Tatsuhiro Tsuchiya,et al.  A new method for constructing pair-wise covering designs for software testing , 2002, Information Processing Letters.

[25]  José Torres-Jiménez,et al.  Simulated Annealing for Constructing Mixed Covering Arrays , 2012, DCAI.

[26]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[27]  Khandakar Rabbi,et al.  EPS2WAY: AN EFFICIENT PAIRWISE TEST DATA GENERATION STRATEGY , 2011 .

[28]  Gadiel Seroussi,et al.  Vector sets for exhaustive testing of logic circuits , 1988, IEEE Trans. Inf. Theory.

[29]  Yoonsik Cheon,et al.  PWiseGen: Generating test cases for pairwise testing using genetic algorithms , 2011, 2011 IEEE International Conference on Computer Science and Automation Engineering.

[30]  Ziyuan Wang,et al.  Generating Variable Strength Covering Array for Combinatorial Software Testing with Greedy Strategy , 2013, J. Softw..

[31]  Jeff Yu Lei,et al.  IPOG: A General Strategy for T-Way Software Testing , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[32]  Kaisa Miettinen,et al.  On initial populations of a genetic algorithm for continuous optimization problems , 2007, J. Glob. Optim..

[33]  Patrick J. Schroeder,et al.  Comparing the fault detection effectiveness of n-way and random test suites , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[34]  Tutut Herawan,et al.  MT2Way: A Novel Strategy for Pair-Wise Test Data Generation , 2012, ISICA.

[35]  Yu-Wen Tung,et al.  Automating test case generation for the new generation mission software system , 2000, 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484).

[36]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[37]  Hareton K. N. Leung,et al.  Empirically Identifying the Best Greedy Algorithm for Covering Array Generation , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[38]  N. J. A. Sloane,et al.  Tables of Orthogonal Arrays , 1999 .

[39]  David E. Goldberg,et al.  The parameter-less genetic algorithm in practice , 2004, Inf. Sci..

[40]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[41]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[42]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[43]  Shahryar Rahnamayan,et al.  A novel population initialization method for accelerating evolutionary algorithms , 2007, Comput. Math. Appl..

[44]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.