Construction of Variable Strength Covering Array for Combinatorial Testing Using a Greedy Approach to Genetic Algorithm

The limitation of time and budget usually prohibits exhaustive testing of interactions between components in a component based software system. Combinatorial testing is a software testing technique that can be used to detect faults in a component based software system caused by the interactions of components in an eective and ecient way. Most of the research in the field of combinatorial testing till now has focused on the construction of optimal covering array (CA) of fixed strength t which covers all t-way interactions among components. The size of CA increases with the increase in strength of testing t, which further increases the cost of testing. However, not all components require higher strength interaction testing. Hence, in a system with k components a technique is required to construct CA of fixed strength t which covers all t-way interactions among k components and all ti-way (where ti > t) interactions between a subset of k components. This is achieved using the variable strength covering array (VSCA). In this paper we propose a greedy based genetic algorithm (GA) to generate optimal VSCA. Experiments are conducted on several benchmark configurations to evaluate the eectiveness of the proposed approach.

[1]  K. Miettinen,et al.  Quasi-random initial population for genetic algorithms , 2004 .

[2]  Zainal Hisham Che Soh,et al.  Variable-Strength Interaction for T-Way Test Generation Strategy , 2013, SOCO 2013.

[3]  Siddhartha R. Dalal,et al.  The Automatic Efficient Test Generator (AETG) system , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[4]  Kaisa Miettinen,et al.  On initial populations of a genetic algorithm for continuous optimization problems , 2007, J. Glob. Optim..

[5]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[6]  Jacek Czerwonka,et al.  Pairwise Testing in Real World , 2006 .

[7]  A. Hartman.  Software and Hardware Testing Using Combinatorial Covering Suites , 2005 .

[8]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[9]  Robert Mandl,et al.  Orthogonal Latin squares: an application of experiment design to compiler testing , 1985, CACM.

[10]  J. Czerwonka.  Pairwise Testing in Real World Practical Extensions to Test Case Generators , 2006 .

[11]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[12]  M. J. Reilly,et al.  An investigation of the applicability of design of experiments to software testing , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..

[13]  Jing Wang,et al.  A New Population Initialization Method Based on Space Transformation Search , 2009, 2009 Fifth International Conference on Natural Computation.

[14]  Yu Lei,et al.  Refining the In-Parameter-Order Strategy for Constructing Covering Arrays , 2008, Journal of research of the National Institute of Standards and Technology.

[15]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[16]  Yvan Labiche,et al.  An Orchestrated Survey of Available Algorithms and Tools for Combinatorial Testing , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[17]  Xiang Chen,et al.  Applying Particle Swarm Optimization to Pairwise Testing , 2010, 2010 IEEE 34th Annual Computer Software and Applications Conference.

[18]  Sangeeta Sabharwal,et al.  An Approach to Test Set Generation for Pair-Wise Testing Using Genetic Algorithms , 2013, SSBSE.

[19]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[20]  Kamal Zuhairi Zamli,et al.  A variable strength interaction test suites generation strategy using Particle Swarm Optimization , 2011, J. Syst. Softw..

[21]  Baowen Xu,et al.  Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite , 2008, 2008 The Eighth International Conference on Quality Software.

[22]  Jeff Yu Lei,et al.  ACTS: A Combinatorial Test Generation Tool , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[23]  Myra B. Cohen,et al.  A variable strength interaction testing of components , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[24]  K. Burr,et al.  Combinatorial Test Techniques : Table-based Automation , Test Generation and Code Coverage , 1998 .

[25]  James D. McCaffrey,et al.  An Empirical Study of Pairwise Test Set Generation Using a Genetic Algorithm , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[26]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[27]  José Torres-Jiménez,et al.  Construction of Mixed Covering Arrays of Variable Strength Using a Tabu Search Approach , 2010, COCOA.

[28]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[29]  Myra B. Cohen,et al.  A deterministic density algorithm for pairwise interaction coverage , 2004, IASTED Conf. on Software Engineering.

[30]  Sam Kwong,et al.  Genetic algorithms: concepts and applications [in engineering design] , 1996, IEEE Trans. Ind. Electron..

[31]  Stefano Marsili-Libelli,et al.  Adaptive mutation in genetic algorithms , 2000, Soft Comput..

[32]  Kamal Z. Zamli,et al.  Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support , 2012, Inf. Softw. Technol..

[33]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.

[34]  N. Garc'ia-Pedrajas,et al.  CIXL2: A Crossover Operator for Evolutionary Algorithms Based on Population Features , 2005, J. Artif. Intell. Res..

[35]  Xiang Chen,et al.  Variable Strength Interaction Testing with an Ant Colony System Approach , 2009, 2009 16th Asia-Pacific Software Engineering Conference.

[36]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[37]  Shahryar Rahnamayan,et al.  A novel population initialization method for accelerating evolutionary algorithms , 2007, Comput. Math. Appl..

[38]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[39]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[40]  Yoonsik Cheon,et al.  PWiseGen: Generating test cases for pairwise testing using genetic algorithms , 2011, 2011 IEEE International Conference on Computer Science and Automation Engineering.

[41]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[42]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[43]  Ziyuan Wang,et al.  Generating Variable Strength Covering Array for Combinatorial Software Testing with Greedy Strategy , 2013, J. Softw..

[44]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..