A Hybrid Artificial Bee Colony and Harmony Search Algorithm to Generate Covering Arrays for Pair-wise Testing

Combinatorial Interaction Testing (CIT) is a cost effective testing technique that aims to detect interaction faults generated as a result of interaction between components or parameters in a software system. CIT requires the generation of effective test sets that cover all possible t-way (t denotes the strength of testing) interactions between parameters. Covering array (CA) and mixed covering array (MCA) are often used to represent test sets. This paper presents a hybrid algorithm that integrates artificial bee colony algorithm (ABC) and harmony search algorithm (HS) to construct CAs for testing all 2-way interactions (pair-wise testing) in software systems. The performance of the proposed hybrid algorithm ABCHS-CAG is compared and analyzed by performing experiments on a set of benchmark problems on pair-wise testing. The results show that ABCHS-CAG generates smaller CAs than its greedy counterparts whereas its performance is comparable to the existing state-of-the-art meta-heuristic algorithms.

[1]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[2]  Yoonsik Cheon,et al.  PWiseGen: Generating test cases for pairwise testing using genetic algorithms , 2011, 2011 IEEE International Conference on Computer Science and Automation Engineering.

[3]  Sangeeta Sabharwal,et al.  An Approach to Test Set Generation for Pair-Wise Testing Using Genetic Algorithms , 2013, SSBSE.

[4]  Bestoun S. Ahmed,et al.  An efficient strategy for covering array construction with fuzzy logic-based adaptive swarm optimization for software testing use , 2015, Expert Syst. Appl..

[5]  Myra B. Cohen,et al.  Learning Combinatorial Interaction Test Generation Strategies Using Hyperheuristic Search , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[6]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[7]  Christian Blum,et al.  Hybrid metaheuristics in combinatorial optimization: A survey , 2011, Appl. Soft Comput..

[8]  N. J. A. Sloane,et al.  Tables of Orthogonal Arrays , 1999 .

[9]  Jacek Czerwonka,et al.  Pairwise Testing in Real World , 2006 .

[10]  Kaile Su,et al.  TCA: An Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[11]  Baowen Xu,et al.  Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite , 2008, 2008 The Eighth International Conference on Quality Software.

[12]  Alan W. Williams,et al.  Determination of Test Configurations for Pair-Wise Interaction Coverage , 2000, TestCom.

[13]  K. Burr,et al.  Combinatorial Test Techniques : Table-based Automation , Test Generation and Code Coverage , 1998 .

[14]  Sangeeta Sabharwal,et al.  ABC-CAG: Covering Array Generator for Pair-wise Testing Using Artificial Bee Colony Algorithm , 2016, e Informatica Softw. Eng. J..

[15]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[16]  Jeff Yu Lei,et al.  IPOG: A General Strategy for T-Way Software Testing , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[17]  Tatsuhiro Tsuchiya,et al.  A new method for constructing pair-wise covering designs for software testing , 2002, Information Processing Letters.

[18]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[19]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[20]  José Torres-Jiménez,et al.  Construction of Mixed Covering Arrays of strengths 2 through 6 Using a Tabu Search Approach , 2012, Discret. Math. Algorithms Appl..

[21]  Yu Lei,et al.  In-parameter-order: a test generation strategy for pairwise testing , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[22]  Kamal Z. Zamli,et al.  The Development of a Particle Swarm Based Optimization Strategy for Pairwise Testing , 2011 .

[23]  Sangeeta Sabharwal,et al.  Construction of Strength Two Mixed Covering Arrays Using Greedy Mutation in Genetic Algorithm , 2015 .

[24]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[25]  Jun Yan,et al.  Generating combinatorial test suite using combinatorial optimization , 2014, J. Syst. Softw..

[26]  Sam Kwong,et al.  Gbest-guided artificial bee colony algorithm for numerical function optimization , 2010, Appl. Math. Comput..

[27]  Yu-Wen Tung,et al.  Automating test case generation for the new generation mission software system , 2000, 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484).

[28]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[29]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[30]  Sangeeta Sabharwal,et al.  Construction of Mixed Covering Arrays for Pair-wise Testing Using Probabilistic Approach in Genetic Algorithm , 2016 .

[31]  José Torres-Jiménez,et al.  Simulated Annealing for Constructing Mixed Covering Arrays , 2012, DCAI.

[32]  Bestoun S. Ahmed,et al.  Achievement of minimized combinatorial test suite for configuration-aware software functional testing using the Cuckoo Search algorithm , 2015, Inf. Softw. Technol..

[33]  Esther Seiden,et al.  On Orthogonal Arrays , 1966 .