A two-stage algorithm for combinatorial testing

Covering arrays are combinatorial structures which have applications in fields like software testing and hardware Trojan detection. In this paper we proposed a two-stage simulated annealing algorithm to construct covering arrays. The proposed algorithm is instanced in this paper through the construction of ternary covering arrays of strength three. We were able to get 579 new upper bounds. In order to show the generality of our proposal, we defined a new benchmark composed of 25 instances of MCAs taken from the literature, all instances were improved.

[1]  Brett Stevens,et al.  Group construction of covering arrays , 2005 .

[2]  A. Hartman.  Software and Hardware Testing Using Combinatorial Covering Suites , 2005 .

[3]  José Torres-Jiménez,et al.  A Max-SAT-Based Approach to Constructing Optimal Covering Arrays , 2013, CCIA.

[4]  Myra B. Cohen,et al.  Augmenting simulated annealing to build interaction test suites , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[5]  Bruno Rouzeyre,et al.  On the limitations of logic testing for detecting Hardware Trojans Horses , 2015, 2015 10th International Conference on Design & Technology of Integrated Systems in Nanoscale Era (DTIS).

[6]  José Torres-Jiménez,et al.  New bounds for binary covering arrays using simulated annealing , 2012, Inf. Sci..

[7]  Yu Lei,et al.  Practical Combinatorial Testing , 2010 .

[8]  Charles J. Colbourn,et al.  Roux-type constructions for covering arrays of strengths three and four , 2006, Des. Codes Cryptogr..

[9]  D E Shasha,et al.  Using combinatorial design to study regulation by multiple input signals. A tool for parsimony in the post-genomics era. , 2001, Plant physiology.

[10]  Nelson Rangel-Valdez,et al.  Construction of Orthogonal Arrays of Index Unity Using Logarithm Tables for Galois Fields , 2012 .

[11]  D.M. Cohen,et al.  The Combinatorial Design Approach to Automatic Test Generation , 1996, IEEE Softw..

[12]  Charles J. Colbourn,et al.  Covering arrays from cyclotomy , 2010, Des. Codes Cryptogr..

[13]  Kamal Z. Zamli,et al.  MIPOG - An Efficient t-Way Minimization Strategy for Combinatorial Testing , 2011 .

[14]  Tatsuhiro Tsuchiya,et al.  Using artificial life techniques to generate test cases for combinatorial testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[15]  Feng Duan,et al.  Improving IPOG's vertical growth based on a graph coloring scheme , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[16]  José Torres-Jiménez,et al.  Simulated Annealing for Constructing Mixed Covering Arrays , 2012, DCAI.

[17]  K. A. Bush.  Orthogonal Arrays of Index Unity , 1952 .

[18]  Charles J. Colbourn,et al.  Tabu search for covering arrays using permutation vectors , 2009 .

[19]  N. Sloane.  Covering arrays and intersecting codes , 1993 .

[20]  Myra B. Cohen,et al.  Constructing strength three covering arrays with augmented annealing , 2003, Discret. Math..

[21]  José Torres-Jiménez,et al.  Metaheuristic approach for constructing functional test-suites , 2013, IET Softw..

[22]  Angelo Gargantini,et al.  T‐wise combinatorial interaction test suites construction based on coverage inheritance , 2012, Softw. Test. Verification Reliab..

[23]  Donald L. Kreher,et al.  On the state of strength‐three covering arrays , 2002 .

[24]  José Torres-Jiménez,et al.  Exciting FPGA cryptographic Trojans using combinatorial testing , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[25]  J. M. Pena,et al.  A branch and bound algorithm for ternary covering arrays construction using trinomial coefficients , 2010 .

[26]  Yu Lei,et al.  Refining the In-Parameter-Order Strategy for Constructing Covering Arrays , 2008, Journal of research of the National Institute of Standards and Technology.

[27]  James Norman Cawse,et al.  Experimental Design for Combinatorial and High Throughput Materials Development , 2002 .