Using feature model knowledge to speed up the generation of covering arrays

Combinatorial Interaction Testing has shown great potential for effectively testing Software Product Lines (SPLs). An important part of this type of testing is determining a subset of SPL products in which interaction errors are more likely to occur. Such sets of products are obtained by computing a so called t-wise Covering Array (tCA), whose computation is known to be NP-complete. Recently, the ICPL algorithm has been proposed to compute these covering arrays. In this research-in-progress paper, we propose a set of rules that exploit basic feature model knowledge to reduce the number of elements (i.e. t-sets) required by ICPL without weakening the strength of the generated arrays. We carried out a comparison of runtime performance that shows a significant reduction of the needed execution time for the majority of our SPL case studies.

[1]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[2]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[3]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[4]  Noga Alon,et al.  Algorithmic construction of sets for k-restrictions , 2006, TALG.

[5]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[6]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[7]  Sungwon Kang,et al.  A survey on software product line testing , 2012, SPLC '12.

[8]  Enrique Alba,et al.  Evolutionary algorithm for prioritized pairwise test data generation , 2012, GECCO '12.

[9]  Eduardo Santana de Almeida,et al.  16th International Software Product Line Conference, SPLC '12, Salvador, Brazil - September 2-7, 2012, Volume 1 , 2012, SPLC.

[10]  Myra B. Cohen,et al.  Incremental covering array failure characterization in large configuration spaces , 2009, ISSTA.

[11]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[12]  Jeff Yu Lei,et al.  IPOG: A General Strategy for T-Way Software Testing , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[13]  Krzysztof Czarnecki,et al.  Generative programming - methods, tools and applications , 2000 .

[14]  Sarfraz Khurshid,et al.  Incremental Test Generation for Software Product Lines , 2010, IEEE Transactions on Software Engineering.

[15]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[16]  Klaus Pohl,et al.  The ScenTED Method for Testing Software Product Lines , 2006, Software Product Lines.

[17]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[18]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[19]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[20]  Øystein Haugen,et al.  Properties of realistic feature models make combinatorial testing of product lines feasible , 2011, MODELS'11.

[21]  Don S. Batory,et al.  Scaling step-wise refinement , 2004, IEEE Transactions on Software Engineering.

[22]  Sergio Segura,et al.  Automated analysis of feature models 20 years later: A literature review , 2010, Inf. Syst..

[23]  Sergio Segura Automated Analysis of Feature Models Using Atomic Sets , 2008, SPLC.