Combinatorial Testing for Feature Models Using CitLab

Feature models are commonly used to represent product lines and systems with a set of features interrelated each others. Test generation from feature models, i.e. generating a valid and representative subset of all the possible product configurations, is still an open challenge. A common approach is to build combinatorial interaction test suites, for instance achieving pairwise coverage among the features. In this paper we show how standard feature models can be translated to combinatorial interaction models in our framework CITLAB, with all the advantages of having a combinatorial testing environment (in terms of a clear semantics, editing facilities, language for seeds and test goals, and generation algorithms). We present our translation which gives a precise semantics to feature models and it tries to minimize the number of parameter and constraints while preserving the original semantics of the feature model. Experiments show the advantages of our approach.

[1]  Thomas Leich,et al.  FeatureIDE: An extensible framework for feature-oriented software development , 2014, Sci. Comput. Program..

[2]  Malte Lochau,et al.  MoSo-PoLiTe: tool support for pairwise and model-based software product line testing , 2011, VaMoS '11.

[3]  Jacques Klein,et al.  Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[4]  Donald D. Cowan,et al.  S.P.L.O.T.: software product lines online tools , 2009, OOPSLA Companion.

[5]  Myra B. Cohen,et al.  Coverage and adequacy in software product line testing , 2006, ROSATEA '06.

[6]  Myra B. Cohen,et al.  An Improved Meta-heuristic Search for Constrained Interaction Testing , 2009, 2009 1st International Symposium on Search Based Software Engineering.

[7]  Don S. Batory,et al.  Feature Models, Grammars, and Propositional Formulas , 2005, SPLC.

[8]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[9]  Angelo Gargantini,et al.  Combinatorial Interaction Testing with CITLAB , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[10]  Angelo Gargantini,et al.  CITLAB: A Laboratory for Combinatorial Interaction Testing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[11]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[12]  Charles J. Colbourn,et al.  One-test-at-a-time heuristic search for interaction test suites , 2007, GECCO '07.

[13]  Yu Lei,et al.  IPOG-IPOG-D: efficient test generation for multi-way combinatorial testing , 2008 .

[14]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[15]  Arnaud Gotlieb,et al.  PACOGEN: Automatic Generation of Pairwise Test Configurations from Feature Models , 2011, 2011 IEEE 22nd International Symposium on Software Reliability Engineering.

[16]  Sergio Segura,et al.  Automated analysis of feature models 20 years later: A literature review , 2010, Inf. Syst..

[17]  Jacques Klein,et al.  Pairwise testing for software product lines: comparison of two approaches , 2012, Software Quality Journal.

[18]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[19]  Pierre-Yves Schobbens,et al.  Generic semantics of feature diagrams , 2007, Comput. Networks.