Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe

Testing Software Product Lines is a very challenging task due to variability. Frequently, approaches such as combinatorial testing are used to generate representative sets of products for testing purposes instead of testing each individual product of the SPL under test. In this contribution we present the results of applying the MoSo-PoLiTe framework at Danfoss Power Electronics A/S to calculate a representative set of product configurations for black box testing purposes. Within this evaluation we use MoSo-PoLiTe's pairwise configuration selection component on the basis of a feature model. This component implements a heuristics finding a minimal subset of configurations covering 100% T-wise feature interaction. According to the best of our knowledge, this is the first publication providing industrial results about pairwise SPL testing.

[1]  Brett Stevens,et al.  Efficient software testing protocols , 1998, CASCON.

[2]  Vijayan N. Nair,et al.  A statistical assessment of some software testing strategies and application of experimental design techniques , 1998 .

[3]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[4]  Emelie Engström,et al.  Regression Test Selection and Product Line System Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[5]  Sigrid Eldh.  Software Testing Techniques , 2007 .

[6]  Erika Mir Olimpiew,et al.  Model-Based Testing for Software Product Lines , 2008 .

[7]  Andy Schürr,et al.  Model-based coverage-driven test suite generation for software product lines , 2011, MODELS'11.

[8]  Andy Schürr,et al.  Model-based pairwise testing for feature interaction coverage in software product line engineering , 2011, Software Quality Journal.

[9]  M. Petre,et al.  AETG vs . Man : an Assessment of the Effectiveness of Combinatorial Test Data Generation , 2007 .

[10]  Juha Taina,et al.  Product family testing: a survey , 2004, SOEN.

[11]  Jacques Klein,et al.  Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[12]  John D. McGregor,et al.  Testing a Software Product Line , 2001, PSSE.

[13]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[14]  Danilo Beuche,et al.  Minimally Invasive Migration to Software Product Lines , 2007, 11th International Software Product Line Conference (SPLC 2007).

[15]  Klaus Schmid,et al.  Software product lines in action - the best industrial practice in product line engineering , 2007 .

[16]  Danilo Beuche,et al.  Running a software product line: standing still is going backwards , 2009, SPLC.

[17]  Per Runeson,et al.  A systematic review on regression test selection techniques , 2010, Inf. Softw. Technol..

[18]  Klaus Pohl,et al.  Model-Based System Testing of Software Product Families , 2005, CAiSE.

[19]  Malte Lochau,et al.  MoSo-PoLiTe: tool support for pairwise and model-based software product line testing , 2011, VaMoS '11.

[20]  Hassan Gomaa,et al.  Model-Based Testing for Software Product Lines A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University , 2008 .

[21]  Antonia Bertolino,et al.  Use case-based testing of product lines , 2003, ESEC/FSE-11.

[22]  D. Richard Kuhn,et al.  FAILURE MODES IN MEDICAL DEVICE SOFTWARE: AN ANALYSIS OF 15 YEARS OF RECALL DATA , 2001 .

[23]  Wolfram Koepf,et al.  Lecture Notes in Computer Science (LNCS) , 2011 .

[24]  Myra B. Cohen,et al.  Interaction testing of highly-configurable systems in the presence of constraints , 2007, ISSTA '07.

[25]  Pieter J. Mosterman,et al.  Model-Based Testing for Embedded Systems , 2011, Computational Analysis, Synthesis, & Design Dynamic Systems.

[26]  M. J. Reilly,et al.  An investigation of the applicability of design of experiments to software testing , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..

[27]  Sarfraz Khurshid,et al.  Reducing combinatorics in testing product lines , 2011, AOSD '11.

[28]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[29]  Kathrin Danielle Scheidemann.  Verifying families of system configurations , 2008 .