A parallel evolutionary algorithm for prioritized pairwise testing of software product lines

Software Product Lines (SPLs) are families of related software systems, which provide different feature combinations. Different SPL testing approaches have been proposed. However, despite the extensive and successful use of evolutionary computation techniques for software testing, their application to SPL testing remains largely unexplored. In this paper we present the Parallel Prioritized product line Genetic Solver (PPGS), a parallel genetic algorithm for the generation of prioritized pairwise testing suites for SPLs. We perform an extensive and comprehensive analysis of PPGS with 235 feature models from a wide range of number of features and products, using 3 different priority assignment schemes and 5 product prioritization selection strategies. We also compare PPGS with the greedy algorithm prioritized-ICPL. Our study reveals that overall PPGS obtains smaller covering arrays with an acceptable performance difference with prioritized-ICPL.

[1]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[2]  Ron Shonkwiler,et al.  Parallel Genetic Algorithms , 1993, ICGA.

[3]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[4]  Antonio Ruiz Cortés,et al.  FAMA Framework , 2008, 2008 12th International Software Product Line Conference.

[5]  Robert Feldt,et al.  Software Product Line , 2008 .

[6]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[7]  Sergio Segura,et al.  Automated analysis of feature models 20 years later: A literature review , 2010, Inf. Syst..

[8]  Informatika.  Software Product Line , 2010 .

[9]  Jacques Klein,et al.  Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[10]  Phil McMinn,et al.  Search-Based Software Testing: Past, Present and Future , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[11]  Sven Apel,et al.  Scalable Prediction of Non-functional Properties in Software Product Lines , 2011, 2011 15th International Software Product Line Conference.

[12]  John D. McGregor,et al.  A systematic mapping study of software product lines testing , 2011, Inf. Softw. Technol..

[13]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[14]  Andy Schürr,et al.  Model-based coverage-driven test suite generation for software product lines , 2011, MODELS'11.

[15]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[16]  Arnaud Gotlieb,et al.  PACOGEN: Automatic Generation of Pairwise Test Configurations from Feature Models , 2011, 2011 IEEE 22nd International Symposium on Software Reliability Engineering.

[17]  Øystein Haugen,et al.  Generating better partial covering arrays by modeling weights on sub-product lines , 2012, MODELS'12.

[18]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[19]  Dragan Gasevic,et al.  Evolutionary Search-Based Test Generation for Software Product Line Feature Models , 2012, CAiSE.

[20]  Jacques Klein,et al.  Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-wise Test Suites for Large Software Product Lines , 2012, ArXiv.

[21]  Enrique Alba,et al.  Evolutionary algorithm for prioritized pairwise test data generation , 2012, GECCO '12.

[22]  C. A. Murthy,et al.  Variance as a Stopping Criterion for Genetic Algorithms with Elitist Model , 2012, Fundam. Informaticae.

[23]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[24]  Lionel C. Briand,et al.  A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering , 2014, Softw. Test. Verification Reliab..