Stability of Product Sampling under Product-Line Evolution: Master's Thesis

Product-line sampling is a common method to cope with the exponential growth of products in product-line testing. Over the years, different sampling algorithms have been developed and validated against each other. Researchers strive to create efficient sampling algorithms to cope with large product lines. Typical criteria to evaluate sampling algorithms are the computation needed to calculate a sample, and the number of configurations a generated sample contains. Until now, no evaluation criteria considers the product-line evolution, as a factor for evaluating sampling algorithms. With this master thesis we present the stability of samples under product-line evolution as new evaluation criteria for sampling algorithms. Therefore, we define the meaning of stability in context of product-line evolution. Furthermore, we develop and implement metrics to measure the stability of sampling algorithms. Moreover, we classify whether established sampling algorithms produce stable samples or not, based on the results of our metrics.

[1]  Mathieu Acher,et al.  Feature Model Differences , 2012, CAiSE.

[2]  Thomas Leich,et al.  FeatureIDE: A tool framework for feature-oriented software development , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[3]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[4]  John D. McGregor,et al.  Testing a Software Product Line , 2001, PSSE.

[5]  Sven Apel,et al.  Granularity in software product lines , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[6]  Thomas Leich,et al.  FeatureIDE in a Nutshell , 2017 .

[7]  Øystein Haugen,et al.  Properties of realistic feature models make combinatorial testing of product lines feasible , 2011, MODELS'11.

[8]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[9]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[10]  Gunter Saake,et al.  Similarity-based prioritization in software product-line testing , 2014, SPLC.

[11]  Sebastian Krieter,et al.  IncLing: efficient product-line testing using incremental pairwise sampling , 2016, GPCE.

[12]  Sebastian Krieter,et al.  An Efficient Algorithm for Feature-Model Slicing , 2016 .

[13]  Malte Lochau,et al.  MoSo-PoLiTe: tool support for pairwise and model-based software product line testing , 2011, VaMoS '11.

[14]  Udo Kelter,et al.  Fault-based product-line testing: effective sample generation based on feature-diagram mutation , 2015, SPLC.

[15]  Arie van Deursen,et al.  Analysing the Linux kernel feature model changes using FMDiff , 2015, Software & Systems Modeling.

[16]  Krzysztof Czarnecki,et al.  Coevolution of variability models and related artifacts: a case study from the Linux kernel , 2013, SPLC '13.

[17]  Carlos José Pereira de Lucena,et al.  Refactoring product lines , 2006, GPCE '06.

[18]  Jacques Klein,et al.  Pairwise testing for software product lines: comparison of two approaches , 2012, Software Quality Journal.

[19]  Gunter Saake,et al.  On the impact of the optional feature problem: analysis and case studies , 2009, SPLC.

[20]  Paul Clements,et al.  A framework for software product line practice - version 2 , 1999 .

[21]  Sergio Segura,et al.  Automated analysis of feature models 20 years later: A literature review , 2010, Inf. Syst..

[22]  Jacques Klein,et al.  Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines , 2014, IEEE Transactions on Software Engineering.

[23]  Klaus Schmid,et al.  Analysing the Kconfig semantics and its analysis tools , 2015, GPCE.

[24]  H. W. Kuhn B R Y N Mawr College.  Variants of the Hungarian Method for Assignment Problems' , 1955 .

[25]  Udo Kelter,et al.  Reasoning about product-line evolution using complex feature model differences , 2016, Automated Software Engineering.

[26]  Gunter Saake,et al.  Effective product-line testing using similarity-based product prioritization , 2016, Software & Systems Modeling.

[27]  Mustafa Al-Hajjaji,et al.  A classification of product sampling for software product lines , 2018, SPLC.

[28]  David M. Weiss,et al.  The Product Line Hall of Fame , 2008, 2008 12th International Software Product Line Conference.

[29]  Sebastian Krieter,et al.  Comparing algorithms for efficient feature-model slicing , 2016, SPLC.

[30]  Gunter Saake,et al.  Feature-Oriented Software Product Lines , 2013, Springer Berlin Heidelberg.

[31]  Thomas Leich,et al.  FeatureIDE: An extensible framework for feature-oriented software development , 2014, Sci. Comput. Program..

[32]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[33]  Christian Kästner.  Differential Testing for Variational Analyses: Experience from Developing KConfigReader , 2017, ArXiv.

[34]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[35]  Thomas Thüm,et al.  Reasoning about edits to feature models , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[36]  Thomas Leich,et al.  Mastering Software Variability with FeatureIDE , 2017, Springer International Publishing.

[37]  Jacques Klein,et al.  Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[38]  Sebastian Krieter,et al.  Tool demo: testing configurable systems with FeatureIDE , 2016, GPCE.

[39]  P. Jaccard.  THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[40]  Krzysztof Czarnecki,et al.  The Variability Model of The Linux Kernel , 2010, VaMoS.

[41]  Alexander Egyed,et al.  A first systematic mapping study on combinatorial interaction testing for software product lines , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[42]  Vasek Chvátal,et al.  A Greedy Heuristic for the Set-Covering Problem , 1979, Math. Oper. Res..