A Decade of Featured Transition Systems

Variability-intensive systems (VIS) form a large and heterogeneous class of systems whose behaviour can be modified by enabling or disabling predefined features. Variability mechanisms allows the adaptation of software to the needs of their users and the environment. However, VIS verification and validation (V&V) is challenging: the combinatorial explosion of the number of possible behaviours and undesired feature interactions are amongst such challenges. To tackle them, Featured Transitions Systems (FTS) were proposed a decade ago to model and verify the behaviours of VIS. In an FTS, each transition is annotated with a combination of features determining which variants can execute it. An FTS can model all possible behaviours of a given VIS. This compact model enabled us to create efficient V&V algorithms taking advantage of the behaviours shared amongst features resulting in a reduction of the V&V effort by several orders of magnitude. In this paper, we will cover the formalism, its applications and sketch promising research directions.

[1]  Jacques Klein,et al.  Pairwise testing for software product lines: comparison of two approaches , 2012, Software Quality Journal.

[2]  Axel Legay,et al.  Long-term average cost in featured transition systems , 2016, SPLC.

[3]  Sarfraz Khurshid,et al.  Incremental Test Generation for Software Product Lines , 2010, IEEE Transactions on Software Engineering.

[4]  Martin Leucker,et al.  Modeling and Model Checking Software Product Lines , 2008, FMOODS.

[5]  Jacques Klein,et al.  Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines , 2014, IEEE Transactions on Software Engineering.

[6]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[7]  Sven Apel,et al.  Toward variability-aware testing , 2012, FOSD '12.

[8]  Axel Legay,et al.  Trace Checking for Dynamic Software Product Lines , 2018, 2018 IEEE/ACM 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[9]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[10]  Pierre-Yves Schobbens,et al.  Counterexample guided abstraction refinement of product-line behavioural models , 2014, Software Engineering.

[11]  Pierre-Yves Schobbens,et al.  Model checking software product lines with SNIP , 2012, International Journal on Software Tools for Technology Transfer.

[12]  Klaus Pohl,et al.  Software Product Line Engineering , 2005 .

[13]  Mehrdad Sabetzadeh,et al.  Testing the Untestable - Model Testing of Complex Software-Intensive Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[14]  Mathieu Acher,et al.  Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack , 2017, Empirical Software Engineering.

[15]  Mahsa Varshosaz,et al.  Basic behavioral models for software product lines: Expressiveness and testing pre-orders , 2016, Sci. Comput. Program..

[16]  Pierre-Yves Schobbens,et al.  Featured Model-Based Mutation Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[17]  David Notkin,et al.  Editorial—looking back , 2013, TSEM.

[18]  Tim Menzies,et al.  On the value of user preferences in search-based software engineering: A case study in software product lines , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[19]  Krzysztof Czarnecki,et al.  Sample Spaces and Feature Models: There and Back Again , 2008, 2008 12th International Software Product Line Conference.

[20]  Gilles Perrouin,et al.  AbsCon: A Test Concretizer for Model-Based Testing , 2019, 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[21]  Sarfraz Khurshid,et al.  Reducing Configurations to Monitor in a Software Product Line , 2010, RV.

[22]  Mohammad Reza Mousavi,et al.  Input-output conformance testing for software product lines , 2016, J. Log. Algebraic Methods Program..

[23]  Patrícia Duarte de Lima Machado,et al.  On the use of a similarity function for test case selection in the context of model‐based testing , 2011, Softw. Test. Verification Reliab..

[24]  Andy Schürr,et al.  Model-based coverage-driven test suite generation for software product lines , 2011, MODELS'11.

[25]  Pierre-Yves Schobbens,et al.  Modeling and Verification for Probabilistic Properties in Software Product Lines , 2015, 2015 IEEE 16th International Symposium on High Assurance Systems Engineering.

[26]  Clémentine Nebut,et al.  Automated requirements-based generation of test cases for product families , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[27]  Gunter Saake,et al.  Feature-Oriented Software Product Lines , 2013, Springer Berlin Heidelberg.

[28]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[29]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[30]  Pierre-Yves Schobbens,et al.  Behavioural modelling and verification of real-time software product lines , 2012, SPLC '12.

[31]  Christel Baier,et al.  ProFeat: feature-oriented engineering for family-based probabilistic model checking , 2017, Formal Aspects of Computing.

[32]  Donald D. Cowan,et al.  S.P.L.O.T.: software product lines online tools , 2009, OOPSLA Companion.

[33]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[34]  Myra B. Cohen,et al.  Coverage and adequacy in software product line testing , 2006, ROSATEA '06.

[35]  Philippe Collet,et al.  Multifaceted Automated Analyses for Variability-Intensive Embedded Systems , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[36]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[37]  Mohammad Reza Mousavi,et al.  Spinal Test Suites for Software Product Lines , 2014, MBT.

[38]  Erik P. de Vink,et al.  Coherent branching feature bisimulation , 2015, FMSPLE.

[39]  Dragan Gasevic,et al.  Evolutionary Search-Based Test Generation for Software Product Line Feature Models , 2012, CAiSE.

[40]  Myra B. Cohen,et al.  Evaluating improvements to a meta-heuristic search for constrained interaction testing , 2011, Empirical Software Engineering.

[41]  Pierre-Yves Schobbens,et al.  Featured Transition Systems: Foundations for Verifying Variability-Intensive Systems and Their Application to LTL Model Checking , 2013, IEEE Transactions on Software Engineering.

[42]  Andy Schürr,et al.  Model-based pairwise testing for feature interaction coverage in software product line engineering , 2011, Software Quality Journal.

[43]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[44]  Andrzej Wąsowski,et al.  From Transition Systems to Variability Models and from Lifted Model Checking Back to UPPAAL , 2017, Models, Algorithms, Logics and Tools.

[45]  Sven Apel,et al.  Strategies for product-line verification: Case studies and experiments , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[46]  Andrzej Wasowski,et al.  Variability-Specific Abstraction Refinement for Family-Based Model Checking , 2017, FASE.

[47]  Wang Yi,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2019, Lecture Notes in Computer Science.

[48]  Cem Kaner,et al.  Foundations of Software Testing , 2013 .

[49]  Erik P. de Vink,et al.  Family-Based Model Checking with mCRL2 , 2017, FASE.

[50]  Pierre-Yves Schobbens,et al.  Model checking lots of systems: efficient verification of temporal properties in software product lines , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[51]  Axel Legay,et al.  A Framework for Quantitative Modeling and Analysis of Highly (Re)configurable Systems , 2017, IEEE Transactions on Software Engineering.

[52]  Pierre-Yves Schobbens,et al.  ProVeLines: a product line of verifiers for software product lines , 2013, SPLC '13 Workshops.

[53]  Antonia Bertolino,et al.  Product Line Use Cases: Scenario-Based Specification and Testing of Requirements , 2006, Software Product Lines.

[54]  Axel Legay,et al.  Variability Abstraction and Refinement for Game-based Lifted Model Checking of full CTL (Extended Version) , 2019, FASE.

[55]  Gunter Saake,et al.  Similarity-based prioritization in software product-line testing , 2014, SPLC.

[56]  Shigeru Hosono,et al.  Genetic Algorithm-based Test Generation for Software Product Line with the Integration of Fault Localization Techniques , 2017, Empirical Software Engineering.

[57]  Jan Tretmans,et al.  Model-Based Testing and Some Steps towards Test-Based Modelling , 2011, SFM.

[58]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[59]  Pierre-Yves Schobbens,et al.  Formal semantics, modular specification, and symbolic verification of product-line behaviour , 2014, Sci. Comput. Program..

[60]  Yves Le Traon,et al.  Combining Multi-Objective Search and Constraint Solving for Configuring Large Software Product Lines , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[61]  John D. McGregor,et al.  Testing a Software Product Line , 2001, PSSE.

[62]  Axel Legay,et al.  Statistical Model Checking: An Overview , 2010, RV.

[63]  Shai Ben-David,et al.  Learnability can be undecidable , 2019, Nat. Mach. Intell..

[64]  Maurice H. ter Beek,et al.  Formal Description of Variability in Product Families , 2011, 2011 15th International Software Product Line Conference.

[65]  Christel Baier,et al.  Probabilistic model checking for energy analysis in software product lines , 2013, MODULARITY.

[66]  Stefania Gnesi,et al.  A behavioural model for product families , 2007, ESEC/SIGSOFT FSE.

[67]  Pierre-Yves Schobbens,et al.  Feature Diagrams: A Survey and a Formal Semantics , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[68]  Pierre-Yves Schobbens,et al.  Statistical prioritization for software product line testing: an experience report , 2015, Software & Systems Modeling.

[69]  Pierre-Yves Schobbens,et al.  Search-based Similarity-driven Behavioural SPL Testing , 2016, VaMoS.

[70]  Malte Lochau,et al.  Incremental Model-Based Testing of Delta-Oriented Software Product Lines , 2012, TAP@TOOLS.

[71]  Krzysztof Czarnecki,et al.  Feature Diagrams and Logics: There and Back Again , 2007, 11th International Software Product Line Conference (SPLC 2007).

[72]  Pierre-Yves Schobbens,et al.  Towards an incremental automata-based approach for software product-line model checking , 2012, SPLC '12.

[73]  Hung Viet Nguyen,et al.  Exploring variability-aware execution for testing plugin-based web applications , 2014, ICSE.

[74]  Krzysztof Czarnecki,et al.  SMTIBEA: a hybrid multi-objective optimization algorithm for configuring large constrained software product lines , 2019, Software & Systems Modeling.

[75]  Maurice H. ter Beek,et al.  From Featured Transition Systems to Modal Transition Systems with Variability Constraints , 2015, SEFM.

[76]  Pierre-Yves Schobbens,et al.  Poster: VIBeS, Transition System Mutation Made Easy , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[77]  Lijun Zhang,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2019, Lecture Notes in Computer Science.

[78]  Lionel C. Briand,et al.  Achieving scalable model-based testing through test case diversity , 2013, TSEM.

[79]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[80]  Pierre-Yves Schobbens,et al.  Simulation-based abstractions for software product-line model checking , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[81]  Markus Roggenbach,et al.  On the use of test cases in model-based software product line development , 2014, SPLC.

[82]  Alexander Egyed,et al.  A first systematic mapping study on combinatorial interaction testing for software product lines , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[83]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[84]  Pierre-Yves Schobbens,et al.  Symbolic model checking of software product lines , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[85]  Antonia Bertolino,et al.  PLUTO: A Test Methodology for Product Families , 2003, PFE.

[86]  Mohammad Reza Mousavi,et al.  Input-output conformance testing based on featured transition systems , 2014, SAC.

[87]  Sven Apel,et al.  The PLA model: on the combination of product-line analyses , 2013, VaMoS '13.

[88]  Gunter Saake,et al.  Feature-Oriented Software Product Lines , 2013, Springer Berlin Heidelberg.

[89]  Axel Legay,et al.  Towards Sampling and Simulation-Based Analysis of Featured Weighted Automata , 2019, 2019 IEEE/ACM 7th International Conference on Formal Methods in Software Engineering (FormaliSE).

[90]  Axel Legay,et al.  A Complexity Tale: Web Configurators , 2016, 2016 IEEE/ACM 1st International Workshop on Variability and Complexity in Software Design (VACE).

[91]  Malte Lochau,et al.  Delta-oriented model-based integration testing of large-scale systems , 2014, J. Syst. Softw..

[92]  Malte Lochau,et al.  Modeling and Testing Product Lines with Unbounded Parametric Real-Time Constraints , 2017, SPLC.

[93]  Axel Legay,et al.  Merging Features in Featured Transition Systems , 2015, MoDeVVa@MoDELS.

[94]  Christel Baier,et al.  Principles of model checking , 2008 .

[95]  Axel Legay,et al.  Statistical analysis of probabilistic models of software product lines with quantitative constraints , 2015, SPLC.

[96]  Maurice H. ter Beek,et al.  Modelling and analysing variability in product families: Model checking of modal transition systems with variability constraints , 2016, J. Log. Algebraic Methods Program..

[97]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .