Using regression testing to analyze the impact of changes to variability models on products

Industrial product lines are typically maintained for a long time and evolve continuously to address changing requirements and new technologies. Already derived products often have to be re-derived after such changes to benefit from new and updated features. Product line engineers thus frequently need to analyze the impact of changes to variability models to prevent unexpected changes of re-derived products. In this paper we present a tool-supported approach that informs engineers about the impacts of variability model changes on existing products. Regression tests are used to determine whether existing product configurations and generated product outputs can be re-derived without unexpected effects. We evaluate the feasibility of the approach based on changes observed in a real-world software product line. More specifically, we show how our approach helps engineers performing specific evolution tasks to analyze the change impacts on existing products. We also evaluate the performance and scalability of our approach. Our results show that variability change impact analyses can be automated using model regression testing and can help reducing the gap between domain engineering and application engineering.

[1]  Florian Heidenreich Towards systematic ensuring well-formedness of software product lines , 2009, FOSD '09.

[2]  Sergio Segura,et al.  Automated metamorphic testing on the analyses of feature models , 2011, Inf. Softw. Technol..

[3]  Don S. Batory,et al.  Feature Models, Grammars, and Propositional Formulas , 2005, SPLC.

[4]  Uirá Kulesza,et al.  Investigating the safe evolution of software product lines , 2011, GPCE '11.

[5]  Paul Grünbacher,et al.  Model-Based Customization and Deployment of Eclipse-Based Tools: Industrial Experiences , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[6]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[7]  Macario Polo,et al.  Testing Product Generation in Software Product Lines Using Pairwise for Features Coverage , 2010, ICTSS.

[8]  Klaus Schmid,et al.  A comparison of decision modeling approaches in product lines , 2011, VaMoS.

[9]  Paul Grünbacher,et al.  Supporting Product Derivation by Adapting and Augmenting Variability Models , 2007, 11th International Software Product Line Conference (SPLC 2007).

[10]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[11]  Paul Grünbacher,et al.  Evolution-Driven Trace Acquisition in Eclipse-Based Product Line Workspaces , 2012, Software and Systems Traceability.

[12]  Peter Knauber Managing the Evolution of Software Product Lines , 2004 .

[13]  Sebastian Oster,et al.  Automated Incremental Pairwise Testing of Software Product Lines , 2010, SPLC.

[14]  Myungchul Kim,et al.  Towards a Formal Framework for Product Line Test Development , 2007, 7th IEEE International Conference on Computer and Information Technology (CIT 2007).

[15]  Dave Astels,et al.  Test Driven Development: A Practical Guide , 2003 .

[16]  Xiaodong Liu,et al.  A product line based aspect-oriented generative unit testing approach to building quality components , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[17]  P. Knauber,et al.  Software Product Line Testing - SPLiT , 2006 .

[18]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[19]  Paul Grünbacher,et al.  The DOPLER meta-tool for decision-oriented variability modeling: a multiple case study , 2011, Automated Software Engineering.

[20]  Birgit Geppert,et al.  Enabling Verifiable Conformance for Product Lines , 2008, 2008 12th International Software Product Line Conference.

[21]  John D. McGregor,et al.  A systematic mapping study of software product lines testing , 2011, Inf. Softw. Technol..

[22]  Krzysztof Czarnecki,et al.  Cool features and tough decisions: a comparison of variability modeling approaches , 2012, VaMoS.

[23]  Reijo Sulonen,et al.  Evolution of Schema and Individuals of Configurable Products , 1999, ER.

[24]  Clémentine Nebut,et al.  Automated requirements-based generation of test cases for product families , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[25]  Tomoji Kishi,et al.  Formal verification and software product lines , 2006, CACM.

[26]  Daniel Diaz,et al.  Using constraint programming to verify DOPLER variability models , 2011, VaMoS.

[27]  Silvio Romero de Lemos Meira,et al.  A Regression Testing Approach for Software Product Lines Architectures , 2010, 2010 Fourth Brazilian Symposium on Software Components, Architectures and Reuse.

[28]  Andy Schürr,et al.  Model-Driven Software Product Line Testing: An Integrated Approach , 2009, SOFSEM.

[29]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[30]  Frank Tip,et al.  Chianti: a tool for change impact analysis of java programs , 2004, OOPSLA.

[31]  Klaus Pohl,et al.  Model-Based System Testing of Software Product Families , 2005, CAiSE.

[32]  Paul Grünbacher,et al.  Facilitating the evolution of products in product line engineering by capturing and replaying configuration decisions , 2012, International Journal on Software Tools for Technology Transfer.

[33]  Paul Grünbacher,et al.  A Flexible Approach for Generating Product-Specific Documents in Product Lines , 2010, SPLC.

[34]  George T. Heineman,et al.  Component-Based Software Engineering: Putting the Pieces Together , 2001 .

[35]  Dirk Muthig,et al.  Making testing product lines more efficient by improving the testability of product line architectures , 2006, ROSATEA '06.

[36]  Antonio Ruiz Cortés,et al.  FAMA Framework , 2008, 2008 12th International Software Product Line Conference.