Model-Based Testing

Software more and more pervades our everyday lives. Hence, we have high requirements towards the trustworthiness of the software. Software testing greatly contributes to the quality assurance of modern software systems. However, as today's software system get more and more complex and exist in many different variants, we need rigorous and systematic approaches towards software testing. In this tutorial, we, first, present model-based testing as an approach for systematic test case generation, test execution and test result evaluation for single system testing. The central idea of model-based testing is to base all testing activities on an executable model-based test specification. Second, we consider model-based testing for variant-rich software systems and review two model-based software product line testing techniques. Sample-based testing generates a set of representative variants for testing, and variability-aware product line testing uses a family-based test model which contains the model-based specification of all considered product variants.

[1]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[2]  Malte Lochau,et al.  Feature Interaction Aware Test Case Generation for Embedded Control Systems , 2010, Electron. Notes Theor. Comput. Sci..

[3]  Malte Lochau,et al.  MoSo-PoLiTe: tool support for pairwise and model-based software product line testing , 2011, VaMoS '11.

[4]  Hassan Gomaa,et al.  Model-Based Testing for Software Product Lines A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University , 2008 .

[5]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems: Advanced Lectures (Lecture Notes in Computer Science) , 2005 .

[6]  Michiel van Osch.  Hybrid Input-Output Conformance and Test Generation , 2006, FATES/RV.

[7]  Goetz Botterweck,et al.  Software diversity: state of the art and perspectives , 2012, International Journal on Software Tools for Technology Transfer.

[8]  Erika Mir Olimpiew,et al.  Model-Based Testing for Software Product Lines , 2008 .

[9]  Kim G. Larsen,et al.  A modal process logic , 1988, [1988] Proceedings. Third Annual Information Symposium on Logic in Computer Science.

[10]  Kim G. Larsen,et al.  Modal I/O Automata for Interface and Product Line Theories , 2007, ESOP.

[11]  Sarfraz Khurshid,et al.  Reducing combinatorics in testing product lines , 2011, AOSD '11.

[12]  John D. McGregor,et al.  Testing a Software Product Line , 2001, PSSE.

[13]  Maurice H. ter Beek,et al.  A Model-Checking Tool for Families of Services , 2011, FMOODS/FORTE.

[14]  Krzysztof Czarnecki,et al.  SAT-based analysis of feature models is easy , 2009, SPLC.

[15]  Jacques Klein,et al.  Multi-objective test generation for software product lines , 2013, SPLC '13.

[16]  Arend Rensink,et al.  Compositional Testing with ioco , 2003, FATES.

[17]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[18]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[19]  Maurice H. ter Beek,et al.  A Logical Framework to Deal with Variability , 2010, IFM.

[20]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[21]  Sebastián Uchitel,et al.  A foundation for behavioural conformance in software product line architectures , 2006, ROSATEA '06.

[22]  Vasek Chvátal,et al.  A Greedy Heuristic for the Set-Covering Problem , 1979, Math. Oper. Res..

[23]  Michal Antkiewicz,et al.  Mapping features to models: a template approach based on superimposed variants , 2005, GPCE'05.

[24]  Andy Schürr,et al.  Model-based coverage-driven test suite generation for software product lines , 2011, MODELS'11.

[25]  Rocco De Nicola,et al.  Extensional equivalences for transition systems , 1987, Acta Informatica.

[26]  Emelie Engström,et al.  Regression Test Selection and Product Line System Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[27]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[28]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[29]  Sandro Schulze,et al.  Towards efficient SPL testing by variant reduction , 2013, VariComp@AOSD.

[30]  Malte Lochau,et al.  Pairwise feature-interaction testing for SPLs: potentials and limitations , 2011, SPLC '11.

[31]  Juha Savolainen,et al.  Feature Interaction and Dependencies: Modeling Features for Reengineering a Legacy Product Line , 2002, SPLC.

[32]  Andy Schürr,et al.  Model-based pairwise testing for feature interaction coverage in software product line engineering , 2011, Software Quality Journal.

[33]  Maurice H. ter Beek,et al.  Formal Description of Variability in Product Families , 2011, 2011 15th International Software Product Line Conference.

[34]  Gilles Bernot,et al.  Testing Against Formal Specifications: A Theoretical View , 1991, TAPSOFT, Vol.2.

[35]  Rolf Hennicker,et al.  Interface Theories for (A)synchronously Communicating Modal I/O-Transition Systems , 2010, FIT.

[36]  Julien Schmaltz,et al.  On Conformance Testing for Timed Systems , 2008, FORMATS.

[37]  Malte Lochau,et al.  Incremental Model-Based Testing of Delta-Oriented Software Product Lines , 2012, TAP@TOOLS.

[38]  Juha Taina,et al.  Product family testing: a survey , 2004, SOEN.

[39]  Jacques Klein,et al.  Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[40]  Paul Clements,et al.  Software product lines - practices and patterns , 2001, SEI series in software engineering.

[41]  Myra B. Cohen,et al.  Coverage and adequacy in software product line testing , 2006, ROSATEA '06.

[42]  Thomas Gustafsson.  An Approach for Selecting Software Product Line Instances for Testing , 2007, SPLC.

[43]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[44]  Axel Legay,et al.  Modal interfaces: unifying interface automata and modal specifications , 2009, EMSOFT '09.

[45]  Krzysztof Czarnecki,et al.  Generative programming - methods, tools and applications , 2000 .

[46]  Øystein Haugen,et al.  Properties of realistic feature models make combinatorial testing of product lines feasible , 2011, MODELS'11.

[47]  Klaus Pohl,et al.  Model-Based System Testing of Software Product Families , 2005, CAiSE.

[48]  Thomas A. Henzinger,et al.  Alternating Refinement Relations , 1998, CONCUR.

[49]  Walter Vogler,et al.  Modal Interface Automata , 2012, IFIP TCS.

[50]  John D. McGregor,et al.  A systematic mapping study of software product lines testing , 2011, Inf. Softw. Technol..

[51]  John D. McGregor,et al.  Corrigendum to: "A systematic mapping study of software product lines testing" [Inf. Softw. Technology 53 (5) (2011) 407-423] , 2012, Information and Software Technology.