Evaluation of Test Case Generation based on a Software Product Line for Model Transformation

Model-Driven Engineering (MDE) supports model evolution and refinement by means of model transformations at several abstraction levels. Validating these transformations is essential to ensure the quality and correctness of such models. However, MDE transformations become more complex to validate, for example, when they are implemented in different languages. One particular example is the transformation of the SyMPLES approach. SyMPLES is a development approach for embedded systems, which is based on concepts of both Software Product Lines (SPL) and MDE. SyMPLES has a model transformation process which creates Simulink models from SysML models. This paper presents a case study which applies test case generation based on SPL to validate this model transformation. An SPL was used to generate a set of test cases based on coverage criteria. The results showed that the test cases generated uncovered errors in the transformation of SyMPLES. In addition, a comparison with the test case generation based on metamodel is presented, in order to analyze the effectiveness of the techniques. The coverage criteria made it possible to reduce the number of test cases generated, thus minimizing test effort and time.

[1]  Itana Maria de Souza Gimenes,et al.  Systematic Management of Variability in UML-based Software Product Lines , 2010, J. Univers. Comput. Sci..

[2]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[3]  Esther Guerra,et al.  Specification-Driven Test Generation for Model Transformations , 2012, ICMT@TOOLS.

[4]  Klaus Schmid,et al.  The Product Line Engineering Approach , 2007 .

[5]  Gianna Reggio,et al.  Early experiences on model transformation testing , 2012, AMT '12.

[6]  Pierre-Yves Schobbens,et al.  Symbolic model checking of software product lines , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[7]  Sergio Segura,et al.  FAMA: Tooling a Framework for the Automated Analysis of Feature Models , 2007, VaMoS.

[8]  Øystein Haugen,et al.  Bow tie testing: a testing pattern for product lines , 2011, EuroPLoP.

[9]  Itana Maria de Souza Gimenes,et al.  Case Study of Test Case Generation Based on Metamodel for Model Transformations , 2017, J. Softw..

[10]  Danilo Beuche.  Modeling and building software product lines with pure::variants , 2011, SPLC '11.

[11]  Jim Steel,et al.  Metamodel-based Test Generation for Model Transformations: an Algorithm and a Tool , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[12]  Jochen Malte Küster,et al.  Validation of model transformations: first experiences using a white box approach , 2006, MoDELS'06.

[13]  Fadi A. Zaraket,et al.  PBCOV: a property-based coverage criterion , 2014, Software Quality Journal.

[14]  Itana Maria de Souza Gimenes,et al.  Application Engineering for Embedded Systems - Transforming SysML Specification to Simulink within a Product-Line based Approach , 2013, ICEIS.

[15]  C.J.H. Mann,et al.  A Practical Guide to SysML: The Systems Modeling Language , 2009 .

[16]  Jeffrey G. Gray,et al.  A Testing Framework for Model Transformations , 2005, Model-Driven Software Development.

[17]  Benoit Baudry,et al.  Automatic Model Generation Strategies for Model Transformation Testing , 2009, ICMT@TOOLS.

[18]  Martin Törngren,et al.  A modular tool integration approach : experiences from two case studies , 2010 .

[19]  Yijun Yu,et al.  Improving feature location using structural similarity and iterative graph mapping , 2013, J. Syst. Softw..

[20]  Frédéric Jouault,et al.  Transforming Models with ATL , 2005, MoDELS.

[21]  Kevin Lano,et al.  A framework for model transformation verification , 2014, Formal Aspects of Computing.

[22]  Andy Schürr,et al.  Model-based pairwise testing for feature interaction coverage in software product line engineering , 2011, Software Quality Journal.

[23]  Stephen J. Mellor,et al.  MDA Distilled Principles Of Model-Driven Architecture , 2004 .

[24]  Flávio Oquendo,et al.  SyMPLES - A SysML-based Approach for Developing Embedded Systems Software Product Lines , 2013, ICEIS.

[25]  B. Baudry,et al.  Validation in model-driven engineering: testing model transformations , 2004, Proceedings. 2004 First International Workshop on Model, Design and Validation, 2004..