Specification-driven model transformation testing

Testing model transformations poses several challenges, among them the automatic generation of appropriate input test models and the specification of oracle functions. Most approaches for the generation of input models ensure a certain coverage of the source meta-model or the transformation implementation code, whereas oracle functions are frequently defined using query or graph languages. However, these two tasks are usually performed independently regardless of their common purpose, and sometimes, there is a gap between the properties exhibited by the generated input models and those considered by the transformations. Recently, we proposed a formal specification language for the declarative formulation of transformation properties (by means of invariants, pre-, and postconditions) from which we generated partial oracle functions used for transformation testing. Here, we extend the usage of our specification language for the automated generation of input test models by SAT solving. The testing process becomes more intentional because the generated models ensure a certain coverage of the transformation requirements. Moreover, we use the same specification to consistently derive both the input test models and the oracle functions. A set of experiments is presented, aimed at measuring the efficacy of our technique.

[1]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[2]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[3]  J. Michael Spivey,et al.  An introduction to Z and formal specifications , 1989, Softw. Eng. J..

[4]  Juan de Lara,et al.  Verification and validation of declarative model-to-model transformations through invariants , 2010, J. Syst. Softw..

[5]  Richard F. Paige,et al.  The Epsilon Transformation Language , 2008, ICMT@TOOLS.

[6]  John C. Grundy,et al.  SoftArch/MTE: Generating Distributed System Test-Beds from High-Level Software Architecture Descriptions , 2004, Automated Software Engineering.

[7]  Yves Le Traon,et al.  Reusable MDA components: a testing-for-trust approach , 2006, MoDELS'06.

[8]  James F. Power,et al.  White-Box Coverage Criteria for Model Transformations , 2009 .

[9]  Ákos Horváth,et al.  Workflow-Driven Tool Integration Using Model Transformations , 2010, Graph Transformations and Model-Driven Engineering.

[10]  Benoit Baudry,et al.  Automatic Model Generation Strategies for Model Transformation Testing , 2009, ICMT@TOOLS.

[11]  Jean-Marc Jézéquel,et al.  Design by Contract to Improve Software Vigilance , 2006, IEEE Transactions on Software Engineering.

[12]  Richard F. Paige,et al.  EUnit: a unit testing framework for model management tasks , 2011, MODELS'11.

[13]  Jordi Cabot,et al.  ATLTest: a white-box test generation approach for ATL transformations , 2012, MODELS'12.

[14]  Martin Gogolla,et al.  Verifying UML/OCL models using Boolean satisfiability , 2010, 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010).

[15]  Ralf Lämmel,et al.  Controllable Combinatorial Coverage in Grammar-Based Testing , 2006, TestCom.

[16]  Yves Le Traon,et al.  Qualifying input test data for model transformations , 2009, Software & Systems Modeling.

[17]  Malte Lochau,et al.  MoSo-PoLiTe: tool support for pairwise and model-based software product line testing , 2011, VaMoS '11.

[18]  Martin Gogolla,et al.  Tractable Model Transformation Testing , 2011, ECMFA.

[19]  Jean Bézivin,et al.  ATL: A model transformation tool , 2008, Sci. Comput. Program..

[20]  Juan de Lara,et al.  Colouring: execution, debug and analysis of QVT-relations transformations through coloured Petri nets , 2012, Software & Systems Modeling.

[21]  Jordi Cabot,et al.  UMLtoCSP: a tool for the formal verification of UML/OCL models using constraint programming , 2007, ASE.

[22]  Shaoying Liu,et al.  Generating test data from SOFL specifications , 1999, J. Syst. Softw..

[23]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Graph Transformation (Monographs in Theoretical Computer Science. An EATCS Series) , 1992 .

[24]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[25]  Nikolai Tillmann,et al.  Automating Software Testing Using Program Analysis , 2008, IEEE Software.

[26]  Jordi Cabot,et al.  Using Models of Partial Knowledge to Test Model Transformations , 2012, ICMT@TOOLS.

[27]  Laurence Duchien,et al.  OCL for the Specification of Model Transformation Contracts , 2004 .

[28]  Gerti Kappel,et al.  Automated verification of model transformations based on visual contracts , 2013, Automated Software Engineering.

[29]  Charles J. Colbourn,et al.  Covering arrays of higher strength from permutation vectors , 2006 .

[30]  Yves Le Traon,et al.  Barriers to systematic model transformation testing , 2010, Commun. ACM.

[31]  Ralf Lämmel,et al.  Two-dimensional Approximation Coverage , 2000, Informatica.

[32]  Jacques Klein,et al.  Pairwise testing for software product lines: comparison of two approaches , 2012, Software Quality Journal.

[33]  Jeffrey G. Gray,et al.  A Testing Framework for Model Transformations , 2005, Model-Driven Software Development.

[34]  Juan de Lara,et al.  A Visual Specification Language for Model-to-Model Transformations , 2010, 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.

[35]  Yves Le Traon,et al.  Mutation Analysis Testing for Model Transformations , 2006, ECMDA-FA.

[36]  Martin Gogolla,et al.  USE: A UML-based specification environment for validating UML and OCL , 2007, Sci. Comput. Program..

[37]  Arend Rensink,et al.  Modelling and analysis using GROOVE , 2010, International Journal on Software Tools for Technology Transfer.

[38]  Juan de Lara,et al.  Engineering model transformations with transML , 2011, Software & Systems Modeling.

[39]  Robert M. Hierons Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[40]  Esther Guerra,et al.  Specification-Driven Test Generation for Model Transformations , 2012, ICMT@TOOLS.

[41]  Hartmut Ehrig,et al.  Formal Analysis of Model Transformations Based on Triple Graph Rules with Kernels , 2008, ICGT.

[42]  Antonio Vallecillo,et al.  A Rewriting Logic Semantics for ATL , 2011, J. Object Technol..

[43]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Graph Transformation , 2006, Monographs in Theoretical Computer Science. An EATCS Series.

[44]  Artur Boronat,et al.  Algebraic Specification of a Model Transformation Engine , 2006, FASE.

[45]  Vicente Pelechano,et al.  Test-Driven Development of Model Transformations , 2009, MoDELS.

[46]  Mario Ornaghi,et al.  A Constructive Approach to Testing Model Transformations , 2010, ICMT@TOOLS.

[47]  Nikolai Tillmann,et al.  Unit tests reloaded: parameterized unit testing with symbolic execution , 2006, IEEE Software.

[48]  Jochen Malte Küster,et al.  Validation of model transformations: first experiences using a white box approach , 2006, MoDELS'06.

[49]  Jochen Malte Küster,et al.  Definition and validation of model transformations , 2006, Software & Systems Modeling.