Testing M2M/M2T/T2M Transformations

As Model-Driven Engineering is becoming adopted by industry, models and model transformations (MTs) are extensively used. Hence, there is the urgent need for systematic testing mechanisms and tools to check their correctness. In this work, we make use of a particular case of contracts for model transformations called Tracts. First, Tracts allow the transformation developer to specify and test a model-to-model transformation in a modular way, and to identify bugs. However, they do not allow to track where the faults in the implementation are. For doing that, we present an approach based on matching functions that automatically establish the alignments between the specification and the implementation of a transformation using the metamodel footprints. Second, we extend Tracts to deal with text-to-model and model-to-text transformations in order to broaden and complete the scope of our testing proposal. Finally, we provide the corresponding tools that realize our proposal. I. PROBLEM AND MOTIVATION Model transformations (MT) are gaining more and more interest as industry is progressively adopting model-driven techniques [1]. The main advantage of using of model transformations is to save effort and reduce errors by automating the creation and alteration of models as long as it is possible. Thus, they are becoming a promising approach in many different scenarios to solve a wide variety of problems, e.g. to deal with the migration of systems, their modernization, for code generation, etc., especially when complex data structures are involved. This complexity may lead to the existence of bugs in the model transformation implementation that make it faulty. Then, the need of testing, validation and verification procedures for model transformations is emerging in recent years [2], [3]. So far, most of the efforts by the research community have focused on testing model-to-model (M2M) transformations for which having explicit model representations for the input and output domain is assumed. There are different approaches that can be classified attending to their characteristics as blackbox vs. white-box and static vs. dynamic. Depending on the concrete situation, the transformation developer needs to make the decision of what mechanism to use. When black-box dynamic approaches such as Tracts [4] are the best option, the developer finds that they do allow the testing of model transformations but they do not track where the problem is in the implementation, i.e., they reveal that there is a problem but they do not point to where it is or what is producing it. Furthermore, text-to-model (T2M) and model-to-text (M2T) transformations are extensively used [5] for code generation and reverse engineering for the modernization of legacy applications [6]. However, they have received little attention so far by the research community. The contribution presented in this paper is twofold. First, we have extended the Tract approach for M2T and T2M transformations. We have created a generic metamodel that represents text repositories and inject the text into a model that conforms to that metamodel. Once both source and target domains count on a concrete and well-defined representation, M2T and T2M transformations are reduced to M2M transformations. Therefore, Tracts can be used for checking their correctness. Second, we define a mechanism based on matching tables that permit relating the rules of a model transformation with its Tracts, i.e., aligning the model transformation implementation with its specification. By analysing the matching tables, the rules that cause a fault can be identified, hence realizing a useful tracking mechanism for locating faults in model transformations. The structure of the paper is as follows. Section II introduces the concepts in which this work stands and the related work. Section III presents the core of our contribution: the extension of Tracts for M2T and T2M transformations and how the matching tables are computed and interpreted. Finally, Section IV shows the results we have obtained and the contributions we have made. II. BACKGROUND AND RELATED WORK The need for systematic verification of model transformations has been studied in previous works and the challenges it has to deal with have been outlined [7], [8]. Many approaches ranging from lightweight certification to full verification have been proposed to reason about different kinds of properties of M2M transformations [2], [3]. One of them is the use of contracts [4], [9], [10]. Tracts, which are a particular case of contracts, are a blackbox testing mechanism for M2M transformations. They consist of a set of constraints on the source and target metamodels, a set of source-target constraints, and a test suite, i.e., a collection of source models. They provide modular pieces of specification, each one focusing on a particular transformation scenario. This permits each model transformation to be specified by means of a set of Tracts, each one covering a specific use case. Usually, they are seen as unit tests, which means that developers identify the scenarios of interest and define a Tract for each one. Then, they check whether the transformation behaves as expected in these scenarios. Other works proposed alternative ways for defining oracles [11]–[15], however, they do not discuss how to apply their approaches for text artifacts. The most closely related work for testing M2T transformations is presented in [16]. Nevertheless, this approach requires the definition of a functional decomposition diagram for the M2T transformation as well as the design specifications for the text produced by the transformation. Tracking guilty transformation rules using a dynamic approach where constraints are involved has also been subject to investigations [17], [18]. In [19]–[21], the authors locate errors using the trace information of the MT executions, i.e., determining the relationship between the source and target elements and the excerpt of the transformation involved. The dynamic approach is also used in [22] to build slices of model transformations and in [23] following a white-box testing approach. All these approaches need to count on input models while our aim is to statically build more general traceability models between the specification of the MTs and their implementations. III. APPROACH AND UNIQUENESS A. Extending Tracts for M2T and T2M transformations In order to test model transformations when text is involved in one of the domains, either in the source or the target, we propose an approach that converts the problem to a M2M transformation testing problem [24]. In order to achieve that, instead of defining a specific grammar or metamodel for each text artifact, we have opted for creating a generic metamodel that represents text repositories. Then, the folder structure and text files are injected to a model conforming to the text metamodel. The metamodel is shown in Figure 1. It counts on a metaclass Repository that represents the entry point to the root folder containing folders and files or to a file if only one single artifact is used. Folders just contain a name while files have in addition an extension as well as a content. The content of files is represented by lines that are sequentially ordered. A derived attribute content is used to allow easy access to the complete content of a file. Figure 2 displays on its left-hand side the folder structure of a Java project while on its right-hand side the content of one of its Java files. Figure 3 presents an excerpt of the text model corresponding to the elements that Figure 2 shows and several lines of the Java file. The specification of the transformation to be tested is composed of a set of Tracts, each one focusing on a particular property that the developer wants to ensure. The constraints defined by those Tracts are OCL expressions. Thus, a problem arises when the developer needs to deal with the text represented by the lines but realises that the variety of libraries and operations that OCL provides to manage Strings is reduced and very restrictive. As the text in the lines may need to be Fig. 1. Metamodel for representing text artifacts and repositories. Fig. 2. Exemplary folder structure and file content. analysed thoroughly, we have enriched OCL with an operation called matchesRE() that checks whether a given string matches a regular expression. Furthermore, we have introduced some auxiliary functions that are currently provided by M2T transformation languages such as toFirstUpper() to end up with more concise OCL constraints than just using the standard OCL String operation library. In order to illustrate what a Tract looks like, let us assume that we are testing a M2T transformation that generates Java code from UML models. Let us also assume a very simplified UML metamodel that only has Packages, Classes and Properties. All of them have a name and the properties have a type as well. Furthermore, each package may contain a set of classes and each class may contain set of properties. The metamodel is shown in Figure 4. The Tract constraint in Listing 1 specifies the correct behavior that a M2T transformation that transforms each UML package to a Java package, each UML class to a Java class and each UML property to a Java attribute must fulfil. Listing 1. Tract constraint for the UML2Java example. UMLPackage.allInstances->forAll(upack | Folder.allInstances->exists(folder | upack.name = folder.name and upack.classes->forAll( uclass | folder.content->selectByType(File)->exists( file | uclass.name = file.name and uclass.properties.allInstances->forAll(uprop | file.lines->exists( line | line.machtesRE( ".*"+uprop.type+".*"+uprop.name+".*;"))))))) In order to provide tool support for our proposal, we have developed a injector (parser) that converts the content of a text repository into a model that conforms to the text metamodel shown in Figure 1, and an extractor that takes models conformin