Metamorphic Testing in Fault Localization of Model Transformations

Model transformations are cornerstone elements of Model Driven Engineering (MDE), and their quality directly affects the successful application of MDE in practice. However, due to the characteristics of model transformation programs, the debugging of model transformations faces the oracle problem. In this paper, we propose an approach of debugging model transformations by using the technique of metamorphic testing (MT). Our approach leverages MT to alleviate the oracle problem, and integrates MT with spectrum-based fault localization technique to locating faulty rules of model transformations. We conduct experiments to evaluate our approach by using open-source model transformation programs, and compare the effectiveness of our approach with that of a fault localization technique using constraint-based oracles. Both of the experimental analysis and the comparison study show that our approach is of promising effectiveness, suggesting that MT can be a good support for debugging model transformations.

[1]  Peter Zoeteweij,et al.  A practical evaluation of spectrum-based fault localization , 2009, J. Syst. Softw..

[2]  Jürgen Dingel,et al.  Model transformation intents and their properties , 2016, Software & Systems Modeling.

[3]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[4]  Sergio Segura,et al.  Automated inference of likely metamorphic relations for model transformations , 2018, J. Syst. Softw..

[5]  Manuel Wimmer,et al.  Towards systematic mutations for and with ATL model transformations , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[6]  Ying Zhang,et al.  Testing bidirectional model transformation using metamorphic testing , 2018, Inf. Softw. Technol..

[7]  David Lo,et al.  Extended comprehensive study of association measures for fault localization , 2014, J. Softw. Evol. Process..

[8]  Baowen Xu,et al.  Metamorphic slice: An application in spectrum-based fault localization , 2013, Inf. Softw. Technol..

[9]  Xia Li,et al.  Boosting spectrum-based fault localization using PageRank , 2017, ISSTA.

[10]  A. Vargha,et al.  A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong , 2000 .

[11]  Tsong Yueh Chen,et al.  Metamorphic Testing: A New Approach for Generating Next Test Cases , 2020, ArXiv.

[12]  Michael Lawley,et al.  Forensic debugging of model transformations , 2007, MODELS'07.

[13]  Spss,et al.  Discovering Statistics (4th Edition ed. , 2013 .

[14]  Jon Whittle,et al.  A survey of approaches for verifying model transformations , 2013, Software & Systems Modeling.

[15]  Yves Le Traon,et al.  Barriers to systematic model transformation testing , 2010, Commun. ACM.

[16]  Antonio Vallecillo,et al.  Static Fault Localization in Model Transformations , 2015, IEEE Transactions on Software Engineering.

[17]  David Lo,et al.  Information retrieval and spectrum based bug localization: better together , 2015, ESEC/SIGSOFT FSE.

[18]  Lars Grunske,et al.  A learning-to-rank based fault localization approach using likely invariants , 2016, ISSTA.

[19]  Zuohua Ding,et al.  Testing Central Processing Unit scheduling algorithms using Metamorphic Testing , 2013, 2013 IEEE 4th International Conference on Software Engineering and Service Science.

[20]  Huai Liu,et al.  Metamorphic Testing , 2018, ACM Comput. Surv..

[21]  Sergio Segura,et al.  Spectrum-Based Fault Localization in Model Transformations , 2018, ACM Trans. Softw. Eng. Methodol..

[22]  Claudia A. Marcos,et al.  Assessing the Refactoring of Brain Methods , 2018, ACM Trans. Softw. Eng. Methodol..

[23]  Baowen Xu,et al.  A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization , 2013, TSEM.