A generic LSTM neural network architecture to infer heterogeneous model transformations

Models capture relevant properties of systems. During the models’ life-cycle, they are subjected to manipulations with different goals such as managing software evolution, performing analysis, increasing developers’ productivity, and reducing human errors. Typically, these manipulation operations are implemented as model transformations. Examples of these transformations are (i) model-to-model transformations for model evolution, model refactoring, model merging, model migration, model refinement, etc., (ii) model-to-text transformations for code generation and (iii) text-to-model ones for reverse engineering. These operations are usually manually implemented, using general-purpose languages such as Java, or domainspecific languages (DSLs) such as ATL or Acceleo. Even when using such DSLs, transformations are still timeconsuming and error-prone. We propose using the advances in artificial intelligence techniques to learn these manipulation operations on models and automate the process, freeing the developer from building specific pieces of code. In particular, our proposal is a generic neural network architecLoli Burgueño Open University of Catalonia, IN3, Spain Institut LIST, CEA, Université Paris-Saclay, France E-mail: lburguenoc@uoc.edu Jordi Cabot ICREA, Spain Open University of Catalonia, IN3, Spain E-mail: jordi.cabot@icrea.cat Shuai Li Institut LIST, CEA, Université Paris-Saclay, France E-mail: shuai.li@cea.fr Sébastien Gérard Institut LIST, CEA, Université Paris-Saclay, France E-mail: sebastien.gerard@cea.fr ture suitable for heterogeneous model transformations. Our architecture comprises an encoder-decoder LSTM (Long Short-Term Memory) with an attention mechanism. It is fed with pairs of input-output examples and, once trained, given an input, automatically produces the expected output. We present the architecture and illustrate the feasibility and potential of our approach through its application in two main operations on models: model-to-model transformations and code generation. The results confirm that neural networks are able to faithfully learn how to perform these tasks as long as enough data is provided and no contradictory examples are given.

[1]  Houari A. Sahraoui,et al.  Multi-Step Learning and Adaptive Search for Learning Complex Model Transformations from Examples , 2016, ACM Trans. Softw. Eng. Methodol..

[2]  Jordi Cabot,et al.  Applying graph kernels to model-driven engineering problems , 2018, MASES@ASE.

[3]  Antonio Vallecillo,et al.  Contents for a Model-Based Software Engineering Body of Knowledge , 2019, Software and Systems Modeling.

[4]  Michel R. V. Chaudron,et al.  Automatic Classification of UML Class Diagrams from Images , 2014, 2014 21st Asia-Pacific Software Engineering Conference.

[5]  Klaus-Dieter Schewe,et al.  Conceptual Modelling and Its Theoretical Foundations , 2012, Lecture Notes in Computer Science.

[6]  Vítor Santos Costa,et al.  Inductive Logic Programming , 2013, Lecture Notes in Computer Science.

[7]  Erhard Rahm,et al.  Generic schema matching, ten years later , 2011, Proc. VLDB Endow..

[8]  Philipp Koehn,et al.  Six Challenges for Neural Machine Translation , 2017, NMT@ACL.

[9]  Timothy Lethbridge,et al.  Problems and opportunities for model-centric versus code-centric software development: a survey of software professionals , 2008, MiSE '08.

[10]  Sébastien Gérard,et al.  An LSTM-Based Neural Network Architecture for Model Transformations , 2019, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS).

[11]  Oszkár Semeráth,et al.  Iterative Generation of Diverse Models for Testing Specifications of DSL Tools , 2018, FASE.

[12]  Jorge J. Gómez-Sanz,et al.  Model Transformation By-Example: An Algorithm for Generating Many-to-Many Transformation Rules in Several Model Transformation Languages , 2009, ICMT@TOOLS.

[13]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[14]  Michael Dorin and Sergio Montenegro.  Coding Standards and Human Nature , 2018 .

[15]  Fabian Gilson,et al.  From User Stories to Use Case Scenarios towards a Generative Approach , 2018, 2018 25th Australasian Software Engineering Conference (ASWEC).

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[18]  Michael Dorin.  Coding Standards and Human Nature , 2018 .

[19]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[20]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[21]  Raja Touahni,et al.  Automatic generation of UML sequence diagrams from user stories in Scrum process , 2015, 2015 10th International Conference on Intelligent Systems: Theories and Applications (SITA).

[22]  Oscar Pastor,et al.  Traceability Link Recovery between Requirements and Models using an Evolutionary Algorithm Guided by a Learning to Rank Algorithm: Train control and management case , 2020, J. Syst. Softw..

[23]  Leslie N. Smith,et al.  A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay , 2018, ArXiv.

[24]  Juri Di Rocco,et al.  Automated Classification of Metamodel Repositories: A Machine Learning Approach , 2019, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS).

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  Liang Chen,et al.  GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks , 2018, ArXiv.

[27]  Jean Bézivin,et al.  ATL: A model transformation tool , 2008, Sci. Comput. Program..

[28]  Birgit Vogel-Heuser,et al.  Family model mining for function block diagrams in automation software , 2014, SPLC '14.

[29]  Yanqing Wang,et al.  Complying with Coding Standards or Retaining Programming Style: A Quality Outlook at Source Code Level , 2008, J. Softw. Eng. Appl..

[30]  Sébastien Gérard,et al.  The Future of Model Transformation Languages: An Open Community Discussion , 2019, J. Object Technol..

[31]  Dániel Varró,et al.  Model transformation by example , 2006, MoDELS'06.

[32]  Stephen Merity,et al.  Single Headed Attention RNN: Stop Thinking With Your Head , 2019, ArXiv.

[33]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[34]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[35]  Emilio Soria Olivas,et al.  Handbook of Research on Machine Learning Applications and Trends : Algorithms , Methods , and Techniques , 2009 .

[36]  Philip A. Bernstein,et al.  Model management 2.0: manipulating richer mappings , 2007, SIGMOD '07.

[37]  Jaime Font,et al.  Feature Location in Model-Based Software Product Lines Through a Genetic Algorithm , 2016, ICSR.

[38]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[39]  Kaneeka Vidanage,et al.  UML generator - an automated system for model driven development , 2016, 2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer).

[40]  Dawn Xiaodong Song,et al.  Tree-to-tree Neural Networks for Program Translation , 2018, NeurIPS.

[41]  Manuel Wimmer,et al.  Towards Model Transformation Generation By-Example , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[42]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[43]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[44]  Gerti Kappel,et al.  Model Transformation By-Example: A Survey of the First Wave , 2012, Conceptual Modelling and Its Theoretical Foundations.

[45]  David Buttler,et al.  A Short Survey of Document Structure Similarity Algorithms , 2004, International Conference on Internet Computing.

[46]  Birger Møller-Pedersen,et al.  Model Comparison to Synthesize a Model-Driven Software Product Line , 2011, 2011 15th International Software Product Line Conference.

[47]  Raja Touahni,et al.  Automatic Transformation of User Stories into UML Use Case Diagrams using NLP Techniques , 2018, ANT/SEIT.

[48]  Shuai Li,et al.  Fostering Software Architect and Programmer Collaboration , 2016, 2016 21st International Conference on Engineering of Complex Computer Systems (ICECCS).

[49]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[50]  Gobinda G. Chowdhury,et al.  Natural language processing , 2005, Annu. Rev. Inf. Sci. Technol..

[51]  Philip A. Bernstein,et al.  Automated Co-evolution of Conceptual Models, Physical Databases, and Mappings , 2010, ER.

[52]  Dániel Varró,et al.  Model transformation by example using inductive logic programming , 2008, Software & Systems Modeling.

[53]  Mohamed Wiem Mkaouer,et al.  Model Transformation Using Multiobjective Optimization , 2014, Adv. Comput..

[54]  Bogdan Dit,et al.  Feature location in source code: a taxonomy and survey , 2013, J. Softw. Evol. Process..

[55]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[56]  Jordi Cabot,et al.  MoDisco: a generic and extensible framework for model driven reverse engineering , 2010, ASE.

[57]  Richard F. Paige,et al.  The Epsilon Transformation Language , 2008, ICMT@TOOLS.

[58]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[59]  Houari A. Sahraoui,et al.  Search-based model transformation by example , 2010, Software & Systems Modeling.

[60]  Antonio Vallecillo,et al.  A Linda-based platform for the parallel execution of out-place model transformations , 2016, Inf. Softw. Technol..

[61]  George R. Doddington,et al.  Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics , 2002 .

[62]  Charles A. Sutton,et al.  Learning natural coding conventions , 2014, SIGSOFT FSE.

[63]  Dawn Xiaodong Song,et al.  Learning Neural Programs To Parse Programs , 2017, ArXiv.

[64]  Bran Selic,et al.  What will it take? A view on adoption of model-based methods in practice , 2012, Software & Systems Modeling.

[65]  Sobhan Yassipour Tehrani,et al.  Enhancing model transformation synthesis using natural language processing , 2020, MoDELS.

[66]  Rogardt Heldal,et al.  Personalized and Automatic Model Repairing using Reinforcement Learning , 2019, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C).

[67]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .