A systematic approach to evaluating domain-specific modeling language environments for multi-agent systems

Multi-agent systems (MASs) include multiple interacting agents within an environment to provide a solution for complex systems that cannot be easily solved with individual agents or monolithic systems. However, the development of MASs is not trivial due to the various agent properties such as autonomy, responsiveness, and proactiveness, and the need for realization of the many different agent interactions. To support the development of MASs various domain-specific modeling languages (DSMLs) have been introduced that provide a declarative approach for modeling and supporting the generation of agent-based systems. To be effective, the proposed DSMLs need to meet the various stakeholder concerns and the related quality criteria for the corresponding MASs. Unfortunately, very often the evaluation of the DSML is completely missing or has been carried out in idiosyncratic approach. If the DSMLs are not well defined, then implicitly this will have an impact on the quality of the MASs. In this paper, we present an evaluation framework and systematic approach for assessing existing or newly defined DSMLs for MASs. The evaluation is specific for MAS DSMLs and targets both the language and the corresponding tools. To illustrate the evaluation approach, we first present SEA_ML, which is a model-driven MAS DSML for supporting the modeling and generation of agent-based systems. The evaluation of SEA_ML is based on a multi-case study research approach and provides both qualitative evaluation and quantitative analysis. We report on the lessons learned considering the adoption of the evaluation approach as well as the SEA_ML for supporting the generation of agent-based systems.

[1]  ISO / IEC 25010 : 2011 Systems and software engineering — Systems and software Quality Requirements and Evaluation ( SQuaRE ) — System and software quality models , 2013 .

[2]  BellifemineFabio,et al.  Developing multi-agent systems with a FIPA-compliant agent framework , 2001 .

[3]  Jorge J. Gómez-Sanz,et al.  Model Driven Development of Multi-Agent Systems , 2006, ECMDA-FA.

[4]  Winfried Lamersdorf,et al.  Jadex: A BDI Reasoning Engine , 2005, Multi-Agent Programming.

[5]  Anna E. Bobkowska,et al.  A Methodology of Visual Modeling Language Evaluation , 2005, SOFSEM.

[6]  Elena Navarro,et al.  Model-driven engineering techniques for the development of multi-agent systems , 2012, Eng. Appl. Artif. Intell..

[7]  Anupriya Ankolekar,et al.  Automated discovery, interaction and composition of Semantic Web services , 2003, J. Web Semant..

[8]  Moharram Challenger,et al.  A Domain Specific Metamodel for Semantic Web Enabled Multi-Agent Systems , 2011, CAiSE Workshops.

[9]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[10]  Semih Bilgen,et al.  A framework for qualitative assessment of domain-specific languages , 2015, Software & Systems Modeling.

[11]  Gregor Kiczales,et al.  Design pattern implementation in Java and aspectJ , 2002, OOPSLA '02.

[12]  Geylani Kardas,et al.  A model driven architecture for the development of smart card software , 2014, Comput. Lang. Syst. Struct..

[13]  Nuno Oliveira,et al.  Comparing general-purpose and domain-specific languages: An empirical study , 2010, Comput. Sci. Inf. Syst..

[14]  João M. Fernandes,et al.  Towards a catalog of aspect-oriented refactorings , 2005, AOSD '05.

[15]  Maria João Varanda Pereira,et al.  Program comprehension for domain-specific languages , 2008, Comput. Sci. Inf. Syst..

[16]  Anneke Kleppe,et al.  The object constraint language: precise modeling with UML , 1998 .

[17]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[18]  Bedir Tekinerdogan,et al.  Evaluation framework for software architecture viewpoint languages , 2013, QoSA '13.

[19]  Klaus Schmid,et al.  Developing, validating and evolving an approach to product line benefit and risk assessment , 2002, Proceedings. 28th Euromicro Conference.

[20]  Roy Grønmo,et al.  Toward Standardised Model to Text Transformations , 2005, ECMDA-FA.

[21]  Michael Winikoff,et al.  Developing intelligent agent systems - a practical guide , 2004, Wiley series in agent technology.

[22]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[23]  Zoran Budimac,et al.  Usage of agents in document management , 2011, Comput. Sci. Inf. Syst..

[24]  Iván García-Magariño,et al.  A technique fordefining agent-oriented engineering processes with tool support , 2010, Eng. Appl. Artif. Intell..

[25]  Reid G. Smith,et al.  The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver , 1980, IEEE Transactions on Computers.

[26]  Viljem Zumer,et al.  Incremental programming language development , 2005, Comput. Lang. Syst. Struct..

[27]  Richard F. Paige,et al.  Principles for modeling language design , 2000, Inf. Softw. Technol..

[28]  Jeffrey C. Carver,et al.  A checklist for integrating student empirical studies with research and teaching goals , 2010, Empirical Software Engineering.

[29]  Miguel Goulão,et al.  Quality in use of domain-specific languages: a case study , 2011, PLATEAU '11.

[30]  Jeffrey C. Carver,et al.  Program comprehension of domain-specific and general-purpose languages: comparison using a family of experiments , 2011, Empirical Software Engineering.

[31]  Alain Abran,et al.  Software Metrics and Software Metrology , 2010 .

[32]  Parastoo Mohagheghi,et al.  Where Is the Proof? - A Review of Experiences from Applying MDE in Industry , 2008, ECMDA-FA.

[33]  Oguz Dikenelli,et al.  Model Driven Development of Semantic Web Enabled Multi-Agent Systems , 2009, Int. J. Cooperative Inf. Syst..

[34]  Barbara Kitchenham,et al.  DESMET: a methodology for evaluating software engineering methods and tools , 1997 .

[35]  Sussy Bayona Oré,et al.  Critical success factors taxonomy for software process deployment , 2012, Software Quality Journal.

[36]  S. Russel and P. Norvig,et al.  “Artificial Intelligence – A Modern Approach”, Second Edition, Pearson Education, 2003. , 2015 .

[37]  Bedir Tekinerdogan,et al.  Software Language Engineering of Architectural Viewpoints , 2011, ECSA.

[38]  Andy Evans,et al.  Language driven development and MDA. , 2004 .

[39]  Janez Brest,et al.  Design and implementation of domain-specific language easytime , 2011, Comput. Lang. Syst. Struct..

[40]  Gabriel Ciobanu,et al.  Flexible software architecture and language for mobile agents , 2012, Concurr. Comput. Pract. Exp..

[41]  Alain Abran,et al.  Software Metrics and Software Metrology: Abran/Software Metrics , 2010 .

[42]  Daniel Jackson,et al.  Alloy: a lightweight object modelling notation , 2002, TSEM.

[43]  Miguel Goulão,et al.  How to reach a usable DSL? Moving toward a Systematic Evaluation , 2012, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[44]  Gerhard Weiss,et al.  Multiagent Systems , 1999 .

[45]  Fausto Giunchiglia,et al.  Tropos: An Agent-Oriented Software Development Methodology , 2004, Autonomous Agents and Multi-Agent Systems.

[46]  Franco Zambonelli,et al.  A Study of Some Multi-agent Meta-models , 2004, AOSE.

[47]  Marian H. Nodine,et al.  A Metamodel for Agents, Roles, and Groups , 2004, AOSE.

[48]  Christian Hahn.  A domain specific modeling language for multiagent systems , 2008, AAMAS.

[49]  Moharram Challenger,et al.  Development of an agent based e-barter system , 2011, 2011 International Symposium on Innovations in Intelligent Systems and Applications.

[50]  Marie-Pierre Gleizes,et al.  Model Driven Engineering for Designing Adaptive Multi-Agents Systems , 2008, ESAW.

[51]  Moharram Challenger,et al.  The Semantics of the Interaction between Agents and Web Services on the Semantic Web , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference Workshops.

[52]  Juha-Pekka Tolvanen,et al.  Domain-Specific Modeling: Enabling Full Code Generation , 2008 .

[53]  Moharram Challenger,et al.  On the use of a domain-specific modeling language in the development of multiagent systems , 2014, Eng. Appl. Artif. Intell..

[54]  Agostino Poggi,et al.  Multiagent Systems , 2006, Intelligenza Artificiale.

[55]  Bernhard Rumpe,et al.  Domain specific modeling , 2005, Software & Systems Modeling.

[56]  Aruna Raja,et al.  Domain Specific Languages , 2010 .

[57]  Miguel A. Fernández,et al.  An empirical study of the state of the practice and acceptance of model-driven engineering in four industrial cases , 2012, Empirical Software Engineering.

[58]  Agostino Poggi,et al.  Developing multi‐agent systems with a FIPA‐compliant agent framework , 2001 .

[59]  David S. Wile,et al.  Lessons learned from real DSL experiments , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[60]  Klaus Fischer,et al.  A platform-independent metamodel for multiagent systems , 2008, Autonomous Agents and Multi-Agent Systems.

[61]  Aniruddha S. Gokhale,et al.  Domain-Specific Modeling , 2008, Handbook of Dynamic System Modeling.

[62]  Diomidis Spinellis,et al.  Guest Editors' Introduction: What Kinds of Nails Need a Domain-Specific Hammer? , 2009, IEEE Software.

[63]  Arie van Deursen,et al.  Domain-specific languages: an annotated bibliography , 2000, SIGP.

[64]  Moharram Challenger,et al.  A DSL for the development of software agents working within a semantic web environment , 2013, Comput. Sci. Inf. Syst..

[65]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[66]  Moharram Challenger,et al.  Design and implementation of a multiagent stock trading system , 2012, Softw. Pract. Exp..

[67]  Oscar Pastor,et al.  Towards an accurate functional size measurement procedure for conceptual models in an MDA environment , 2010, Data Knowl. Eng..

[68]  Franco Zambonelli,et al.  Developing multiagent systems: The Gaia methodology , 2003, TSEM.

[69]  M Mernik,et al.  When and how to develop domain-specific languages , 2005, CSUR.

[70]  Douglas C. Schmidt,et al.  Guest Editor's Introduction: Model-Driven Engineering , 2006, Computer.

[71]  Jerry R. Hobbs,et al.  DAML-S: Semantic Markup for Web Services , 2001, SWWS.

[72]  Matthias Klusch,et al.  Integration of Multiagent Systems and Semantic Web Services on a Platform Independent Level , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[73]  David C. Yen,et al.  A new reverse auction agent system for m-commerce using mobile agents , 2005, Comput. Stand. Interfaces.

[74]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[75]  Alain Abran,et al.  DSML success factors and their assessment criteria , 2008 .

[76]  Uday P. Khedker.  What Makes a Good Programming Language ? , 2008 .

[77]  Bedir Tekinerdogan,et al.  An Aspect-Oriented Tool Framework for Developing Process-Sensitive Embedded User Assistance Systems , 2011, LNCS Trans. Aspect Oriented Softw. Dev..

[78]  Jean Bézivin,et al.  ATL: A model transformation tool , 2008, Sci. Comput. Program..

[79]  TR,et al.  Information technology — Software process assessment — Part 2 : A reference model for processes and process capability , 1998 .

[80]  Miguel Goulão,et al.  Evaluating the Usability of Domain-Specific Languages , 2013 .

[81]  G Stix,et al.  The mice that warred. , 2001, Scientific American.

[82]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[83]  Jorge J. Gómez-Sanz,et al.  FAML: A Generic Metamodel for MAS Development , 2009, IEEE Transactions on Software Engineering.

[84]  Andrea Omicini,et al.  Artifacts in the A&A meta-model for multi-agent systems , 2008, Autonomous Agents and Multi-Agent Systems.

[85]  Ankica Barisic,et al.  Evaluating the Quality in the Use of Domain-Specific Languages in an Agile Way , 2013, MoDELS.

[86]  F. Migeon,et al.  Model Driven Engineering for Designing Adaptive Multi-Agent Systems , 2007 .

[87]  Miguel Goulão,et al.  Patterns for evaluating usability of domain-specific languages , 2012 .

[88]  Carlos José Pereira de Lucena,et al.  Modularizing design patterns with aspects: a quantitative study , 2005, AOSD '05.

[89]  Rafael H. Bordini,et al.  Multi-Agent Programming: Languages, Platforms and Applications , 2005, Multi-Agent Programming.

[90]  Anand S. Rao,et al.  BDI Agents: From Theory to Practice , 1995, ICMAS.

[91]  Wendy Hall,et al.  The Semantic Web Revisited , 2006, IEEE Intelligent Systems.

[92]  K. P. Sycara.  Multiagent systems : Special issue on agents , 1998 .

[93]  Moharram Challenger,et al.  The Formal Semantics of a Domain-Specific Modeling Language for Semantic Web Enabled Multi-Agent Systems , 2014, Int. J. Cooperative Inf. Syst..