A Two-Layer Approach to Developing Self-Adaptive Multi-Agent Systems in Open Environment

Development of self-adaptive systems situated in open and uncertain environments is a great challenge in the community of software engineering due to the unpredictability of environment changes and the variety of self-adaptation manners. Explicit specification of expected changes and various self-adaptations at design-time, an approach often adopted by developers, seems ineffective. This paper presents an agent-based approach that combines two-layer self-adaptation mechanisms and reinforcement learning together to support the development and running of self-adaptive systems. The approach takes self-adaptive systems as multi-agent organizations and enables the agent itself to make decisions on self-adaptation by learning at run-time and at different levels. The proposed self-adaptation mechanisms that are based on organization metaphors enable self-adaptation at two layers: fine-grain behavior level and coarse-grain organization level. Corresponding reinforcement learning algorithms on self-adaptation are designed and integrated with the two-layer self-adaptation mechanisms. This paper further details developmental technologies, based on the above approach, in establishing self-adaptive systems, including extended software architecture for self-adaptation, an implementation framework, and a development process. A case study and experiment evaluations are conducted to illustrate the effectiveness of the proposed approach.

[1]  Ji Wang,et al.  An adaptive casteship mechanism for developing multi-agent systems , 2008, Int. J. Comput. Appl. Technol..

[2]  Ladan Tahvildari,et al.  Self-adaptive software: Landscape and research challenges , 2009, TAAS.

[3]  Antonio Brogi,et al.  A formalized, taxonomy-driven approach to cross-layer application adaptation , 2012, TAAS.

[4]  Eunseok Lee,et al.  An Architecture for Multi-agent Based Self-adaptive System in Mobile Environment , 2005, IDEAL.

[5]  Jørgen Villadsen,et al.  A comparison of organization-centered and agent-centered multi-agent systems , 2013, Artif. Intell. Res..

[6]  Frank Dignum,et al.  Enacting and Deacting Roles in Agent Programming , 2004, AOSE.

[7]  Richard Schilling Agent Feedback Messaging: A Messaging Infrastructure for Distributed Message Delivery , 2009 .

[8]  Wang Huaimin,et al.  Constructing Software with Self-Adaptability , 2013 .

[9]  Mingyuan Zhang Theoretical and Practical Frameworks for Agent-Based Systems , 2012 .

[10]  Bireshwar Dass Mazumdar,et al.  Multi-Agent Negotiation Paradigm for Agent Selection in B2C E-Commerce , 2011, Int. J. Agent Technol. Syst..

[11]  Mary Shaw,et al.  Software Engineering for Self-Adaptive Systems: A Research Roadmap , 2009, Software Engineering for Self-Adaptive Systems.

[12]  H. Van Dyke Parunak,et al.  Temporal Aspects of Dynamic Role Assignment , 2003, AOSE.

[13]  Bradley R. Schmerl,et al.  Rainbow: Architecture-Based Self-Adaptation with Reusable Infrastructure , 2004, Computer.

[14]  Abder Koukam,et al.  A Mechanism for Dynamic Role Playing , 2002, Agent Technologies, Infrastructures, Tools, and Applications for E-Services.

[15]  Ji Wang,et al.  Towards an agent oriented programming language with caste and scenario mechanisms , 2005, AAMAS '05.

[16]  Richard Anthony,et al.  Autonomic computing in the first Decade: trends and direction , 2012 .

[17]  Goran Trajkovski,et al.  Handbook of Research on Agent-Based Societies: Social and Cultural Interactions , 2009 .

[18]  Douglas C. Schmidt,et al.  Ultra-Large-Scale Systems: The Software Challenge of the Future , 2006 .

[19]  Radu Calinescu,et al.  Large-scale complex IT systems , 2011, Commun. ACM.

[20]  Franco Zambonelli,et al.  Adaptive organizational changes in agent-oriented methodologies , 2011, Knowl. Eng. Rev..

[21]  Jun Han,et al.  Roles, players and adaptable organizations , 2007, Appl. Ontology.

[22]  Ladan Tahvildari,et al.  Adaptive Action Selection in Autonomic Software Using Reinforcement Learning , 2008, Fourth International Conference on Autonomic and Autonomous Systems (ICAS'08).

[23]  Zhichang Qi,et al.  SADE: A Development Environment for Adaptive Multi-Agent Systems , 2009, PRIMA.

[24]  Leon Sterling,et al.  A meta-model for intelligent adaptive multi-agent systems in open environments , 2003, AAMAS '03.

[25]  Franco Zambonelli,et al.  Dealing with Adaptive Multi-agent Organizations in the Gaia Methodology , 2005, AOSE.

[26]  Mohd Sharifuddin Ahmad,et al.  A Collaborative Framework for Multiagent Systems , 2010, Int. J. Agent Technol. Syst..

[27]  Ralph E. Johnson,et al.  Architecture and design of adaptive object-models , 2001, SIGP.

[28]  Qianxiang Wang,et al.  Towards a rule model for self-adaptive software , 2005, SOEN.

[29]  Yu Zhang,et al.  Simulating Cooperative Behaviors in Dynamic Networks , 2010, Int. J. Agent Technol. Syst..

[30]  Hong Zhu,et al.  CAMLE: A Caste-Centric Agent-Oriented Modelling Language and Environment , 2004, SELMAS.

[31]  MengChu Zhou,et al.  Role-based collaboration and its kernel mechanisms , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[32]  Franco Zambonelli,et al.  Organisational Rules as an Abstraction for the Analysis and Design of Multi-Agent Systems , 2001, Int. J. Softw. Eng. Knowl. Eng..

[33]  Seyed Masoud Sadjadi,et al.  Composing adaptive software , 2004, Computer.

[34]  Ali G. Hessami,et al.  Multi-Agent Systems: A new paradigm for Systems of Systems , 2013, ICONS 2013.

[35]  David Garlan,et al.  Rainbow: architecture-based self-adaptation with reusable infrastructure , 2004 .

[36]  Deborah Richards,et al.  Multi-Agent Systems for Education and Interactive Entertainment: Design, Use and Experience , 2010 .

[37]  Gita Sukthankar,et al.  Communications for Agent-Based Human Team Support , 2009, Handbook of Research on Multi-Agent Systems.

[38]  Hong Zhu,et al.  Caste: A Step beyond Object Orientation , 2003, JMLC.

[39]  Danny Weyns,et al.  SA-MAS: self-adaptation to enhance software qualities in multi-agent systems , 2013, AAMAS.

[40]  Sooyong Park,et al.  Reinforcement learning-based dynamic adaptation planning method for architecture-based self-managed software , 2009, 2009 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems.

[41]  Vijayan Sugumaran Distributed Artificial Intelligence, Agent Technology, and Collaborative Applications , 2008 .

[42]  Virginia. Virginia Dignum . Dignum,et al.  Handbook of Research on Multi-Agent Systems - Semantics and Dynamics of Organizational Models , 2009, Handbook of Research on Multi-Agent Systems.