Self-adaptation of multi-agent systems in dynamic environments based on experience exchanges

A self-adaptation mechanism based on experience exchanges is proposed for MASs.A method for agents to induce adaptation policies from experiences is put forward.Action matrix is used for agents' decision-making. In complex and changing environments, MASs (Multi-Agent Systems) have to be faced with many challenges and their adaptivity can hardly emerge from the localized, phased and timely behaviors and decisions of agents. In this paper, a self-adaptation mechanism based on experience exchanges is proposed for MASs. In the mechanism, agents can dynamically induce environmental constraints and behavior patterns for generating new adaptation policies from two types of experiences exchanged by others, i.e., immediate and retrospective experiences, and then build action matrix based on the adaptation policies for determining their behaviors that can positively contribute to the improvement of global adaptivity of MASs. In the end, an example MAS system simulating a delivery system is described and experiments are conducted on the system to validate the mechanism for self-adaptive MASs.

[1]  Baozhen Yao,et al.  Production , Manufacturing and Logistics An improved ant colony optimization for vehicle routing problem , 2008 .

[2]  Jeff Magee,et al.  From goals to components: a combined approach to self-management , 2008, SEAMS '08.

[3]  Tom De Wolf,et al.  Emergence Versus Self-Organisation: Different Concepts but Promising When Combined , 2004, Engineering Self-Organising Systems.

[4]  V. I. Gorodetskii.  Self-organization and multiagent systems: I. Models of multiagent self-organization , 2012, Journal of Computer and Systems Sciences International.

[5]  Mickaël Gardoni,et al.  A SURVEY OF CONTEXT MODELING: APPROACHES, THEORIES AND USE FOR ENGINEERING DESIGN RESEARCHES , 2003 .

[6]  Wenpin Jiao,et al.  Supporting adaptation of decentralized software based on application scenarios , 2013, J. Syst. Softw..

[7]  Jean-Pierre Mano,et al.  Bio-inspired Mechanisms for Artificial Self-organised Systems , 2006, Informatica.

[8]  Jesper Andersson,et al.  On the challenges of self-adaptation in systems of systems , 2013, SESoS.

[9]  Jirí Vokrínek,et al.  Agents Toward Vehicle Routing Problem With Time Windows , 2015, J. Intell. Transp. Syst..

[10]  Jiming Liu,et al.  Autonomy-oriented computing (AOC): formulating computational systems with autonomous components , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[11]  Saeed Jalili,et al.  Towards modeling and runtime verification of self-organizing systems , 2016, Expert Syst. Appl..

[12]  Andres J. Ramirez,et al.  A taxonomy of uncertainty for dynamically adaptive systems , 2012, 2012 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[13]  Sam Malek,et al.  FUSION: a framework for engineering self-tuning self-adaptive software systems , 2010, FSE '10.

[14]  Guy Theraulaz,et al.  A Brief History of Stigmergy , 1999, Artificial Life.

[15]  Huaimin Wang,et al.  An Integrated Approach to Developing Self-Adaptive Software , 2014, J. Inf. Sci. Eng..

[16]  Franco Zambonelli,et al.  Towards a paradigm change in computer science and software engineering: a synthesis , 2003, The Knowledge Engineering Review.

[17]  Danny Weyns,et al.  Research on Environments in Multiagent Systems: Reflection on the State-of-the-Art , 2006 .

[18]  Wolfgang Renz,et al.  Structural Adaptations for Self-Organizing Multi-Agent Systems , 2015 .

[19]  George Angelos Papadopoulos,et al.  Please Scroll down for Article Enterprise Information Systems a Survey of Software Adaptation in Mobile and Ubiquitous Computing a Survey of Software Adaptation in Mobile and Ubiquitous Computing , 2022 .

[20]  Bradley R. Schmerl,et al.  Software Engineering for Self-Adaptive Systems: A Second Research Roadmap , 2010, Software Engineering for Self-Adaptive Systems.

[21]  Dragan Simic,et al.  A Survey of Hybrid Artificial Intelligence Algorithms for Dynamic Vehicle Routing Problem , 2015, HAIS.

[22]  Richard N. Taylor,et al.  Towards a knowledge-based approach to architectural adaptation management , 2004, WOSS '04.

[23]  Carlos Gershenson,et al.  Design and Control of Self-organizing Systems , 2007 .

[24]  Ladan Tahvildari,et al.  Self-adaptive software: Landscape and research challenges , 2009, TAAS.

[25]  Mohammed Abufouda,et al.  A Framework for Enhancing Performance and Handling Run-Time Uncertainty in Self-Adaptive Systems , 2014, ArXiv.

[26]  Bruce Edmonds,et al.  Making Self-Organising Adaptive Multiagent Systems Work , 2004 .

[27]  Yu Zheng,et al.  Effective and Efficient: Large-Scale Dynamic City Express , 2016, IEEE Transactions on Knowledge and Data Engineering.

[28]  Jeff Magee,et al.  Self-Managed Systems: an Architectural Challenge , 2007, Future of Software Engineering (FOSE '07).

[29]  Peyman Oreizy.  Issues in the Runtime Modification of Software Architectures , 1997 .

[30]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[31]  Katsumi Inoue,et al.  Learning revised models for planning in adaptive systems , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[32]  Gail E. Kaiser,et al.  A control theory foundation for self-managing computing systems , 2005, IEEE Journal on Selected Areas in Communications.

[33]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[34]  Marie-Pierre Gleizes,et al.  Self-Organisation and Emergence in MAS: An Overview , 2006, Informatica.

[35]  Mary Shaw,et al.  Software Engineering for Self-Adaptive Systems: A Research Roadmap , 2009, Software Engineering for Self-Adaptive Systems.

[36]  Sam Malek,et al.  Taming uncertainty in self-adaptive software , 2011, ESEC/FSE '11.

[37]  Salima Hassas,et al.  Self-Organising Mechanisms from Social and Business/Economics Approaches , 2006, Informatica.

[38]  K. Rajeswari,et al.  A hybrid Genetic Algorithm for Vehicle Routing Problem with Time Windows , 2015, 2015 International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS).

[39]  Naeem Esfahani,et al.  A framework for managing uncertainty in self-adaptive software systems , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[40]  Peter Stone,et al.  TEXPLORE: real-time sample-efficient reinforcement learning for robots , 2012, Machine Learning.

[41]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[42]  Yu Zheng,et al.  Real-Time City-Scale Taxi Ridesharing , 2015, IEEE Transactions on Knowledge and Data Engineering.

[43]  Jesper Andersson,et al.  Software Engineering Processes for Self-Adaptive Systems , 2013, Software Engineering for Self-Adaptive Systems.

[44]  Marie-Pierre Gleizes,et al.  Self-organising Software - From Natural to Artificial Adaptation , 2011, Natural Computing Series.

[45]  Bing Xie,et al.  Quality Driven Design of Program Frameworks for Intelligent Sensor Applications , 2013, 2013 20th Asia-Pacific Software Engineering Conference (APSEC).

[46]  Hui Cheng,et al.  Genetic Algorithms With Immigrants and Memory Schemes for Dynamic Shortest Path Routing Problems in Mobile Ad Hoc Networks , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[47]  Carlo Ghezzi,et al.  Managing non-functional uncertainty via model-driven adaptivity , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[48]  Jian Lu,et al.  Fuzzy Self-Adaptation of Mission-Critical Software Under Uncertainty , 2013, Journal of Computer Science and Technology.

[49]  H. Van Dyke Parunak,et al.  Software engineering for self-organizing systems , 2015, The Knowledge Engineering Review.

[50]  Zbigniew Michalewicz,et al.  Adaptive Business Intelligence: Three Case Studies , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[51]  Zbigniew Michalewicz,et al.  Adaptive Business Intelligence , 2009, Encyclopedia of Artificial Intelligence.

[52]  Jalel Euchi,et al.  The dynamic vehicle routing problem: Solution with hybrid metaheuristic approach , 2015, Swarm Evol. Comput..

[53]  Sooyong Park,et al.  Reinforcement learning-based dynamic adaptation planning method for architecture-based self-managed software , 2009, 2009 ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems.

[54]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[55]  Sam Malek,et al.  Uncertainty in Self-Adaptive Software Systems , 2010, Software Engineering for Self-Adaptive Systems.

[56]  A Koestler,et al.  Ghost in the Machine , 1970 .

[57]  Wenpin Jiao,et al.  Measurements for Adaptation Level and Efficiency of Adaptive Software Systems , 2013, 2013 18th International Conference on Engineering of Complex Computer Systems.