Towards Multiple-Layer Self-Adaptations of Multi-Agent Organizations Using Reinforcement Learning

This chapter proposes a multi-agent organization model for self-adaptive software to examine the autonomous components and their self-adaptation that can be occurred at either the fine-grain behavior layer of a software agent or the coarsegrain organization layer of the roles that the agent plays. The authors design twolayer self-adaptation mechanisms and combine them with reinforcement learning together to tackle the uncertainty issues of self-adaptation, which enables software agents to make decisions on self-adaptation by learning at run-time to deal with various unanticipated changes. The reinforcement learning algorithms supporting fine-grain and coarse-grain adaptation mechanisms are designed. In order to support the development of self-adaptive software, the software architecture for individual agents, the development process and the software framework are proposed. A sample is developed in detail to illustrate our method and experiments are conducted to evaluate the effectiveness and efficiency of the proposed approach. Towards Multiple-Layer Self-Adaptations of MultiAgent Organizations Using Reinforcement Learning