Adaptive and Dynamic Service Composition via Multi-agent Reinforcement Learning

In the era of big data, data intensive applications have posed new challenges to the filed of service composition, i.e. composition efficiency and scalability. How to compose massive and evolving services in such dynamic scenarios is a vital problem demanding prompt solutions. As a consequence, we propose a new model for large-scale adaptive service composition in this paper. This model integrates the knowledge of reinforcement learning aiming at the problem of adaptability in a highly-dynamic environment and game theory used to coordinate agents' behavior for a common task. In particular, a multi-agent Q-learning algorithm for service composition based on this model is also proposed. The experimental results demonstrate the effectiveness and efficiency of our approach, and show a better performance compared with the single-agent Q-learning method.

[1]  Haiyan Zhao,et al.  A Multi-agent Learning Model for Service Composition , 2012, 2012 IEEE Asia-Pacific Services Computing Conference.

[2]  Danilo Ardagna,et al.  Adaptive Service Composition in Flexible Processes , 2007, IEEE Transactions on Software Engineering.

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  H. Young,et al.  The Evolution of Conventions , 1993 .

[5]  L. Shapley,et al.  Fictitious Play Property for Games with Identical Interests , 1996 .

[6]  Pascal Poizat,et al.  Self-Adaptive Service Composition Through Graphplan Repair , 2010, 2010 IEEE International Conference on Web Services.

[7]  Marco Saerens,et al.  Dynamic Web Service Composition within a Service-Oriented Architecture , 2007, IEEE International Conference on Web Services (ICWS 2007).

[8]  Xiaofeng Wang,et al.  Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games , 2002, NIPS.

[9]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[10]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[11]  J. Nash,et al.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[12]  Xiang Zhou,et al.  Adaptive Service Composition Based on Reinforcement Learning , 2010, ICSOC.

[13]  Hongbing Wang,et al.  Preference-Aware Web Service Composition by Reinforcement Learning , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[14]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[15]  Shensheng Zhang,et al.  A Distributed Algorithm for Web Service Composition Based on Service Agent Model , 2011, IEEE Transactions on Parallel and Distributed Systems.

[16]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[17]  Hongbing Wang,et al.  Preference-Aware Web Service Composition Using Hierarchical Reinforcement Learning , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[18]  Hongbing Wang,et al.  A Novel Approach to Large-Scale Services Composition , 2013, APWeb.

[19]  Michael L. Littman,et al.  Value-function reinforcement learning in Markov games , 2001, Cognitive Systems Research.