Large-Scale and Adaptive Service Composition Using Deep Reinforcement Learning

Service composition provides an effective way to implement a Service-Oriented Architecture (SOA) by combining existing multiple services to meet user requirements. The increasingly complex user requirements and large amount of services pose a significant challenge to service selection and composition. Furthermore, web services are network based, which are inherently dynamic. The environment of service composition may also be complex and unstable. These demand a service composition solution to adapt to the change of environment. In this paper, we propose a new service composition solution based on Deep Reinforcement Learning (DRL) for adaptive and large-scale service composition problems. The experimental results demonstrate the effectiveness, scalability and self-adaptivity of our approach.

[1]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Maria Luisa Villani,et al.  A framework for QoS-aware binding and re-binding of composite web services , 2008, J. Syst. Softw..

[4]  Wenbin Li,et al.  Service farming: an ad-hoc and QoS-aware web service composition approach , 2013, SAC '13.

[5]  Xiang Zhou,et al.  Adaptive Service Composition Based on Reinforcement Learning , 2010, ICSOC.

[6]  Zibin Zheng,et al.  Integrating On-policy Reinforcement Learning with Multi-agent Techniques for Adaptive Service Composition , 2014, ICSOC.

[7]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[8]  Soundar R. T. Kumara,et al.  Effective Web Service Composition in Diverse and Large-Scale Service Networks , 2008, IEEE Transactions on Services Computing.

[9]  Pascal Poizat,et al.  Repairing Service Compositions in a Changing World , 2010, SERA.

[10]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[11]  Boi Faltings,et al.  Optimizing the Tradeoff between Discovery, Composition, and Execution Cost in Service Composition , 2011, 2011 IEEE International Conference on Web Services.

[12]  Boi Faltings,et al.  Large scale, type-compatible service composition , 2004 .

[13]  Martin A. Riedmiller,et al.  Deep auto-encoder neural networks in reinforcement learning , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).