An Adaptive Solution for Web Service Composition

Dynamic Web service composition is challenging problem and has been intensively investigated in recent years. Nevertheless, most of existing approaches do not provide satisfactory composition results, especially when confronted with a large scale of services. In this paper, we present a novel algorithm called HRLPLA for composing Web services. The algorithm considers functional properties and QoS properties simultaneously. By using hierarchical reinforcement learning, it can deal with large scales of services and generate efficient service compositions. Moreover, the algorithm is suitable for composing Web services in dynamic environment, as reinforcement learning his highly adaptive. We conducted experimental study to verify the effectiveness and efficiency of our method in dynamic service composition.

[1]  David J. C. Mackay,et al.  Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[2]  Yanlong Zhai,et al.  SOA Middleware Support for Service Process Reconfiguration with End-to-End QoS Constraints , 2009, 2009 IEEE International Conference on Web Services.

[3]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[4]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[5]  Patrick Girard,et al.  Von Wright’s preference logic reconsidered , 2006 .

[6]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[7]  Ee-Peng Lim,et al.  Dynamic Web Service Selection for Reliable Web Service Composition , 2008, IEEE Transactions on Services Computing.

[8]  A.G. Barto,et al.  Reinforcement learning in the real world , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[9]  H. Katzgraber.  Introduction to Monte Carlo Methods , 2009, 0905.1629.

[10]  Hongbing Wang,et al.  RLPLA: A Reinforcement Learning Algorithm of Web Service Composition with Preference Consideration , 2008, 2008 IEEE Congress on Services Part II (services-2 2008).

[11]  Wil M. P. van der Aalst,et al.  Workflow Patterns , 2004, Distributed and Parallel Databases.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[14]  Stephan Reiff-Marganiec,et al.  Markov-HTN Planning Approach to Enhance Flexibility of Automatic Web Service Composition , 2009, 2009 IEEE International Conference on Web Services.

[15]  Anne H. H. Ngu,et al.  Dynamic composition and optimization of Web services , 2008, Distributed and Parallel Databases.

[16]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.