User to User QoE Routing System

Recently, wealthy network services such as Internet protocol television (IPTV) and Voice over IP (VoIP) are expected to become more pervasive over the Next Generation Network (NGN). In order to serve this purpose, the quality of these services should be evaluated subjectively by users. This is referred to as the quality of experience (QoE). The most important tendency of actual network services is maintaining the best QoE with network functions such as admission control, resource management, routing, traffic control, etc. Among of them, we focus here on routing mechanism. We propose in this paper a protocol integrating QoE measurement in routing paradigm to construct an adaptive and evolutionary system. Our approach is based on Reinforcement Learning concept. More concretely, we have used a least squares reinforcement learning technique called Least Squares Policy Iteration. Experimental results showed a significant performance gain over traditional routing protocols.

[1]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[2]  Gianni A. Di Caro,et al.  AntNet: A Mobile Agents Approach to Adaptive Routing , 1999 .

[3]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[4]  Jon Crowcroft,et al.  Quality-of-Service Routing for Supporting Multimedia Applications , 1996, IEEE J. Sel. Areas Commun..

[5]  Leonid Peshkin,et al.  Reinforcement learning for adaptive routing , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Adlen Ksentini,et al.  An Adaptive Mechanism for Multipath Video Streaming over Video Distribution Network (VDN) , 2009, 2009 First International Conference on Advances in Multimedia.

[8]  Maarten Wijnants,et al.  End-to-end QoE Optimization Through Overlay Network Deployment , 2008, 2008 International Conference on Information Networking.

[9]  Ting Wang,et al.  Adaptive Routing for Sensor Networks using Reinforcement Learning , 2006, The Sixth IEEE International Conference on Computer and Information Technology (CIT'06).

[10]  Sherali Zeadally,et al.  Design and performance analysis of an inductive QoS routing algorithm , 2009, Comput. Commun..

[11]  Marco Dorigo,et al.  Mobile agents for adaptive routing , 1998, Proceedings of the Thirty-First Hawaii International Conference on System Sciences.

[12]  Marco Mellia,et al.  An Adaptive Routing Algorithm for Best-effort Traffic in Integrated-Services Networks , 1999 .

[13]  Michail G. Lagoudakis,et al.  Least-Squares Policy Iteration , 2003, J. Mach. Learn. Res..