User QoE-based adaptive routing system for future Internet CDN

The most important tendency of future Internet architectures is maintaining the best Quality of Experience (QoE), which represents the subjective perception of end-users using network services with network functions such as admission control, resource management, routing, traffic control, etc. Among of them, we focus on routing mechanism driven by QoE end-users. Nowadays, most existing routing protocols have encountered NP-complete problem when trying to satisfy multi QoS constraints criteria simultaneously. With the intention for avoiding the classification problem of these multiple criteria reducing the complexity problem for the future Internet, we propose a protocol based on user QoE measurement in routing paradigm to construct an adaptive and evolutionary system. Our approach, namely QQAR (QoE Q-learning based Adaptive Routing), is based on Q-Learning, a Reinforcement Learning algorithm. QQAR uses Pseudo Subjective Quality Assessment (PSQA), a real-time QoE assessment tool based on Random Neural Network, to evaluate QoE. Experimental results showed a significant performance against over other traditional routing protocols.

[1]  Sherali Zeadally,et al.  Design and performance analysis of an inductive QoS routing algorithm , 2009, Comput. Commun..

[2]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[3]  Maarten Wijnants,et al.  End-to-end QoE Optimization Through Overlay Network Deployment , 2008, 2008 International Conference on Information Networking.

[4]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[5]  Gerardo Rubino,et al.  Quantifying the Quality of Audio and Video Transmissions over the Internet: The PSQA Approach , 2006 .

[6]  S. Hemminger Network Emulation with NetEm , 2022 .

[7]  Said Hoceini,et al.  Average-Bandwidth Delay Q-Routing Adaptive Algorithm , 2008, 2008 IEEE International Conference on Communications.

[8]  Leonid Peshkin,et al.  Reinforcement learning for adaptive routing , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[9]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[10]  Adlen Ksentini,et al.  An Adaptive Mechanism for Multipath Video Streaming over Video Distribution Network (VDN) , 2009, 2009 First International Conference on Advances in Multimedia.

[11]  Jon Crowcroft,et al.  Quality-of-Service Routing for Supporting Multimedia Applications , 1996, IEEE J. Sel. Areas Commun..

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.