Rldish: Edge-Assisted QoE Optimization of HTTP Live Streaming with Reinforcement Learning

Recent years have seen a rapidly increasing traffic demand for HTTP-based high-quality live video streaming. The surging traffic demand, as well as the real-time property of live videos, make it challenging for content delivery networks (CDNs) to guarantee the Quality-of-Experiences (QoE) of viewers. The initial video segment (IVS) of live streaming plays an important role in the QoE of live viewers, particularly when users require fast join time and smooth view experience. State-of-the-art research on this regard estimates network throughput for each viewer and thus may incur a large overhead that offsets the benefit. To tackle the problem, we propose Rldish, a scheme deployed at the edge CDN server, to dynamically select a suitable IVS for new live viewers based on Reinforcement Learning (RL). Rldish is transparent to both the client and the streaming server. It collects the real-time QoE observations from the edge without any client-side assistance, then uses these QoE observations as real-time rewards in RL. We deploy Rldish as a virtualized network function (VNF) in a real HTTP cache server, and evaluate its performance using streaming servers distributed over the world. Our experiments show that Rldish improves the state- of-the-art IVS selection scheme w.r.t. the average QoE of live viewers by up to 22%.

[1]  Filip De Turck,et al.  HTTP/2-Based Methods to Improve the Live Experience of Adaptive Streaming , 2015, ACM Multimedia.

[2]  Yi Sun,et al.  CS2P: Improving Video Bitrate Selection and Adaptation with Data-Driven Throughput Prediction , 2016, SIGCOMM.

[3]  Yong Man Ro,et al.  An Evaluation of Bitrate Adaptation Methods for HTTP Live Streaming , 2014, IEEE Journal on Selected Areas in Communications.

[4]  Narciso García,et al.  Perceptual Quality of HTTP Adaptive Streaming Strategies: Cross-Experimental Analysis of Multi-Laboratory and Crowdsourced Subjective Studies , 2016, IEEE Journal on Selected Areas in Communications.

[5]  Eric Moulines,et al.  On Upper-Confidence Bound Policies for Switching Bandit Problems , 2011, ALT.

[6]  Shijie Sun,et al.  Pytheas: Enabling Data-Driven Quality of Experience Optimization Using Group-Based Exploration-Exploitation , 2017, NSDI.

[7]  Srinivasan Seshan,et al.  Practical, Real-time Centralized Control for CDN-based Live Video Delivery , 2015, SIGCOMM.

[8]  Hermann Hellwagner,et al.  QoE-Assured 4K HTTP Live Streaming via Transient Segment Holding at Mobile Edge , 2018, IEEE Journal on Selected Areas in Communications.

[9]  Dan Pei,et al.  Reducing Web Latency Through Dynamically Setting TCP Initial Window with Reinforcement Learning , 2018, 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS).

[10]  Bruno Sinopoli,et al.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP , 2015, Comput. Commun. Rev..

[11]  Mick Wilson,et al.  Toward QoE-Assured 4K Video-on-Demand Delivery Through Mobile Edge Virtualization With Adaptive Prefetching , 2017, IEEE Transactions on Multimedia.

[12]  Ramesh K. Sitaraman,et al.  Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs , 2013, IEEE/ACM Transactions on Networking.

[13]  Nicola Blefari-Melazzi,et al.  Tracker-assisted rate adaptation for MPEG DASH live streaming , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[14]  Albert Y. Zomaya,et al.  Secure and Sustainable Load Balancing of Edge Data Centers in Fog Computing , 2018, IEEE Communications Magazine.

[15]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[16]  Adam Wolisz,et al.  QoE-Based Low-Delay Live Streaming Using Throughput Predictions , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[17]  Phuoc Tran-Gia,et al.  A Survey on Quality of Experience of HTTP Adaptive Streaming , 2015, IEEE Communications Surveys & Tutorials.

[18]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, CACM.

[19]  Haitian Pang,et al.  Intelligent Edge-Assisted Crowdcast with Deep Reinforcement Learning for Personalized QoE , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.