Enhancing the crowdsourced live streaming: a deep reinforcement learning approach

With the growing demand for crowdsourced live streaming (CLS), how to schedule the large-scale dynamic viewers effectively among different Content Delivery Network (CDN) providers has become one of the most significant challenges for CLS platforms. Although abundant algorithms have been proposed in recent years, they suffer from a critical limitation: due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we propose LTS (Learn to schedule), a deep reinforcement learning (DRL) based scheduling approach that can dynamically adapt to the variation of both viewer traffics and CDN performance. After the extensive evaluation the real data from a leading CLS platform in China, we demonstrate that LTS improves the average quality of experience (QoE) over state-of-the-art approach by 8.71%-15.63%.

[1]  Chris Dyer,et al.  Neural Arithmetic Logic Units , 2018, NeurIPS.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Cheng-Zhong Xu,et al.  URL: A unified reinforcement learning approach for autonomic cloud management , 2012, J. Parallel Distributed Comput..

[4]  Chi Harold Liu,et al.  Experience-driven Networking: A Deep Reinforcement Learning based Approach , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[5]  Cong Zhang,et al.  When Cloud Meets Uncertain Crowd: An Auction Approach for Crowdsourced Livecast Transcoding , 2017, ACM Multimedia.

[6]  Aurélien Garivier,et al.  On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems , 2008, 0805.3415.

[7]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[8]  Srinivasan Seshan,et al.  A network measurement architecture for adaptive applications , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[9]  Zhu Han,et al.  User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning Approach , 2018, IEEE Transactions on Wireless Communications.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Fang Hao,et al.  A tale of three CDNs: An active measurement study of Hulu and its CDNs , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[12]  Vyas Sekar,et al.  CFA: A Practical Prediction System for Video QoE Optimization , 2016, NSDI.

[13]  Shijie Sun,et al.  Pytheas: Enabling Data-Driven Quality of Experience Optimization Using Group-Based Exploration-Exploitation , 2017, NSDI.

[14]  Marco Mellia,et al.  Dissecting Video Server Selection Strategies in the YouTube CDN , 2011, 2011 31st International Conference on Distributed Computing Systems.

[15]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, SIGCOMM.

[16]  Xiaofeng Wang,et al.  Cloud-Assisted Live Streaming for Crowdsourced Multimedia Content , 2015, IEEE Transactions on Multimedia.

[17]  Fang Hao,et al.  Unreeling netflix: Understanding and improving multi-CDN movie delivery , 2012, 2012 Proceedings IEEE INFOCOM.

[18]  I. Stoica,et al.  A case for a coordinated internet video control plane , 2012, CCRV.

[19]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.