QFlow: A Learning Approach to High QoE Video Streaming at the Wireless Edge

The predominant use of wireless access networks is for media streaming applications, which are only gaining popularity as ever more devices become available for this purpose. However, current access networks treat all packets identically, and lack the agility to determine which clients are most in need of service at a given time. Software reconfigurability of networking devices has seen wide adoption, and this in turn implies that agile control policies can be now instantiated on access networks. The goal of this work is to design, develop and demonstrate QFlow, a learning approach to create a value chain from the application on one side, to algorithms operating over reconfigurable infrastructure on the other, so that applications are able to obtain necessary resources for optimal performance. Using YouTube video streaming as an example, we illustrate how QFlow is able to adaptively provide such resources and attain a high QoE for all clients at a wireless access point.

[1]  Panagiotis Georgopoulos,et al.  Towards network-wide QoE fairness using openflow-assisted adaptive video streaming , 2013, FhMN@SIGCOMM.

[2]  Xiaoqing Zhu,et al.  SDN Based QoE Optimization for HTTP-Based Adaptive Video Streaming , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[3]  Stefan Schmid,et al.  AeroFlux: A Near-Sighted Controller Architecture for Software-Defined Wireless Networks , 2014, ONS.

[4]  Vivek S. Borkar,et al.  A Theory of QoS for Wireless , 2009, IEEE INFOCOM 2009.

[5]  Panganamala Ramana Kumar,et al.  PULS: Processor-Supported Ultra-Low Latency Scheduling , 2018, MobiHoc.

[6]  A Mean Field Game Approach to Scheduling in Cellular Systems , 2020, IEEE Transactions on Control of Network Systems.

[7]  Henning Schulzrinne,et al.  Towards QoE-aware video streaming using SDN , 2014, 2014 IEEE Global Communications Conference.

[8]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[9]  Andrew Sutton,et al.  {\AE}therFlow: Principled Wireless Support in SDN , 2015 .

[10]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[11]  Alexander Sprintson,et al.  Enabling Dynamic Reconfigurability of SDRs Using SDN Principles , 2016, ADHOCNETS.

[12]  Weichao Li,et al.  IRate: Initial Video Bitrate Selection System for HTTP Streaming , 2016, IEEE Journal on Selected Areas in Communications.

[13]  Phuoc Tran-Gia,et al.  SDN-Based Application-Aware Networking on the Example of YouTube Video Streaming , 2013, 2013 Second European Workshop on Software Defined Networks.

[14]  Kaigui Bian,et al.  DRL360: 360-degree Video Streaming with Deep Reinforcement Learning , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[15]  Lifeng Sun,et al.  QARC: Video Quality Aware Rate Control for Real-Time Video Streaming based on Deep Reinforcement Learning , 2018, ACM Multimedia.

[16]  Nagabhushan Eswara,et al.  A Continuous QoE Evaluation Framework for Video Streaming Over HTTP , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Leandros Tassiulas,et al.  Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks , 1992 .

[19]  R. Srikant,et al.  Stable scheduling policies for fading wireless channels , 2005, IEEE/ACM Transactions on Networking.

[20]  Krishna R. Pattipati,et al.  A Control Theoretic Approach to ABR Video Streaming: A Fresh Look at PID-Based Rate Adaptation , 2020, IEEE Transactions on Mobile Computing.

[21]  Zhi Zhou,et al.  DeepVR: Deep Reinforcement Learning for Predictive Panoramic Video Streaming , 2019, IEEE Transactions on Cognitive Communications and Networking.

[22]  Anja Feldmann,et al.  OpenSDWN: programmatic control over home and enterprise WiFi , 2015, SOSR.

[23]  Alan C. Bovik,et al.  Delivery quality score model for Internet video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[24]  Alan C. Bovik,et al.  Learning a Continuous-Time Streaming Video QoE Model , 2018, IEEE Transactions on Image Processing.

[25]  Alexander Sprintson,et al.  CrossFlow: A cross-layer architecture for SDR using SDN principles , 2015, 2015 IEEE Conference on Network Function Virtualization and Software Defined Network (NFV-SDN).

[26]  Eytan Modiano,et al.  Wireless channel allocation using an auction algorithm , 2006, IEEE Journal on Selected Areas in Communications.

[27]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[28]  Bruno Sinopoli,et al.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP , 2015, Comput. Commun. Rev..

[29]  P. Whittle.  Restless bandits: activity allocation in a changing world , 1988, Journal of Applied Probability.

[30]  Panganamala Ramana Kumar,et al.  Optimizing quality of experience of dynamic video streaming over fading wireless networks , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[31]  Alexander Sprintson,et al.  ÆtherFlow: Principled Wireless Support in SDN , 2015, 2015 IEEE 23rd International Conference on Network Protocols (ICNP).

[32]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[33]  Dileep M. Kalathil,et al.  QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks , 2019, MobiHoc.

[34]  Srinivas Shakkottai,et al.  A mean field game approach to scheduling in cellular systems , 2013, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.