CFA: A Practical Prediction System for Video QoE Optimization

Many prior efforts have suggested that Internet video Quality of Experience (QoE) could be dramatically improved by using data-driven prediction of video quality for different choices (e.g., CDN or bitrate) to make optimal decisions. However, building such a prediction system is challenging on two fronts. First, the relationships between video quality and observed session features can be quite complex. Second, video quality changes dynamically. Thus, we need a prediction model that is (a) expressive enough to capture these complex relationships and (b) capable of updating quality predictions in near real-time. Unfortunately, several seemingly natural solutions (e.g., simple machine learning approaches and simple network models) fail on one or more fronts. Thus, the potential benefits promised by these prior efforts remain unrealized. We address these challenges and present the design and implementation of Critical Feature Analytics (CFA). The design of CFA is driven by domain-specific insights that video quality is typically determined by a small subset of critical features whose criticality persists over several tens of minutes. This enables a scalable and accurate workflow where we automatically learn critical features for different sessions on coarse-grained timescales, while updating quality predictions in near real-time. Using a combination of a real-world pilot deployment and trace-driven analysis, we demonstrate that CFA leads to significant improvements in video quality; e.g., 32% less buffering time and 12% higher bitrate than a random decision maker.

[1]  R. Weber On the Gittins Index for Multiarmed Bandits , 1992 .

[2]  D. W. Scott,et al.  Variable Kernel Density Estimation , 1992 .

[3]  Srinivasan Seshan,et al.  SPAND: Shared Passive Network Performance Discovery , 1997, USENIX Symposium on Internet Technologies and Systems.

[4]  Stefan Savage,et al.  The case for informed transport protocols , 1999, Proceedings of the Seventh Workshop on Hot Topics in Operating Systems.

[5]  Srinivasan Seshan,et al.  A network measurement architecture for adaptive applications , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[6]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[7]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[8]  Nick Feamster,et al.  Design and implementation of a routing control platform , 2005, NSDI.

[9]  Qi He,et al.  On the predictability of large transfer TCP throughput , 2005, SIGCOMM '05.

[10]  Aleksandar Kuzmanovic,et al.  Drafting behind Akamai (travelocity-based detouring) , 2006, SIGCOMM '06.

[11]  Hong Yan,et al.  Tesseract: A 4D Network Control Plane , 2007, NSDI.

[12]  Warren B. Powell,et al.  Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics) , 2007 .

[13]  Ning Xia,et al.  Inside the bird's nest: measurements of large-scale live VoD from the 2008 olympics , 2009, IMC '09.

[14]  Zihui Ge,et al.  Crowdsourcing service-level network event monitoring , 2010, SIGCOMM '10.

[15]  Paul Barford,et al.  A Machine Learning Approach to TCP Throughput Prediction , 2007, IEEE/ACM Transactions on Networking.

[16]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[17]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, SIGCOMM.

[18]  Larry Peterson,et al.  Framework for CDN Interconnection , 2011 .

[19]  Marco Mellia,et al.  Dissecting Video Server Selection Strategies in the YouTube CDN , 2011, 2011 31st International Conference on Distributed Computing Systems.

[20]  Gregory R. Ganger,et al.  Diagnosing Performance Changes by Comparing Request Flows , 2011, NSDI.

[21]  Aleksandrs Slivkins,et al.  Contextual Bandits with Similarity Information , 2009, COLT.

[22]  Iraj Sodagar,et al.  The MPEG-DASH Standard for Multimedia Streaming Over the Internet , 2011, IEEE MultiMedia.

[23]  Yin Zhang,et al.  Q-score: proactive service quality assessment in a large IPTV system , 2011, IMC '11.

[24]  Wei Chu,et al.  Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms , 2010, WSDM '11.

[25]  Jay Kreps,et al.  Kafka : a Distributed Messaging System for Log Processing , 2011 .

[26]  Minlan Yu,et al.  Tradeoffs in CDN designs for throughput oriented traffic , 2012, CoNEXT '12.

[27]  I. Stoica,et al.  A case for a coordinated internet video control plane , 2012, CCRV.

[28]  Rob Sherwood,et al.  On Controller Performance in Software-Defined Networks , 2012, Hot-ICE.

[29]  Chen Tian,et al.  Optimizing cost and performance for content multihoming , 2012, SIGCOMM '12.

[30]  Ernst W. Biersack,et al.  A longitudinal view of HTTP video streaming performance , 2012, MMSys '12.

[31]  Fang Hao,et al.  A tale of three CDNs: An active measurement study of Hulu and its CDNs , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[32]  Paulo J. G. Lisboa,et al.  Making machine learning models interpretable , 2012, ESANN.

[33]  Srinivasan Seshan,et al.  Analyzing the potential benefits of CDN augmentation strategies for internet video workloads , 2013, Internet Measurement Conference.

[34]  Scott Shenker,et al.  CAP for networks , 2013, HotSDN '13.

[35]  I. Stoica,et al.  Developing a predictive model of quality of experience for internet video , 2013, Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication.

[36]  Ramesh K. Sitaraman,et al.  Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs , 2012, IEEE/ACM Transactions on Networking.

[37]  Vyas Sekar,et al.  Shedding light on the structure of internet video quality problems in the wild , 2013, CoNEXT.

[38]  Anja Feldmann,et al.  Pushing CDN-ISP collaboration to the limit , 2013, CCRV.

[39]  Fang Hao,et al.  Towards an elastic distributed SDN controller , 2013, HotSDN '13.

[40]  Shobha Venkataraman,et al.  Prometheus: toward quality-of-experience estimation for mobile apps from passive network measurements , 2014, HotMobile.

[41]  Vyas Sekar,et al.  Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with FESTIVE , 2012, CoNEXT '12.

[42]  G. Varghese,et al.  Adtributor: Revenue Debugging in Advertising Systems , 2014, NSDI.

[43]  Xi Liu,et al.  EONA: Experience-Oriented Network Architecture , 2014, HotNets.

[44]  Xi Liu,et al.  C3: Internet-Scale Control Plane for Video Quality Optimization , 2015, NSDI.

[45]  Anja Feldmann,et al.  C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection , 2015, NSDI.

[46]  Te-Yuan Huang,et al.  A buffer-based approach to rate adaptation: evidence from a large video streaming service , 2015, SIGCOMM 2015.

[47]  Michael I. Jordan,et al.  The Missing Piece in Complex Analytics: Low Latency, Scalable Model Management and Serving with Velox , 2014, CIDR.

[48]  Srinivasan Seshan,et al.  Practical, Real-time Centralized Control for CDN-based Live Video Delivery , 2015, SIGCOMM.

[49]  Bruno Sinopoli,et al.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP , 2015, Comput. Commun. Rev..

[50]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..