SENSEI: Aligning Video Streaming Quality with Dynamic User Sensitivity

This paper aims to improve video streaming by leveraging a simple observation: users are more sensitive to low quality in certain parts of a video than in others. For instance, rebuffering during key moments of a sports video (e.g., before a goal is scored) is more annoying than rebuffering during normal gameplay. Such dynamic quality sensitivity, however, is rarely captured by current approaches, which predict QoE (quality-of-experience) using one-size-fits-all heuristics that are too simplistic to understand the nuances of video content. Instead of proposing yet another heuristic, we take a different approach: we run a separate crowdsourcing experiment for each video to derive users' quality sensitivity at different parts of the video. Of course, the cost of doing this at scale can be prohibitive, but we show that careful experiment design combined with a suite of pruning techniques can make the cost negligible compared to how much content providers invest in content generation and distribution. Our ability to accurately profile time-varying user sensitivity inspires a new approach: dynamically aligning higher (lower) quality with higher (lower) sensitivity periods. We present a new video streaming system called SENSEI that incorporates dynamic quality sensitivity into existing quality adaptation algorithms. We apply SENSEI to two state-of-the-art adaptation algorithms. SENSEI can take seemingly unusual actions: e.g., lowering bitrate (or initiating a rebuffering event) even when bandwidth is sufficient so that it can maintain a higher bitrate without rebuffering when quality sensitivity becomes higher in the near future. Compared to state-of-the-art approaches, SENSEI improves QoE by 15.1% or achieves the same QoE with 26.8% less bandwidth on average.

[1]  Judith Redi,et al.  Best Practices and Recommendations for Crowdsourced QoE - Lessons learned from the Qualinet Task Force Crowdsourcing , 2014 .

[2]  Chris Callison-Burch,et al.  A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk , 2017, CHI.

[3]  C.-C. Jay Kuo,et al.  Compressed image quality metric based on perceptually weighted distortion , 2015, IEEE Transactions on Image Processing.

[4]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Cagri Ozcinar,et al.  Visual Attention-Aware Omnidirectional Video Streaming Using Optimal Tiles for Virtual Reality , 2019, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[6]  Alan C. Bovik,et al.  A Subjective and Objective Study of Stalling Events in Mobile Streaming Videos , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Ramesh K. Sitaraman,et al.  BOLA: Near-Optimal Bitrate Adaptation for Online Videos , 2016, IEEE/ACM Transactions on Networking.

[8]  Zhengfang Duanmu,et al.  A Quality-of-Experience Database for Adaptive Video Streaming , 2018, IEEE Transactions on Broadcasting.

[9]  Kai Zeng,et al.  Display device-adapted video quality-of-experience assessment , 2015, Electronic Imaging.

[10]  Ben R. Newell,et al.  The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers , 2015, Judgment and Decision Making.

[11]  Xiaoyun Zhang,et al.  DVC: An End-To-End Deep Video Compression Framework , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Sujit Dey,et al.  Video-Aware Scheduling and Caching in the Radio Access Network , 2014, IEEE/ACM Transactions on Networking.

[14]  Alan C. Bovik,et al.  Continuous Prediction of Streaming Video QoE Using Dynamic Networks , 2017, IEEE Signal Processing Letters.

[15]  Phuoc Tran-Gia,et al.  Quantification of YouTube QoE via Crowdsourcing , 2011, 2011 IEEE International Symposium on Multimedia.

[16]  Alexander Raake,et al.  A modular HTTP adaptive streaming QoE model — Candidate for ITU-T P.1203 (“P.NATS”) , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[17]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jianguo Lu,et al.  Estimating deep web data source size by capture–recapture method , 2010, Information Retrieval.

[20]  Ramesh K. Sitaraman,et al.  From theory to practice: improving bitrate adaptation in the DASH reference player , 2018, MMSys.

[21]  Anh Nguyen,et al.  Your Attention is Unique: Detecting 360-Degree Video Saliency in Head-Mounted Display for Head Movement Prediction , 2018, ACM Multimedia.

[22]  Bruno Ribeiro,et al.  Oboe: auto-tuning video ABR algorithms to network conditions , 2018, SIGCOMM.

[23]  Peter Schelkens,et al.  Optimized segmentation of H.264/AVC video for HTTP adaptive streaming , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[24]  Vyas Sekar,et al.  Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with FESTIVE , 2012, CoNEXT '12.

[25]  Jianfei Cai,et al.  Optimizing Quality of Experience for Adaptive Bitrate Streaming via Viewer Interest Inference , 2018, IEEE Transactions on Multimedia.

[26]  Alan C. Bovik,et al.  Learning a Continuous-Time Streaming Video QoE Model , 2018, IEEE Transactions on Image Processing.

[27]  Marcia J. Simmering,et al.  Data Quality from Crowdsourced Surveys: A Mixed Method Inquiry into Perceptions of Amazon's Mechanical Turk Masters , 2018 .

[28]  Min Chen,et al.  Online Cloud Transcoding and Distribution for Crowdsourced Live Game Video Streaming , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[30]  Brice Augustin,et al.  Crowd-sourcing framework to assess QoE , 2014, 2014 IEEE International Conference on Communications (ICC).

[31]  Dongsu Han,et al.  Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning , 2020, SIGCOMM.

[32]  Yonggang Wen,et al.  Content-Aware Personalised Rate Adaptation for Adaptive Streaming via Deep Video Analysis , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[33]  Balu Adsumilli,et al.  YouTube UGC Dataset for Video Compression Research , 2019, 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP).

[34]  Carsten Griwodz,et al.  Commute path bandwidth traces from 3G networks: analysis and applications , 2013, MMSys.

[35]  Michael Seufert,et al.  The Impact of Adaptation Strategies on Perceived Quality of HTTP Adaptive Streaming , 2014, VideoNext '14.

[36]  Ramesh K. Sitaraman,et al.  Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs , 2012, IEEE/ACM Transactions on Networking.

[37]  Purnamrita Sarkar,et al.  Answering enumeration queries with the crowd , 2015, Commun. ACM.

[38]  Sujit Dey,et al.  Deriving and Validating User Experience Model for DASH Video Streaming , 2015, IEEE Transactions on Broadcasting.

[39]  Zhi Li,et al.  Towards Perceptually Optimized End-to-end Adaptive Video Streaming. , 2018, 1808.03898.

[40]  Gustavo de Veciana,et al.  Modeling the Time—Varying Subjective Quality of HTTP Video Streams With Rate Adaptations , 2013, IEEE Transactions on Image Processing.

[41]  Rajiv Soundararajan,et al.  Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Phuoc Tran-Gia,et al.  Best Practices for QoE Crowdtesting: QoE Assessment With Crowdsourcing , 2014, IEEE Transactions on Multimedia.

[43]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, SIGCOMM.

[44]  Chin-Laung Lei,et al.  Crowdsourcing Multimedia QoE Evaluation: A Trusted Framework , 2013, IEEE Transactions on Multimedia.

[45]  Zhengfang Duanmu,et al.  A Quality-of-Experience Index for Streaming Video , 2017, IEEE Journal of Selected Topics in Signal Processing.

[46]  Hua Huang,et al.  Image Quality Assessment Using Directional Anisotropy Structure Measurement , 2017, IEEE Transactions on Image Processing.

[47]  Alberto Blanc,et al.  Transcoding live adaptive video streams at a massive scale in the cloud , 2015, MMSys.

[48]  Zygmunt Pizlo,et al.  Camera Motion-Based Analysis of User Generated Video , 2010, IEEE Transactions on Multimedia.

[49]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[50]  Nagabhushan Eswara,et al.  Streaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[51]  Lilly Irani,et al.  Amazon Mechanical Turk , 2018, Advances in Intelligent Systems and Computing.

[52]  Wen Gao,et al.  A Knowledge-Driven Quality-of-Experience Model for Adaptive Streaming Videos , 2019, ArXiv.

[53]  Christian Timmerer,et al.  A web based subjective evaluation platform , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[54]  Touradj Ebrahimi,et al.  Attention Driven Foveated Video Quality Assessment , 2014, IEEE Transactions on Image Processing.

[55]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[56]  Ke Zhang,et al.  Video Summarization with Long Short-Term Memory , 2016, ECCV.

[57]  Bruno Sinopoli,et al.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP , 2015, Comput. Commun. Rev..

[58]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[59]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[60]  Konstantina Papagiannaki,et al.  EYEORG: A Platform For Crowdsourcing Web Quality Of Experience Measurements , 2016, CoNEXT.

[61]  Philip Levis,et al.  Learning in situ: a randomized experiment in video streaming , 2019, NSDI.

[62]  Yi Sun,et al.  CS2P: Improving Video Bitrate Selection and Adaptation with Data-Driven Throughput Prediction , 2016, SIGCOMM.

[63]  Edward J. Delp,et al.  Introduction to the Issue on Perception-Driven 360° Video Processing , 2020, IEEE J. Sel. Top. Signal Process..

[64]  Alan C. Bovik,et al.  An Augmented Autoregressive Approach to HTTP Video Stream Quality Prediction , 2017, ArXiv.

[65]  Itu-T and Iso Iec Jtc Advanced video coding for generic audiovisual services , 2010 .

[66]  Prateesh Goyal,et al.  End-to-end transport for video QoE fairness , 2019, SIGCOMM.

[67]  Danny De Vleeschauwer,et al.  Model for estimating QoE of video delivered using HTTP adaptive streaming , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[68]  Jinwoo Shin,et al.  Neural Adaptive Content-aware Internet Video Delivery , 2018, OSDI.

[69]  Yale Song,et al.  Video2GIF: Automatic Generation of Animated GIFs from Video , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Chin-Laung Lei,et al.  Quadrant of euphoria: a crowdsourcing platform for QoE assessment , 2010, IEEE Network.

[71]  Sujit Dey,et al.  Adaptive Bit Rate capable video caching and scheduling , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[72]  Panagiotis G. Ipeirotis,et al.  Demographics and Dynamics of Mechanical Turk Workers , 2018, WSDM.

[73]  Ali C. Begen,et al.  Probe and Adapt: Rate Adaptation for HTTP Video Streaming At Scale , 2013, IEEE Journal on Selected Areas in Communications.

[74]  Jinwoo Kim,et al.  Deep Video Quality Assessor: From Spatio-Temporal Visual Sensitivity to a Convolutional Neural Aggregation Network , 2018, ECCV.

[75]  Christian Timmerer,et al.  Statistically Indifferent Quality Variation: An Approach for Reducing Multimedia Distribution Cost for Adaptive Video Streaming Services , 2017, IEEE Transactions on Multimedia.

[76]  Nick McKeown,et al.  A buffer-based approach to rate adaptation , 2014, SIGCOMM.

[77]  Lea Skorin-Kapov,et al.  Cloud gaming QoE models for deriving video encoding adaptation strategies , 2016, MMSys.