QoE Modeling for HTTP Adaptive Video Streaming–A Survey and Open Challenges

With the recent increased usage of video services, the focus has recently shifted from the traditional quality of service-based video delivery to quality of experience (QoE)-based video delivery. Over the past 15 years, many video quality assessment metrics have been proposed with the goal to predict the video quality as perceived by the end user. HTTP adaptive streaming (HAS) has recently gained much attention and is currently used by the majority of video streaming services, such as Netflix and YouTube. HAS, using reliable transport protocols, such as TCP, does not suffer from image artifacts due to packet losses, which are common in traditional streaming technologies. Hence, the QoE models developed for other streaming technologies alone are not sufficient. Recently, many works have focused on developing QoE models targeting HAS-based applications. Also, the recently published ITU-T Recommendation series P.1203 proposes a parametric bitstream-based model for the quality assessment of progressive download and adaptive audiovisual streaming services over a reliable transport. The main contribution of this paper is to present a comprehensive overview of recent and currently undergoing works in the field of QoE modeling for HAS. The HAS QoE models, influence factors, and subjective test methodologies are discussed, as well as existing challenges and shortcomings. The survey can serve as a guideline for researchers interested in QoE modeling for HAS and also discusses possible future work.

[1]  Alan C. Bovik,et al.  Learning a Continuous-Time Streaming Video QoE Model , 2018, IEEE Transactions on Image Processing.

[2]  Alan C. Bovik,et al.  An Augmented Autoregressive Approach to HTTP Video Stream Quality Prediction , 2017, ArXiv.

[3]  Yao Wang,et al.  Assessing the visual effect of non-periodic temporal variation of quantization stepsize in compressed video , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[4]  Zhengfang Duanmu,et al.  A Quality-of-Experience Index for Streaming Video , 2017, IEEE Journal of Selected Topics in Signal Processing.

[5]  Ryan Hamilton,et al.  QUIC: A UDP-Based Secure and Reliable Transport for HTTP/2 , 2016 .

[6]  Jonathan Kua,et al.  A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP , 2017, IEEE Communications Surveys & Tutorials.

[7]  Rajiv Soundararajan,et al.  Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Christian Timmerer,et al.  Automated QoE evaluation of Dynamic Adaptive Streaming over HTTP , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[9]  Alan Conrad Bovik,et al.  Study of Temporal Effects on Subjective Video Quality of Experience , 2017, IEEE Transactions on Image Processing.

[10]  J. Astola,et al.  ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS , 2007 .

[11]  Gustavo de Veciana,et al.  Modeling the Time—Varying Subjective Quality of HTTP Video Streams With Rate Adaptations , 2013, IEEE Transactions on Image Processing.

[12]  Alexander Raake,et al.  IP-Based Mobile and Fixed Network Audiovisual Media Services , 2011, IEEE Signal Processing Magazine.

[13]  Graça Bressan,et al.  Quality metric to assess video streaming service over TCP considering temporal location of pauses , 2012, IEEE Transactions on Consumer Electronics.

[14]  Patrick Le Callet,et al.  Objective quality assessment of color images based on a generic perceptual reduced reference , 2008, Signal Process. Image Commun..

[15]  Sebastian Möller,et al.  Audiovisual quality integration for interactive communications , 2013, EURASIP Journal on Audio, Speech, and Music Processing.

[16]  Alexander Raake,et al.  Impact of video resolution changes on QoE for adaptive video streaming , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[17]  Christian Timmerer,et al.  Challenges of QoE management for cloud applications , 2012, IEEE Communications Magazine.

[18]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[19]  Takanori Hayashi,et al.  Parametric Quality-Estimation Model for Adaptive-Bitrate-Streaming Services , 2017, IEEE Transactions on Multimedia.

[20]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[22]  Michael Seufert,et al.  Assessing effect sizes of influence factors towards a QoE model for HTTP adaptive streaming , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[23]  S. E. Avons,et al.  Recency and duration neglect in subjective assessment of television picture quality , 2001 .

[24]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[25]  Liam Murphy,et al.  User perception of adapting video quality , 2006, Int. J. Hum. Comput. Stud..

[26]  Markus Fiedler,et al.  Initial delay vs. interruptions: Between the devil and the deep blue sea , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[27]  Alexander Raake,et al.  On the accuracy of short-term quality models for long-term quality prediction , 2015, 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX).

[28]  Zhou Wang,et al.  Reduced-Reference Image Quality Assessment Using Divisive Normalization-Based Image Representation , 2009, IEEE Journal of Selected Topics in Signal Processing.

[29]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[30]  Cisco Visual Networking Index: Forecast and Methodology 2016-2021.(2017) http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual- networking-index-vni/complete-white-paper-c11-481360.html. High Efficiency Video Coding (HEVC) Algorithms and Architectures https://jvet.hhi.fraunhofer. , 2017 .

[31]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[32]  Lea Skorin-Kapov,et al.  On Additive and Multiplicative QoS-QoE Models for Multiple QoS Parameters , 2016 .

[33]  Dong-Qing Zhang,et al.  Assessing quality of experience for adaptive HTTP video streaming , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[34]  Rocky K. C. Chang,et al.  Measuring the quality of experience of HTTP video streaming , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[35]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[36]  Phuoc Tran-Gia,et al.  A Survey on Quality of Experience of HTTP Adaptive Streaming , 2015, IEEE Communications Surveys & Tutorials.

[37]  Sujit Dey,et al.  Deriving and Validating User Experience Model for DASH Video Streaming , 2015, IEEE Transactions on Broadcasting.

[38]  Alan C. Bovik,et al.  Learning to Predict Streaming Video QoE: Distortions, Rebuffering and Memory , 2017, ArXiv.

[39]  Zhi Li,et al.  Towards Perceptually Optimized End-to-end Adaptive Video Streaming. , 2018, 1808.03898.

[40]  Shahid M. Satti,et al.  P.1203 evaluation of real OTT video services , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[41]  Christian Timmerer,et al.  A Practical Evaluation of Video Codecs for Large-Scale HTTP Adaptive Streaming Services , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[42]  Maria G. Martini,et al.  A reduced-reference perceptual image and video quality metric based on edge preservation , 2012, EURASIP Journal on Advances in Signal Processing.

[43]  Phuoc Tran-Gia,et al.  Quantification of YouTube QoE via Crowdsourcing , 2011, 2011 IEEE International Symposium on Multimedia.

[44]  Markus Fiedler,et al.  A generic quantitative relationship between quality of experience and quality of service , 2010, IEEE Network.

[45]  Michael Seufert,et al.  The Impact of Adaptation Strategies on Perceived Quality of HTTP Adaptive Streaming , 2014, VideoNext '14.

[46]  Adrian Munteanu,et al.  Perceptual video quality assessment in HTTP adaptive streaming , 2015, 2015 IEEE International Conference on Consumer Electronics (ICCE).

[47]  Danny De Vleeschauwer,et al.  Model for estimating QoE of video delivered using HTTP adaptive streaming , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[48]  Narciso García,et al.  Perceptual Quality of HTTP Adaptive Streaming Strategies: Cross-Experimental Analysis of Multi-Laboratory and Crowdsourced Subjective Studies , 2016, IEEE Journal on Selected Areas in Communications.

[49]  Gerardo Rubino,et al.  Quality of experience estimation for adaptive HTTP/TCP video streaming using H.264/AVC , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[50]  Deep Medhi,et al.  Measurement of Quality of Experience of Video-on-Demand Services: A Survey , 2016, IEEE Communications Surveys & Tutorials.

[51]  Alexander Raake,et al.  HTTP adaptive streaming QoE estimation with ITU-T rec. P. 1203: open databases and software , 2018, MMSys.

[52]  F. Bossen,et al.  Common test conditions and software reference configurations , 2010 .

[53]  Luigi Atzori,et al.  Qualia: A multilayer solution for QoE passive monitoring at the user terminal , 2017, 2017 IEEE International Conference on Communications (ICC).

[54]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[55]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality assessment for 3D video compression and transmission , 2011, IEEE Transactions on Consumer Electronics.

[56]  Lea Skorin-Kapov,et al.  Survey and Challenges of QoE Management Issues in Wireless Networks , 2013, J. Comput. Networks Commun..

[57]  Alexander Raake,et al.  A modular HTTP adaptive streaming QoE model — Candidate for ITU-T P.1203 (“P.NATS”) , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[58]  Alexander Raake,et al.  Quality of experience and HTTP adaptive streaming: A review of subjective studies , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[59]  Gary J. Sullivan,et al.  Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC) , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[60]  Vincent Barriac,et al.  Standardization activities in the ITU for a QoE assessment of IPTV , 2008, IEEE Communications Magazine.

[61]  Luigi Atzori,et al.  Challenges of future multimedia QoE monitoring for internet service providers , 2017, Multimedia Tools and Applications.

[62]  Fan Zhang,et al.  Reduced-Reference Image Quality Assessment Using Reorganized DCT-Based Image Representation , 2011, IEEE Transactions on Multimedia.

[63]  Chaminda T. E. R. Hewage,et al.  Edge-Based Reduced-Reference Quality Metric for 3-D Video Compression and Transmission , 2012, IEEE Journal of Selected Topics in Signal Processing.

[64]  Lea Skorin-Kapov,et al.  A multi-dimensional view of QoE: the ARCU model , 2012, 2012 Proceedings of the 35th International Convention MIPRO.

[65]  Prashant J. Shenoy,et al.  Multimedia streaming via TCP: an analytic performance study , 2004, MULTIMEDIA '04.

[66]  Fei Wang,et al.  HAS QoE prediction based on dynamic video features with data mining in LTE network , 2015, Science China Information Sciences.

[67]  Yanjiao Chen,et al.  From QoS to QoE: A Tutorial on Video Quality Assessment , 2015, IEEE Communications Surveys & Tutorials.

[68]  Nagabhushan Eswara,et al.  Streaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[69]  Xiapu Luo,et al.  Inferring the QoE of HTTP video streaming from user-viewing activities , 2011, W-MUST '11.

[70]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[71]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[72]  Iraj Sodagar,et al.  The MPEG-DASH Standard for Multimedia Streaming Over the Internet , 2011, IEEE MultiMedia.

[73]  Nagabhushan Eswara,et al.  A Continuous QoE Evaluation Framework for Video Streaming Over HTTP , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[74]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[75]  Dacheng Yang,et al.  A method of QoE evaluation for adaptive streaming based on bitrate distribution , 2014, 2014 IEEE International Conference on Communications Workshops (ICC).

[76]  Hermann Ebbinghaus (1885).  Memory: A Contribution to Experimental Psychology , 2013, Annals of Neurosciences.

[77]  Kai Zeng,et al.  Display device-adapted video quality-of-experience assessment , 2015, Electronic Imaging.

[78]  Alan C. Bovik,et al.  Continuous Prediction of Streaming Video QoE Using Dynamic Networks , 2017, IEEE Signal Processing Letters.

[79]  Truong Cong Thang,et al.  A novel quality model for HTTP adaptive streaming , 2016, 2016 IEEE Sixth International Conference on Communications and Electronics (ICCE).

[80]  Zhengfang Duanmu,et al.  Quality-of-Experience for Adaptive Streaming Videos: An Expectation Confirmation Theory Motivated Approach , 2018, IEEE Transactions on Image Processing.

[81]  Jan De Cock,et al.  Compression Performance Comparison of x264, x265, libvpx and aomenc for On-Demand Adaptive Streaming Applications , 2018, 2018 Picture Coding Symposium (PCS).

[82]  Martin Slanina,et al.  “To pool or not to pool”: A comparison of temporal pooling methods for HTTP adaptive video streaming , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[83]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[84]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[85]  Alexander Raake,et al.  On the Experimental Biases in User Behavior and QoE Assessment in the Lab , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[86]  Anh T. Pham,et al.  A Multi-Factor QoE Model for Adaptive Streaming over Mobile Networks , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[87]  Yao Wang,et al.  Q-STAR: A Perceptual Video Quality Model Considering Impact of Spatial, Temporal, and Amplitude Resolutions , 2012, IEEE Transactions on Image Processing.