Learning from users: a data-driven method of QoE evaluation for Internet video

Improving quality of experience (QoE) is increasingly significant for Internet video content providers. The essential issue is how to evaluate QoE under the complex circumstance of Internet video. Based on the massive user data extracted from a large scale Video-on-Demand (VoD) provider, we present a data-driven, comprehensive and extendible study on the problems of QoE evaluation. The main works of this paper include obtaining QoE-associated features via feature engineering and building an evaluation model on features of different aspects for Internet video QoE. Firstly, for feature engineering, we propose to introduce pattern features of user viewing behaviors that interact with user-perceived video quality. A new method of frequent time series pattern mining is proposed to find typical patterns. We correlate user experience with user-perceived quality features and user behavior pattern features, and consider the impact of confounding factors by applying them as context features into modeling. Secondly, interdependency among features is challenging for QoE evaluation modeling. And the high dimension of feature vector should be considered. To address these challenges, we develop an ensemble method to model the interactions between features and their intricate relationships to user experience. Experiments demonstrate that our approach could achieve sound results in comparison with other related works.

[1]  Yin Zhang,et al.  Q-score: proactive service quality assessment in a large IPTV system , 2011, IMC '11.

[2]  Dalu Zhang,et al.  A content-adaptive video quality assessment method for online media service , 2016, Multimedia Tools and Applications.

[3]  Ramakrishnan Srikant,et al.  Mining sequential patterns , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[4]  Mohammed J. Zaki,et al.  CHARM: An Efficient Algorithm for Closed Itemset Mining , 2002, SDM.

[5]  Pedro M. Domingos A few useful things to know about machine learning , 2012, Commun. ACM.

[6]  Devavrat Shah,et al.  Turbo-charging vertical mining of large databases , 2000, SIGMOD '00.

[7]  Ben Y. Zhao,et al.  Understanding user behavior in large-scale video-on-demand systems , 2006, EuroSys.

[8]  Srinivasan Seshan,et al.  Understanding internet video viewing behavior in the wild , 2013, SIGMETRICS '13.

[9]  Francis K. H. Quek,et al.  Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets , 2003, Pattern Recognit..

[10]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[11]  Manuela Pereira,et al.  Factors Influencing Quality of Experience , 2014, Quality of Experience.

[12]  Johannes Gehrke,et al.  Sequential PAttern mining using a bitmap representation , 2002, KDD.

[13]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, CACM.

[14]  Xiapu Luo,et al.  Inferring the QoE of HTTP video streaming from user-viewing activities , 2011, W-MUST '11.

[15]  Jian Pei,et al.  Mining frequent patterns without candidate generation , 2000, SIGMOD '00.

[16]  Yipeng Zhou,et al.  A study of user behavior in online VoD services , 2014, Comput. Commun..

[17]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[18]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[19]  Jaideep Chandrashekar,et al.  Predicting user dissatisfaction with Internet application performance at end-hosts , 2013, 2013 Proceedings IEEE INFOCOM.

[20]  Ramesh K. Sitaraman,et al.  Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs , 2012, IEEE/ACM Transactions on Networking.

[21]  Antonio Liotta,et al.  Quality of experience management in mobile content delivery systems , 2009, Telecommun. Syst..

[22]  Shiduan Cheng,et al.  A comprehensive data-driven approach to evaluating quality of experience on large-scale internet video service , 2016, 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[23]  K. K. Ramakrishnan,et al.  Understanding couch potatoes: measurement and modeling of interactive usage of IPTV at large scale , 2011, IMC '11.

[24]  M. Greenacre Correspondence analysis in practice , 1993 .

[25]  Ramesh C Agarwal,et al.  Depth first generation of long patterns , 2000, KDD '00.

[26]  Prasant Mohapatra,et al.  QoE prediction model for mobile video telephony , 2016, Multimedia Tools and Applications.

[27]  W. Diewert,et al.  Weighted Country Product Dummy Variable Regressions and Index Number Formulae , 2005 .

[28]  Zhang Dalu,et al.  A content-adaptive video quality assessment method for online media service , 2017 .

[29]  Bo Hu,et al.  Modeling Streaming QoE in Wireless Networks with Large-Scale Measurement of User Behavior , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[30]  Chabane Djeraba,et al.  On Clustering Users' Behaviors in Video Sessions , 2007, DMIN.

[31]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[32]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[33]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[34]  Christoph Freudenthaler,et al.  Bayesian Factorization Machines , 2011 .

[35]  Roberto J. Bayardo,et al.  Efficiently mining long patterns from databases , 1998, SIGMOD '98.

[36]  Gaogang Xie,et al.  Watching videos from everywhere: a study of the PPTV mobile VoD system , 2012, IMC '12.

[37]  Tobias Hoßfeld,et al.  Monitoring YouTube QoE: Is Your Mobile Network Delivering the Right Experience to your Customers? , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[38]  Weicong Ding,et al.  Just One More: Modeling Binge Watching Behavior , 2016, KDD.

[39]  Rocky K. C. Chang,et al.  Measuring the quality of experience of HTTP video streaming , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[40]  Srinivasan Seshan,et al.  Developing a predictive model of quality of experience for internet video , 2013, SIGCOMM.

[41]  Biswanath Mukherjee,et al.  A Survey of User Behavior in VoD Service and Bandwidth-Saving Multicast Streaming Schemes , 2012, IEEE Communications Surveys & Tutorials.

[42]  Srinivasan Seshan,et al.  Analyzing the potential benefits of CDN augmentation strategies for internet video workloads , 2013, Internet Measurement Conference.

[43]  Jacob Søgaard,et al.  On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments , 2017, Multimedia Tools and Applications.

[44]  Srinivasan Parthasarathy,et al.  New Algorithms for Fast Discovery of Association Rules , 1997, KDD.

[45]  SeshanSrinivasan,et al.  Developing a predictive model of quality of experience for internet video , 2013 .

[46]  Jiawei Han,et al.  Discriminative Frequent Pattern Analysis for Effective Classification , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[47]  Alexander Raake,et al.  (Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[48]  Ramesh K. Sitaraman,et al.  Understanding the effectiveness of video ads: a measurement study , 2013, Internet Measurement Conference.

[49]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .