Prometheus: toward quality-of-experience estimation for mobile apps from passive network measurements

Cellular network operators are now expected to maintain a good Quality of Experience (QoE) for many services beyond circuit-switched voice and messaging. However, new smart-phone "app" services, such as Over The Top (OTT) video delivery, are not under an operator's control. Furthermore, complex interactions between network protocol layers make it challenging for operators to understand how network-level parameters (e.g., inactivity timers, handover thresholds, middle boxes) will influence a specific app's QoE. This paper takes a first step to address these challenges by presenting a novel approach to estimate app QoE using passive network measurements. Our approach uses machine learning to obtain a function that relates passive measurements to an app's QoE. In contrast to previous approaches, our approach does not require any control over app services or domain knowledge about how an app's network traffic relates to QoE. We implemented our approach in Prometheus, a prototype system in a large U.S. cellular operator. We show with anonymous data that Prometheus can measure the QoE of real video-on-demand and VoIP apps with over 80% accuracy, which is close to or exceeds the accuracy of approaches suggested by domain experts.

[1]  Tobias Hoßfeld,et al.  Passive YouTube QoE Monitoring for ISPs , 2012, 2012 Sixth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[2]  Valtteri Niemi,et al.  UMTS Networks: Architecture, Mobility and Services , 2001 .

[3]  Philip J. Corriveau,et al.  VQEG evaluation of objective methods of video quality assessment , 1999 .

[4]  Kuan-Ta Chen,et al.  OneClick: A Framework for Measuring Network Quality of Experience , 2009, IEEE INFOCOM 2009.

[5]  Andries P. Hekstra,et al.  Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[6]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[7]  Carey Williamson,et al.  Multimedia application performance on a WiMAX network , 2009, Electronic Imaging.

[8]  Yin Zhang,et al.  Q-score: proactive service quality assessment in a large IPTV system , 2011, IMC '11.

[9]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[10]  Jaideep Chandrashekar,et al.  Predicting user dissatisfaction with Internet application performance at end-hosts , 2013, 2013 Proceedings IEEE INFOCOM.

[11]  Qiang Xu,et al.  Identifying diverse usage behaviors of smartphone apps , 2011, IMC '11.

[12]  Aruna Seneviratne,et al.  A comparison of mechanisms for improving mobile IP handoff latency for end-to-end TCP , 2003, MobiCom '03.

[13]  Feng Qian,et al.  TOP: Tail Optimization Protocol For Cellular Radio Resource Allocation , 2010, The 18th IEEE International Conference on Network Protocols.

[14]  Heng Cui On the relationship between QoS and QoE for web sessions , 2012 .

[15]  Ramesh K. Sitaraman,et al.  Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs , 2012, IEEE/ACM Transactions on Networking.

[16]  Rob Miller,et al.  GUI testing using computer vision , 2010, CHI.

[17]  Chun-Ying Huang,et al.  Quantifying Skype user satisfaction , 2006, SIGCOMM.

[18]  Shigeyuki Sakazawa,et al.  Objective perceptual video quality measurement method based on hybrid no reference framework , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[19]  METHODS FOR SUBJECTIVE DETERMINATION OF TRANSMISSION QUALITY Summary , 2022 .

[20]  David Soldani Means and methods for collecting and analyzing QoE measurements in wireless networks , 2006, 2006 International Symposium on a World of Wireless, Mobile and Multimedia Networks(WoWMoM'06).

[21]  K. K. Ramakrishnan,et al.  Over the top video: the gorilla in cellular networks , 2011, IMC '11.

[22]  Zhi-Li Zhang,et al.  NEVERMIND, the problem is already fixed: proactively detecting and troubleshooting customer DSL problems , 2010, CoNEXT.

[23]  Andrej Kos,et al.  An approach to modeling and control of QoE in next generation networks [Next Generation Telco IT Architectures] , 2010, IEEE Communications Magazine.

[24]  Srinivasan Seshan,et al.  A quest for an Internet video quality-of-experience metric , 2012, HotNets-XI.