QWatch: Detecting and Locating QoE Anomaly for VoD in the Cloud

Commercial large-scale VoD systems such as Netflix and Hulu rely on CDNs to deliver videos to users around the world. Various anomalies occur often and degrade users' Quality of Experience (QoE). Detecting and locating such anomalies are highly complex due to a large number of different entities involved in the end-to-end video delivery. These entities include VoD provider, CDN/Cloud providers, transit ISPs, access ISPs, and end user devices. QoE perceived by the users is a critical metric for VoD providers. We propose QWatch, a scalable monitoring system, which detects and locates anomalies based on the end user QoE in real-time. We evaluate QWatch in a controlled VoD system and production Microsoft Azure Cloud and CDN. QWatch effectively detects and locates QoE anomalies in our extensive experiments. We discuss insights obtained from running VoD system with 200 worldwide users in production Cloud.

[1]  Arian Bär,et al.  On the detection of network traffic anomalies in content delivery network services , 2014, 2014 26th International Teletraffic Congress (ITC).

[2]  K. Claffy,et al.  Topology discovery by active probing , 2002, Proceedings 2002 Symposium on Applications and the Internet (SAINT) Workshops.

[3]  Mark Crovella,et al.  Efficient algorithms for large-scale topology discovery , 2004, SIGMETRICS '05.

[4]  Pedro Casas,et al.  Who to blame when YouTube is not working? detecting anomalies in CDN-provisioned services , 2014, 2014 International Wireless Communications and Mobile Computing Conference (IWCMC).

[5]  Alexander Raake,et al.  Quality of experience and HTTP adaptive streaming: A review of subjective studies , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[6]  Qi Zhao,et al.  Towards automated performance diagnosis in a large IPTV network , 2009, SIGCOMM '09.

[7]  Thomas Stockhammer,et al.  Dynamic adaptive streaming over HTTP --: standards and design principles , 2011, MMSys.

[8]  F. Pukelsheim.  The Three Sigma Rule , 1994 .

[9]  Fang Hao,et al.  Unreeling netflix: Understanding and improving multi-CDN movie delivery , 2012, 2012 Proceedings IEEE INFOCOM.

[10]  Kjell Brunnström,et al.  Estimating the impact of single and multiple freezes on video quality , 2011, Electronic Imaging.

[11]  Prashant J. Shenoy,et al.  Empirical evaluation of latency-sensitive application performance in the cloud , 2010, MMSys '10.

[12]  Vyas Sekar,et al.  Shedding light on the structure of internet video quality problems in the wild , 2013, CoNEXT.

[13]  Peter Reichl,et al.  Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience , 2013, Telecommun. Syst..

[14]  Antonio Pescapè,et al.  Cloud monitoring: A survey , 2013, Comput. Networks.

[15]  Hong Jiang,et al.  Meeting service level agreement cost-effectively for video-on-demand applications in the cloud , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[16]  Chen Wang,et al.  QoE Driven Server Selection for VoD in the Cloud , 2015, 2015 IEEE 8th International Conference on Cloud Computing.

[17]  Ramesh K. Sitaraman,et al.  The Akamai network: a platform for high-performance internet applications , 2010, OPSR.

[18]  Konstantina Papagiannaki,et al.  Identifying the root cause of video streaming issues on mobile devices , 2015, CoNEXT.

[19]  D. Goderis,et al.  Service level agreements: a main challenge for next generation networks , 2002, 2nd European Conference on Universal Multiservice Networks. ECUMN'2001 (Cat. No.02EX563).

[20]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[21]  Chen Wang,et al.  Users Know Better: A QoE Based Adaptive Control System for VoD in the Cloud , 2014, GLOBECOM 2014.

[22]  Arian Bär,et al.  When YouTube Does not Work—Analysis of QoE-Relevant Degradation in Google CDN Traffic , 2014, IEEE Transactions on Network and Service Management.

[23]  Srinivasan Seshan,et al.  A quest for an Internet video quality-of-experience metric , 2012, HotNets-XI.

[24]  David E. Culler,et al.  PlanetLab: an overlay testbed for broad-coverage services , 2003, CCRV.

[25]  Ali C. Begen,et al.  An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP , 2011, MMSys.