OneClick: A Framework for Measuring Network Quality of Experience

As the service requirements of network applications shift from high throughput to high media quality, interactivity, and responsiveness, the definition of QoE (Quality of Experience) has become multidimensional. Although it may not be difficult to measure individual dimensions of the QoE, how to capture users' overall perceptions when they are using network applications remains an open question. In this paper, we propose a framework called OneClick to capture users' perceptions when they are using network applications. The framework only requires a subject to click a dedicated key whenever he/she feels dissatisfied with the quality of the application in use. OneClick is particularly effective because it is intuitive, lightweight, efficient, time-aware, and application-independent. We use two objective quality assessment methods, PESQ and VQM, to validate OneClick's ability to evaluate the quality of audio and video clips. To demonstrate the proposed framework's efficiency and effectiveness in assessing user experiences, we implement it on two applications, one for instant messaging applications, and the other for first- person shooter games. A Flash implementation of the proposed framework is also presented.

[1]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[2]  Andries P. Hekstra,et al.  Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[3]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[4]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[5]  Mark Claypool,et al.  The effects of latency on online madden NFL football , 2004, NOSSDAV '04.

[6]  Chun-Ying Huang,et al.  Quantifying Skype user satisfaction , 2006, SIGCOMM.

[7]  Grenville J. Armitage,et al.  An experimental estimation of latency sensitivity in multiplayer Quake 3 , 2003, The 11th IEEE International Conference on Networks, 2003. ICON2003..

[8]  Kuan-Ta Chen,et al.  Effect of Network Quality on Player Departure Behavior in Online Games , 2009, IEEE Transactions on Parallel and Distributed Systems.

[9]  Peter A. Dinda,et al.  The user in experimental computer systems research , 2007, ExpCS '07.

[10]  H.G. De Meer,et al.  Utility curves: mean opinion scores considered biased , 1999, 1999 Seventh International Workshop on Quality of Service. IWQoS'99. (Cat. No.98EX354).

[11]  M. Angela Sasse,et al.  Measuring perceived quality of speech and video in multimedia conferencing applications , 1998, MULTIMEDIA '98.

[12]  Methods for objective and subjective assessment of quality Perceptual evaluation of speech quality ( PESQ ) : An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs , 2002 .

[13]  Kuan-Ta Chen,et al.  Tuning Skype's Redundancy Control Algorithm for User Satisfaction , 2009, IEEE INFOCOM 2009.

[14]  Chin-Laung Lei,et al.  How sensitive are online gamers to network quality? , 2006, CACM.