From Packets to People: Quality of Experience as a New Measurement Challenge

Over the course of the last decade, the concept of Quality of Experience (QoE) has gained strong momentum, both from an academic research and an industry perspective. Being linked very closely to the subjective perception of the end user, QoE is supposed to enable a broader, more holistic understanding of the qualitative performance of networked communication systems and thus to complement the traditional, more technology-centric Quality of Service (QoS) perspective. The purpose of this chapter is twofold: firstly, it introduces the reader to QoE by discussing the origins and the evolution of the concept. Secondly, it provides an overview of the current state of the art of QoE research, with focus on work that particularly addresses QoE as a measurement challenge on the technology as well as on the end-user level. This is achieved by surveying the different streams of QoE research that have emerged in the context of Video, Voice and Web services with respect to the following aspects: fundamental relationships and perceptual principles, QoE assessment, modeling and monitoring.

[1]  Phuoc Tran-Gia,et al.  Quantification of YouTube QoE via Crowdsourcing , 2011, 2011 IEEE International Symposium on Multimedia.

[2]  Paul Strauch,et al.  A robust 60 GHz wireless network with parallel relaying , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[3]  Hendrik Knoche,et al.  Quality of experience in digital mobile multimedia services , 2011 .

[4]  Eric S. Crawley,et al.  A Framework for QoS-based Routing in the Internet , 1998, RFC.

[5]  Allan Kuchinsky,et al.  Integrating user-perceived quality into Web server design , 2000, Comput. Networks.

[6]  P. Killeen,et al.  Optimal timing and the Weber function. , 1987, Psychological review.

[7]  Peter Reichl,et al.  From charging for Quality of Service to charging for Quality of Experience , 2010, Ann. des Télécommunications.

[8]  Gerardo Rubino,et al.  Quality assessment of interactive voice applications , 2008 .

[9]  Jason Nieh,et al.  Understanding the management of client perceived response time , 2006, SIGMETRICS '06/Performance '06.

[10]  Kalevi Kilkki,et al.  Applying user segmentation and estimating user value in techno-economic modeling , 2007, 2007 6th Conference on Telecommunication Techno-Economics.

[11]  Markus Fiedler.  Euro-NGI D.JRA.6.1.1 : State-of-the-art with regards to user-perceived Quality of Service and quality feedback , 2004 .

[12]  Kay Connelly,et al.  Toward total quality of experience: A QoE model in a communication ecosystem , 2012, IEEE Communications Magazine.

[13]  Pamela C. Cosman,et al.  Modeling packet-loss visibility in MPEG-2 video , 2006, IEEE Transactions on Multimedia.

[14]  Eelco Herder,et al.  Not quite the average: An empirical study of Web use , 2008, TWEB.

[15]  Moorsel A van.  Metrics for the Internet Age: Quality of Experience and Quality of Business , 2001 .

[16]  丸井 淳史.  Soren Bech and Nick Zacharov, Perceptual Audio Evaluation-Theory, Method, and Application, John Wiley & Sons Ltd., 2006(Coffee Break) , 2014 .

[17]  Helmut Hlavacs,et al.  Slider or glove? Proposing an alternative quality rating methodology , 2010 .

[18]  Akira Takahashi,et al.  Objective Assessment Methodology for Estimating Conversational Quality in VoIP , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  A. Bouch,et al.  Of packets and people: a user-centered approach to quality of service , 2000, 2000 Eighth International Workshop on Quality of Service. IWQoS 2000 (Cat. No.00EX400).

[20]  M. Angela Sasse,et al.  Do Users Always Know What's Good For Them? Utilising Physiological Responses to Assess Media Quality , 2000, BCS HCI.

[21]  Sebastian Möller,et al.  Speech Quality Estimation: Models and Trends , 2011, IEEE Signal Processing Magazine.

[22]  L. Allan.  The perception of time , 1979 .

[23]  Eva Ibarrola,et al.  Web QoE Evaluation in Multi-agent Networks: Validation of ITU-T G.1030 , 2009, 2009 Fifth International Conference on Autonomic and Autonomous Systems.

[24]  Allan Kuchinsky,et al.  Quality is in the eye of the beholder: meeting users' requirements for Internet quality of service , 2000, CHI.

[25]  Regan L. Mandryk,et al.  Using psychophysiological techniques to measure user experience with entertainment technologies , 2006, Behav. Inf. Technol..

[26]  Stefan Winkler,et al.  Video quality measurement standards — Current status and trends , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[27]  Alexander Raake,et al.  Frame-layer packet-based parametric video quality model for encrypted video in IPTV services , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[28]  Akira Takahashi.  Opinion model for estimating conversational quality of VoIP , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[29]  Raimund Schatz,et al.  It takes two to tango - assessing the impact of delay on conversational interactivity on perceived speech quality , 2010, INTERSPEECH.

[30]  Markus Fiedler,et al.  Testing the IQX Hypothesis for Exponential Interdependency between QoS and QoE of Voice Codecs iLBC and G.711 , 2008 .

[31]  Gerardo Rubino,et al.  Quantifying the Quality of Audio and Video Transmissions over the Internet: The PSQA Approach , 2006 .

[32]  Stefan Winkler,et al.  The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics , 2008, IEEE Transactions on Broadcasting.

[33]  Alexander Raake,et al.  Towards content-related features for parametric video quality prediction of IPTV services , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[34]  Yongxia Skadberg,et al.  Visitors' flow experience while browsing a Web site: its measurement, contributing factors and consequences , 2004, Comput. Hum. Behav..

[35]  Nobuhiko Kitawaki,et al.  A model for evaluating talker echo and sidetone in a telephone transmission network , 1992, IEEE Trans. Commun..

[36]  Itu-T and Iso Iec Jtc.  Advanced video coding for generic audiovisual services , 2010 .

[37]  Markus Fiedler,et al.  The memory effect and its implications on Web QoE modeling , 2011, 2011 23rd International Teletraffic Congress (ITC).

[38]  Sebastian Möller,et al.  A new dimension-based framework model for the quality of speech communication services , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[39]  Peter Brooks,et al.  User measures of quality of experience: why being objective and quantitative is important , 2010, IEEE Network.

[40]  Mousa Al-Akhras,et al.  Non-intrusive speech quality prediction in VoIP networks using a neural network approach , 2009, Neurocomputing.

[41]  Nirwan Ansari,et al.  On assuring end-to-end QoE in next generation networks: challenges and a possible solution , 2011, IEEE Communications Magazine.

[42]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[43]  Thomas Plagemann,et al.  Interactive Distributed Multimedia Systems and Telecommunication Services , 1998, Lecture Notes in Computer Science.

[44]  Luc Martens,et al.  Proposed Framework for Evaluating Quality of Experience in a Mobile, Testbed-oriented Living Lab Setting , 2010, Mob. Networks Appl..

[45]  Yubing Wang.  Survey of Objective Video Quality Measurements , 2006 .

[46]  Zhuoqun Sun,et al.  Voice quality prediction models and their application in VoIP networks , 2006, IEEE Transactions on Multimedia.

[47]  D. L. Richards,et al.  Calculation of opinion scores for telephone connections , 1974 .

[48]  Peter Reichl,et al.  Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience , 2013, Telecommun. Syst..

[49]  Jin Cao,et al.  Measuring Human Satisfaction in Data Networks , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[50]  H Eisler,et al.  Subjective duration and psychophysics. , 1975, Psychological review.

[51]  S. S. Stevens.  On the psychophysical law. , 1957, Psychological review.

[52]  Peter Reichl,et al.  “Time is bandwidth”? Narrowing the gap between subjective time perception and Quality of Experience , 2012, 2012 IEEE International Conference on Communications (ICC).

[53]  Raimund Schatz,et al.  Poor, Good Enough or Even Better? Bridging the Gap between Acceptability and QoE of Mobile Broadband Data Services , 2011, 2011 IEEE International Conference on Communications (ICC).

[54]  Peter C. Verhoef,et al.  Erim Report Series Research in Management Consumer Perception and Evaluation of Waiting Time: a Field Experiment Bibliographic Data and Classifications , 2022 .

[55]  Jing Hu,et al.  Use of content complexity factors in video over IP quality monitoring , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[56]  Margaret H. Pinson,et al.  Comparing subjective video quality testing methodologies , 2003, Visual Communications and Image Processing.

[57]  Vikas Kumar.  Managing Customers for Profit: Strategies to Increase Profits and Build Loyalty , 2008 .

[58]  Raimund Schatz,et al.  Vienna surfing: assessing mobile broadband quality in the field , 2011, W-MUST '11.

[59]  Hans-Jurgen Zepernick,et al.  Quality of Experience of Multimedia Services : Past, Present, and Future , 2011 .

[60]  Marcus Barkowsky,et al.  NEW REQUIREMENTS OF SUBJECTIVE VIDEO QUALITY ASSESSMENT METHODOLOGIES FOR 3DTV , 2010 .

[61]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[62]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[63]  Alexander Raake.  Short- and Long-Term Packet Loss Behavior: Towards Speech Quality Prediction for Arbitrary Loss Distributions , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[64]  Ute Jekosch,et al.  Voice and Speech Quality Perception: Assessment and Evaluation , 2005 .

[65]  Chin-Laung Lei,et al.  Quadrant of euphoria: a crowdsourcing platform for QoE assessment , 2010, IEEE Network.

[66]  Wolfgang Kellerer,et al.  QoE-Based Cross-Layer Optimization of Wireless Video with Unperceivable Temporal Video Quality Fluctuation , 2011, 2011 IEEE International Conference on Communications (ICC).

[67]  Marcus Barkowsky,et al.  The Influence of Subjects and Environment on Audiovisual Subjective Tests: An International Study , 2012, IEEE Journal of Selected Topics in Signal Processing.

[68]  Andrew B. Watson,et al.  Digital images and human vision , 1993 .

[69]  Lucjan Janowski,et al.  Modeling subjective tests of quality of experience with a Generalized Linear Model , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[70]  S. Grondin,et al.  From physical time to the first and second moments of psychological time. , 2001, Psychological bulletin.

[71]  Janusz Gozdecki,et al.  Quality of service terminology in IP networks , 2003, IEEE Commun. Mag..

[72]  Sebastian Möller,et al.  T-V-model: Parameter-based prediction of IPTV quality , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[73]  Doh-Suk Kim,et al.  ANIQUE+: A new American national standard for non-intrusive estimation of narrowband speech quality: Research Articles , 2007 .

[74]  Markus Fiedler,et al.  Quality of Experience from user and network perspectives , 2010, Ann. des Télécommunications.

[75]  Simon Grondin,et al.  Sensory modalities and temporal processing. , 2003 .

[76]  Steven C. Seow.  Designing and Engineering Time: The Psychology of Time Perception in Software , 2008 .

[77]  H.G. De Meer,et al.  Utility curves: mean opinion scores considered biased , 1999, 1999 Seventh International Workshop on Quality of Service. IWQoS'99. (Cat. No.98EX354).

[78]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[79]  Sebastian Möller,et al.  Telephone speech quality prediction: Towards network planning and monitoring models for modern network scenarios , 2002, Speech Commun..

[80]  B. Joseph Pine,et al.  The Experience Economy , 1999 .

[81]  Florian Wickelmaier,et al.  Perceptual Audio Evaluation - Theory, Method and Application , 2006 .

[82]  Vyas Sekar,et al.  Understanding the impact of video quality on user engagement , 2011, SIGCOMM 2011.

[83]  Douglas D. O'Shaughnessy.  Speech Communications: Human and Machine , 2012 .

[84]  Hede Helfrich,et al.  Time and mind II : information processing perspectives , 2003 .

[85]  Dominik Strohmeier,et al.  Open Profiling of Quality: A Mixed Method Approach to Understanding Multimodal Quality Perception , 2010, Adv. Multim..

[86]  Andrzej Jajszczyk,et al.  A survey of QoE assurance in converged networks , 2011, Comput. Networks.

[87]  Villy Bæk Iversen,et al.  TELETRAFFIC ENGINEERING HANDBOOK , 2001 .

[88]  Alan Clark,et al.  Modeling the effects of burst packet loss and recency on subjective voice quality , 2001 .

[89]  Jean-Laurent Costeux,et al.  Passive Estimation of Quality of Experience , 2008, J. Univers. Comput. Sci..

[90]  Yoshihiro Ito,et al.  Quantitative assessment of user-level QoS and its mapping , 2005, IEEE Transactions on Multimedia.

[91]  Sebastian Möller,et al.  Quality Dimensions of Narrowband and Wideband Speech Transmission , 2010 .

[92]  R. Hogarth.  Judgement and choice: The psychology of decision , 1982 .

[93]  Markus Fiedler,et al.  A generic quantitative relationship between quality of experience and quality of service , 2010, IEEE Network.

[94]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[95]  Chang Wen Chen,et al.  A study on perception of mobile video with surrounding contextual influences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[96]  Jakob Nielsen,et al.  Speed Up Your Site: Web Site Optimization , 2003 .

[97]  David Hands,et al.  A Study of the Impact of Network Loss and Burst Size on Video Streaming Quality and Acceptability , 1999, IDMS.

[98]  S Niida,et al.  Mobile Services , 2010, IEEE Vehicular Technology Magazine.

[99]  J. Berger,et al.  P.563—The ITU-T Standard for Single-Ended Speech Quality Assessment , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[100]  Alexander Raake,et al.  Speech Quality of VoIP - Assessment and Prediction , 2006 .

[101]  Nicolas Staelens,et al.  Integrating the everyday-life context in subjective video quality experiments , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[102]  Antony William Rix,et al.  Perceptual evaluation of speech quality (PESQ): The new ITU standard for end-to-end speech quality a , 2002 .

[103]  Søren Bech,et al.  Perceptual Audio Evaluation-Theory, Method and Application: Bech/Perceptual Audio Evaluation-Theory, Method and Application , 2006 .

[104]  Sebastian Möller,et al.  Assessment and Prediction of Speech Quality in Telecommunications , 2000 .

[105]  Doh-Suk Kim,et al.  ANIQUE+: A new American national standard for non-intrusive estimation of narrowband speech quality , 2007, Bell Labs Technical Journal.

[106]  David Soldani,et al.  QoS and QoE Management in UMTS Cellular Systems: Soldani/QoS and QoE Management in UMTS Cellular Systems , 2006 .

[107]  S. Grondin.  Timing and time perception: A review of recent behavioral and neuroscience findings and theoretical directions , 2010, Attention, perception & psychophysics.

[108]  M. Angela Sasse,et al.  Why Value Is Everything: A User-Centered Approach to Internet Quality of Service and Pricing , 2001, IWQoS.

[109]  Stefan Winkler,et al.  Issues in vision modeling for perceptual video quality assessment , 1999, Signal Process..

[110]  Gernot Kubin,et al.  Same but different? — Using speech signal features for comparing conversational VoIP quality studies , 2012, 2012 IEEE International Conference on Communications (ICC).

[111]  Philip J. Corriveau,et al.  Study of Rating Scales for Subjective Quality Assessment of High-Definition Video , 2011, IEEE Transactions on Broadcasting.

[112]  Christian Timmerer,et al.  Challenges of QoE management for cloud applications , 2012, IEEE Communications Magazine.

[113]  Peter Reichl,et al.  The Logarithmic Nature of QoE and the Role of the Weber-Fechner Law in QoE Assessment , 2010, 2010 IEEE International Conference on Communications.

[114]  John Woods,et al.  QoS arbitration for improving the QoE in multimedia transmission , 2003 .

[115]  Heng Cui,et al.  Trouble shooting interactive web sessions in a home environment , 2011, HomeNets '11.

[116]  Gerardo Gómez,et al.  End-to-end quality of service over cellular networks : data services performance and optimization in 2G/3G , 2006 .

[117]  Marcus Barkowsky,et al.  Standardized toolchain and model development for video quality assessment — The mission of the Joint Effort Group in VQEG , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[118]  Ernst Heinrich Weber,et al.  De pulsu, resorptione, auditu et tactu. Annotationes anatomicae et physiologicae , 1834 .

[119]  Markus Fiedler,et al.  Quantification of Quality of Experience for Edge-Based Applications , 2007, ITC.