Quality of Experience of VoIP Service: A Survey of Assessment Approaches and Open Issues

This survey gives a comprehensive review of recent advances related to the topic of VoIP QoE (Quality of user' Experience). It starts by providing some insight into the QoE arena and outlines the principal building blocks of a VoIP application. The sources of impairments over data IP networks are identified and distinguished from signal-oriented sources of quality degradation observed over telecom networks. An overview of existing subjective and objective methodologies for the assessment of the QoE of voice conversations is then presented outlining how subjective and objective speech quality methodologies have evolved to consider time-varying QoS transport networks. A description of practical procedures for measuring VoIP QoE and illustrative results is then given. Utilization methodology of several speech quality assessment frameworks is summarized. A survey of emerging single-ended parametric-model speech quality assessment algorithms dedicated to VoIP service is then given. In particular, after presenting a primitive single-ended parametric-model algorithm especially conceived for the evaluation of VoIP conversations, new artificial assessors of VoIP service are detailed. In particular, we describe speech quality assessment algorithms that consider, among others, packet loss burstiness, unequal importance of speech wave, and transient loss of connectivity. The following section concentrates on the integration of VoIP service over mobile data networks. The impact of quality-affecting phenomena, such as handovers and CODEC changeover are enumerated and some primary subjective results are summarized. The survey concludes with a review of open issues relating to automatic assessment of VoIP.

[1]  Hossam Afifi,et al.  Audio quality assessment in packet networks: an "inter-subjective" neural network model , 2001, Proceedings 15th International Conference on Information Networking.

[2]  Choon Shim,et al.  Packet loss burstiness and enhancement to the E-Model , 2005, Sixth International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing and First ACIS International Workshop on Self-Assembling Wireless Network.

[3]  Sangjin Hong,et al.  VoMESH: voice over wireless MESH networks , 2006, IEEE Wireless Communications and Networking Conference, 2006. WCNC 2006..

[4]  Stephen D. Voran,et al.  Objective estimation of perceived speech quality .II. Evaluation of the measuring normalizing block technique , 1999, IEEE Trans. Speech Audio Process..

[5]  Guy Pujolle,et al.  Voicing-aware parametric speech quality models over VoIP networks , 2009, 2009 Global Information Infrastructure Symposium.

[6]  Sofiene Jelassi,et al.  Evom: a software based platform for voice transmission and quality assessment over wireless ad-hoc networks , 2007, MSWiM '07.

[7]  Nobuhiko Kitawaki,et al.  Objective quality evaluation for low-bit-rate speech coding systems , 1988, IEEE J. Sel. Areas Commun..

[8]  Harilaos Koumaras,et al.  Quality of experience issues in multimedia provision , 2012, Telecommun. Syst..

[9]  Davis Yen Pan Digital Audio Compression , 1993, Digit. Tech. J..

[10]  METHODS FOR SUBJECTIVE DETERMINATION OF TRANSMISSION QUALITY Summary , 2022 .

[11]  Liam Murphy,et al.  Exploring the Extent and Impact of Playout Adjustments within VoIP Applications on the MOS Scale , 2005 .

[12]  Zhuoqun Sun,et al.  Voice quality prediction models and their application in VoIP networks , 2006, IEEE Transactions on Multimedia.

[13]  Chin-Laung Lei,et al.  Quadrant of euphoria: a crowdsourcing platform for QoE assessment , 2010, IEEE Network.

[14]  Akira Takahashi,et al.  Objective Assessment Methodology for Estimating Conversational Quality in VoIP , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[15]  Benjamin W. Wah,et al.  Analyzing Voice Quality in Popular VoIP Applications , 2009, IEEE Multimedia.

[16]  Pekka Loula,et al.  Assessment of objective voice quality over best-effort networks , 2005, Comput. Commun..

[17]  Sajal K. Das,et al.  Improving quality of experience for network services [Guest Editorial] , 2010, IEEE Netw..

[18]  Pierre Tirilly,et al.  Evaluating Users' Satisfaction in Packet Networks Using Random Neural Networks , 2006, ICANN.

[19]  Alan Clark,et al.  Modeling the effects of burst packet loss and recency on subjective voice quality , 2001 .

[20]  Juanita Ellis,et al.  Voice, Video, and Data Network Convergence: Architecture and Design, From VoIP to Wireless , 2003 .

[21]  Oded Ghitza,et al.  Objective Assessment of Speech and Audio Quality - Technology and Applications , 2006, IEEE Trans. Speech Audio Process..

[22]  Yali Zhu,et al.  A simulation-based methodology and tool for automating the modeling and analysis of voice-over-IP perceptual quality , 2003, Perform. Evaluation.

[23]  Lingfen Sun,et al.  Perceived speech quality driven retransmission mechanism for wireless VoIP , 2003 .

[24]  Alexander Raake Short- and Long-Term Packet Loss Behavior: Towards Speech Quality Prediction for Arbitrary Loss Distributions , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[25]  Sebastian Möller,et al.  Speech Quality While Roaming in Next Generation Networks , 2009, 2009 IEEE International Conference on Communications.

[26]  Liam Murphy,et al.  Adaptive VoIP playout scheduling: assessing user satisfaction , 2005, IEEE Internet Computing.

[27]  Sofiene Jelassi,et al.  Connectivity Aware Instrumental Approach for Measuring Vocal Transmission Quality Over a Wireless Ad-Hoc Network , 2008, 2008 New Technologies, Mobility and Security.

[28]  Alexander Raake,et al.  Elements of interactivity in telephone conversations , 2004, INTERSPEECH.

[29]  Guy Pujolle,et al.  NIDA: A Parametric Vocal Quality Assessment Algorithm over Transient Connections , 2009, MMNS.

[30]  Takanori Hayashi,et al.  Non-intrusive Quality Monitoring Method of VoIP Speech Based on Network Performance Metrics , 2006, IEICE Trans. Commun..

[31]  Lingfen Sun,et al.  New methods for voice quality evaluation for IP networks , 2003 .

[32]  S. R. Broom,et al.  VoIP Quality Assessment: Taking Account of the Edge-Device , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[33]  Samarth H. Shah,et al.  Optimized Adaptive Jitter Buffer Design for Wireless Internet Telephony , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[34]  Christian Hoene,et al.  Internet Telephony over Wireless Links , 2006 .

[35]  Sangjin Hong,et al.  Performance Optimizations for Deploying VoIP Services in Mesh Networks , 2006, IEEE Journal on Selected Areas in Communications.

[36]  M. Motani,et al.  Cross-layer design: a survey and the road ahead , 2005, IEEE Communications Magazine.

[37]  Frank H. P. Fitzek,et al.  Voice quality evaluation in wireless packet communication systems: a tutorial and performance results for RHC , 2005, IEEE Wireless Communications.

[38]  Kuan-Ta Chen,et al.  OneClick: A Framework for Measuring Network Quality of Experience , 2009, IEEE INFOCOM 2009.

[39]  Henning Schulzrinne,et al.  Application-layer mobility using SIP , 2000, MOCO.

[40]  Nobuhiko Kitawaki,et al.  Pure Delay Effects on Speech Quality in Telecommunications , 1991, IEEE J. Sel. Areas Commun..

[41]  Peter Brooks,et al.  User measures of quality of experience: why being objective and quantitative is important , 2010, IEEE Network.

[42]  Schuyler Quackenbush,et al.  Objective measures of speech quality , 1995 .

[43]  Ritu Singh,et al.  Voice over the internet: A tutorial discussing problems and solutions associated with alternative transport , 2004, IEEE Communications Surveys & Tutorials.

[44]  Ayman Radwan,et al.  Non-intrusive single-ended speech quality assessment in VoIP , 2007, Speech Commun..

[45]  Alexander Raake,et al.  Speech Quality of VoIP - Assessment and Prediction , 2006 .

[46]  Stephen D. Voran,et al.  Objective estimation of perceived speech quality. I. Development of the measuring normalizing block technique , 1999, IEEE Trans. Speech Audio Process..

[47]  S. Voran Perception of Temporal Discontinuity Impairments in Coded Speech - A Proposal for Objective Estimators and Some Subjective Test Results , 2003 .

[48]  Xudong Wang,et al.  VoIP over wireless mesh networks: challenges and approaches , 2006, WICON '06.

[49]  Sebastian Möller,et al.  Impairment Factor Framework for Wide-Band Speech Codecs , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[50]  Ehab Al-Shaer,et al.  Real-Time Audio Quality Evaluation for Adaptive Multimedia Protocols , 2005, MMNS.

[51]  Lingfen Sun,et al.  Speech quality prediction for voice over Internet protocol networks , 2004 .

[52]  Nobuhiko Kitawaki Multimedia Quality Prediction Methodologies for Advanced Mobile and IP-Based Telephony , 2006, IEICE Trans. Commun..

[53]  Guy Pujolle,et al.  Single-ended parametric voicing-aware models for live assessment of packetized VoIP conversations , 2012, Telecommun. Syst..

[54]  Methods for objective and subjective assessment of quality Perceptual evaluation of speech quality ( PESQ ) : An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs , 2002 .

[55]  Yiwei Thomas Hou,et al.  QoS-enabled voice support in the next-generation Internet: issues, existing approaches and challenges , 2000, IEEE Commun. Mag..

[56]  Benjamin W. Wah,et al.  Analysis and Evaluation of the Skype and Google-Talk Voip Systems , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[57]  Soung Chang Liew,et al.  Voice over wireless LAN via IIEE 802.16 wireless MAN and IEEE 802.11 wireless distribution system , 2005, 2005 International Conference on Wireless Networks, Communications and Mobile Computing.

[58]  Akira Takahashi Opinion model for estimating conversational quality of VoIP , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[59]  Robert G. Cole,et al.  Voice over IP performance monitoring , 2001, CCRV.

[60]  Chun-Ying Huang,et al.  Quantifying Skype user satisfaction , 2006, SIGCOMM.

[61]  Ian F. Akyildiz,et al.  Spectrum management in cognitive radio ad hoc networks , 2009, IEEE Network.

[62]  One-view visualization of speech quality measurement results Summary , 2006 .