Packet losses and objective video quality metrics in H.264 video streaming

We analyzed PSNR, BIPSNR, SSIM, 3SSIM, MSSSIM, VQM for various packet loss fractions.A lower bitrate mitigates PSNR impairments caused by packet losses.BIPSNR has less probability than PSNR to be significantly influenced by packet losses.Among structural similarity metrics, SSIM is the least affected by packet losses.VQM metric is the most affected by packet losses. Video quality is a key factor in modern video streaming systems. Video compression artifacts affect it but also delay, jitter and packet loss may compromise it. Objective metrics have been proposed to emulate the human visual system: several experimental works have evaluated their adherence to opinions expressed by real users. Instead of dealing with subjective tests, we focused on the effects of network packet losses on objective metrics: since quality metrics are generally computationally intensive, a convenient approach could consist in inferring information about transmission quality from network impairment statistics. We did experimental tests on two computer-animated videos, whose high color contrasts allow a fair comparison between content dependent and content independent metrics. We studied the encoding parameters that minimize/maximize the values of some metrics (PSNR, BI-PSNR, SSIM, 3SSIM, MSSSIM, VQM) for several packet loss percentages. We analyzed also the Empirical Cumulative Density Function of the degradations of quality metrics.

[1]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Nikolay N. Ponomarenko,et al.  METRICS PERFORMANCE COMPARISON FOR COLOR IMAGE DATABASE , 2008 .

[3]  Adam Wolisz,et al.  Cross-layer optimization of OFDM transmission systems for MPEG-4 video streaming , 2004, Comput. Commun..

[4]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[5]  Imad Abdeljaouad,et al.  A Loss-Based Utility Function for Predicting IPTV Quality of Experience over an Overlay Network , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[6]  Gregory W. Cermak,et al.  Subjective video quality as a function of bit rate frame rate, packet loss, and codec , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[7]  Mainak Chatterjee,et al.  Evaluating Quality of Experience for Streaming Video in Real Time , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[8]  Impairments Oliver Hohlfeld.  Stochastic Packet Loss Model to Evaluate QoE , 2009 .

[9]  Rosario El-Feghali,et al.  Quality metric for video sequences with temporal scalability , 2005, IEEE International Conference on Image Processing 2005.

[10]  Danny De Vleeschauwer,et al.  Protecting IPTV against packet loss: Techniques and trade-offs , 2008 .

[11]  Itu-T and Iso Iec Jtc.  Advanced video coding for generic audiovisual services , 2010 .

[12]  Chih-Heng Ke,et al.  An Evaluation Framework for More Realistic Simulations of MPEG Video Transmission , 2008, J. Inf. Sci. Eng..

[13]  Tiago Gonçalves,et al.  Using metadata in video quality assessment based on the structural similarity (SSIM) index metric , 2010 .

[14]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[15]  Gerardo Rubino,et al.  Optimal Quality-of-Experience Design for a P2P Multi-Source Video Streaming , 2008, 2008 IEEE International Conference on Communications.

[16]  Colin Bailey,et al.  Pause Intensity: A No-Reference Quality Assessment Metric for Video Streaming in TCP Networks , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[17]  Huu-Thanh Nguyen,et al.  On Relationship between Quality of Experience and Quality of Service Metrics for IMS-Based IPTV Networks , 2012, 2012 IEEE RIVF International Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future.

[18]  Hocine Cherifi,et al.  Temporal masking effect on dropped frames at video scene cuts , 2004, IS&T/SPIE Electronic Imaging.

[19]  Pamela C. Cosman,et al.  Predicting H.264 Packet Loss Visibility using a Generalized Linear Model , 2006, 2006 International Conference on Image Processing.

[20]  Adam Wolisz,et al.  EvalVid - A Framework for Video Transmission and Quality Evaluation , 2003, Computer Performance Evaluation / TOOLS.

[21]  Kjell Brunnström,et al.  VQeg validation and ITU standardization of objective perceptual video quality metrics [Standards in a Nutshell] , 2009, IEEE Signal Processing Magazine.

[22]  Marília Curado,et al.  An Overview of Quality of Experience Measurement Challenges for Video Applications in IP Networks , 2010, WWIC.

[23]  Hua Yang,et al.  Subjective Quality Evaluation of Decoded Video in the Presence of Packet Losses , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[24]  Peter Lambert,et al.  Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments , 2010, IEEE Transactions on Broadcasting.

[25]  Zhan Ma,et al.  Modeling the impact of frame rate on perceptual quality of video , 2008, 2008 15th IEEE International Conference on Image Processing.

[26]  Mohammed Ghanbari,et al.  Content-aware intra-refresh for video streaming over lossy links , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[27]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[28]  Alan C. Bovik,et al.  Motion-based perceptual quality assessment of video , 2009, Electronic Imaging.

[29]  Ralf Lehnert,et al.  Prediction of video perceptual quality in the presence of packet loss , 2011, Proceedings of the 11th International Conference on Telecommunications.

[30]  Amy R. Reibman,et al.  Predicting packet-loss visibility using scene characteristics , 2007, Packet Video 2007.

[31]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[32]  Martin Reisslein,et al.  Video Transport Evaluation With H.264 Video Traces , 2012, IEEE Communications Surveys & Tutorials.

[33]  Shuji Tasaka,et al.  The effectiveness of a QoE-based video output scheme for audio-video ip transmission , 2008, ACM Multimedia.

[34]  David Zhang,et al.  A comprehensive evaluation of full reference image quality assessment algorithms , 2012, 2012 19th IEEE International Conference on Image Processing.

[35]  Aidong Men,et al.  Perceptually Optimized Error-Resilient H.264 Video Streaming System over the Best-Effort Internet , 2005, Sixth International Conference on Parallel and Distributed Computing Applications and Technologies (PDCAT'05).

[36]  Alan C. Bovik,et al.  Efficient motion weighted spatio-temporal video SSIM index , 2010, Electronic Imaging.

[37]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[38]  Gerardo Rubino,et al.  QoE Monitoring Platform for Video Delivery Networks , 2007, IPOM.

[39]  Werner Almesberger,et al.  Linux Network Traffic Control -- Implementation Overview , 1999 .

[40]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[41]  Oliver Hohlfeld.  Stochastic Packet Loss Model to Evaluate QoE Impairments , 2009, KiVS.

[42]  Alan C. Bovik,et al.  Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos , 2010, IEEE Transactions on Image Processing.

[43]  Alf Inge Wang,et al.  An Empirical Study of NetEm Network Emulation Functionalities , 2011, 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN).

[44]  Bernd Girod,et al.  Analysis of Packet Loss for Compressed Video: Effect of Burst Losses and Correlation Between Error Frames , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Patrick Le Callet,et al.  PERCEPTUAL EFFECTS OF PACKET LOSS ON H.264/AVC ENCODED VIDEOS , 2009 .

[46]  Zhou Wang,et al.  Video quality assessment using a statistical model of human visual speed perception. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[47]  Stefan Winkler,et al.  Video quality measurement standards — Current status and trends , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[48]  Imad Abdeljaouad,et al.  Utility function for predicting IPTV Quality of Experience based on delay in Overlay Networks , 2013, 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC).

[49]  Margaret H. Pinson,et al.  Comparing subjective video quality testing methodologies , 2003, Visual Communications and Image Processing.

[50]  Osman Ghazali,et al.  Evalvid-RASV: Shaped VBR rate adaptation stored video system , 2010, 2010 2nd International Conference on Education Technology and Computer.

[51]  Rafidah Md Noor,et al.  A generic quantitative relationship between quality of experience and packet loss in video streaming services , 2012, 2012 Fourth International Conference on Ubiquitous and Future Networks (ICUFN).

[52]  Arne Lie,et al.  Evalvid-RA: trace driven simulation of rate adaptive MPEG-4 VBR video , 2008, Multimedia Systems.

[53]  Chih-Heng Ke,et al.  MyEvalvid-NT - A Simulation Tool-set for Video Transmission and Quality Evaluation , 2006, TENCON 2006 - 2006 IEEE Region 10 Conference.

[54]  Zhou Wang,et al.  Translation insensitive image similarity in complex wavelet domain , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[55]  Hocine Cherifi,et al.  Sporadic frame dropping impact on quality perception , 2004, IS&T/SPIE Electronic Imaging.

[56]  Jorge Navarro-Ortiz,et al.  Video Tester — A multiple-metric framework for video quality assessment over IP networks , 2012, IEEE international Symposium on Broadband Multimedia Systems and Broadcasting.

[57]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[58]  Colin Perkins,et al.  Packet Loss Characteristics of IPTV-Like Traffic on Residential Links , 2010, 2010 7th IEEE Consumer Communications and Networking Conference.

[59]  Catiuscia Melle,et al.  A library for RTP relaying and QoS monitoring in Application Layer Multicast , 2013, 2013 Fifth International Conference on Ubiquitous and Future Networks (ICUFN).

[60]  A.C. Begen.  Error Control for IPTV over xDSL Networks , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[61]  Alexander Raake,et al.  Parametric packet-layer video quality model for IPTV , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[62]  Mesud Hadzialic,et al.  An approach to estimate correlation between QoS and perceptual video quality in packet-switched networks , 2012, 2012 Proceedings of the 35th International Convention MIPRO.

[63]  Pamela C. Cosman,et al.  Modeling packet-loss visibility in MPEG-2 video , 2006, IEEE Transactions on Multimedia.

[64]  Maria Jose Bustamante.  Comparison of algorithms for concealing packet losses in the transmission of compressed video , 2010 .

[65]  Gerardo Rubino,et al.  A study of real-time packet video quality using random neural networks , 2002, IEEE Trans. Circuits Syst. Video Technol..

[66]  Amy R. Reibman,et al.  Quality monitoring of video over a packet network , 2004, IEEE Transactions on Multimedia.

[67]  Stefan Winkler,et al.  The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics , 2008, IEEE Transactions on Broadcasting.

[68]  Gary J. Sullivan,et al.  Video Compression - From Concepts to the H.264/AVC Standard , 2005, Proceedings of the IEEE.

[69]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[70]  Markus Fiedler,et al.  A generic quantitative relationship between quality of experience and quality of service , 2010, IEEE Network.

[71]  Chaofeng Li,et al.  Content-weighted video quality assessment using a three-component image model , 2010, J. Electronic Imaging.

[72]  Hong Liu,et al.  H.264 error resilience adaptation to IPTV applications , 2012, IEEE international Symposium on Broadband Multimedia Systems and Broadcasting.

[73]  Sally Floyd,et al.  Why we don't know how to simulate the Internet , 1997, WSC '97.

[74]  Mohammed Ghanbari,et al.  Scope of validity of PSNR in image/video quality assessment , 2008 .

[75]  Tomas Zeman,et al.  Influence of Noise on IPTV over xDSL , 2009, 2009 16th International Conference on Systems, Signals and Image Processing.

[76]  Zhou Wang,et al.  Spatial Pooling Strategies for Perceptual Image Quality Assessment , 2006, 2006 International Conference on Image Processing.

[77]  Patrick Le Callet,et al.  Subjective quality assessment IRCCyN/IVC database , 2004 .

[78]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[79]  Eero P. Simoncelli,et al.  Noise characteristics and prior expectations in human visual speed perception , 2006, Nature Neuroscience.