Perceptually Unequal Packet Loss Protection by Weighting Saliency and Error Propagation

We describe a method for achieving perceptually minimal video distortion over packet-erasure networks using perceptually unequal loss protection (PULP). There are two main ingredients in the algorithm. First, a perceptual weighting scheme is employed wherein the compressed video is weighted as a function of the nonuniform distribution of retinal photoreceptors. Secondly, packets are assigned temporal importance within each group of pictures (GOP), recognizing that the severity of error propagation increases with elapsed time within a GOP. Using both frame-level perceptual importance and GOP-level hierarchical importance, the PULP algorithm seeks efficient forward error correction assignment that balances efficiency and fairness by controlling the size of identified salient region(s) relative to the channel state. PULP demonstrates robust performance and significantly improved subjective and objective visual quality in the face of burst packet losses.

[1]  J. Robson,et al.  Probability summation and regional variation in contrast sensitivity across the visual field , 1981, Vision Research.

[2]  Thomas Schierl,et al.  Transport and Signaling of SVC in IP Networks , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  C.-C. Jay Kuo,et al.  Robust video transmission over wideband wireless channel using space-time coded OFDM systems , 2002, 2002 IEEE Wireless Communications and Networking Conference Record. WCNC 2002 (Cat. No.02TH8609).

[4]  L. Itti.  Author address: , 1999 .

[5]  Giovanni Schembra,et al.  Performance analysis of priority encoding transmission of MPEG video streams , 1996, Proceedings of GLOBECOM'96. 1996 IEEE Global Telecommunications Conference.

[6]  Madhu Sudan,et al.  Priority encoding transmission , 1996, IEEE Trans. Inf. Theory.

[7]  Kin-Man Lam,et al.  Channel-adaptive error protection for streaming stored MPEG-4 FGS over error-prone environments , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  James Hu,et al.  DVQ: A digital video quality metric based on human vision , 2001 .

[9]  Bernd Girod,et al.  Analysis of video transmission over lossy channels , 2000, IEEE Journal on Selected Areas in Communications.

[10]  Bernd Girod,et al.  Packet-loss-resilient Internet video streaming , 1998, Electronic Imaging.

[11]  Bernd Girod,et al.  Robust Internet video transmission based on scalable coding and unequal error protection , 1999, Signal Process. Image Commun..

[12]  Zhengguo Li,et al.  A Novel Rate Control Scheme for Low Delay Video Communication of H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Bernd Girod,et al.  What's wrong with mean-squared error? , 1993 .

[14]  Chang Wen Chen,et al.  Video Transmission over MIMO-OFDM System: MDC and Space-Time Coding-Based Approaches , 2007, Adv. Multim..

[15]  Wilson S. Geisler,et al.  Real-time foveated multiresolution system for low-bandwidth video communication , 1998, Electronic Imaging.

[16]  Pascal Frossard,et al.  AMISP: a complete content-based MPEG-2 error-resilient scheme , 2001, IEEE Trans. Circuits Syst. Video Technol..

[17]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[18]  Marios S. Pattichis,et al.  Foveated video quality assessment , 2002, IEEE Trans. Multim..

[19]  Zhou Wang,et al.  Embedded foveation image coding , 2001, IEEE Trans. Image Process..

[20]  Jianhua Zheng,et al.  Macroblock-Level Adaptive Frequency Weighting for Perceptual Video Coding , 2007, IEEE Transactions on Consumer Electronics.

[21]  Dong-Ho Cho,et al.  An adaptive redundancy control method for erasure-code-based real-time data transmission over the Internet , 2001, IEEE Trans. Multim..

[22]  E. O. Elliott.  A model of the switched telephone network for data communications , 1965 .

[23]  Khaled El-Maleh,et al.  Perceptual Temporal Quality Metric for Compressed Video , 2007, IEEE Transactions on Multimedia.

[24]  Jack Y. B. Lee.  On a unified architecture for video-on-demand services , 2002, IEEE Trans. Multim..

[25]  Marios S. Pattichis,et al.  Foveated video compression with optimal rate control , 2001, IEEE Trans. Image Process..

[26]  S J Anderson,et al.  Peripheral spatial vision: limits imposed by optics, photoreceptors, and receptor pooling. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[27]  Nick Feamster,et al.  Packet Loss Recovery for Streaming Video , 2002 .

[28]  Si Wu,et al.  A unified architecture for real-time video-coding systems , 2003, IEEE Trans. Circuits Syst. Video Technol..

[29]  Richard E. Ladner,et al.  Unequal loss protection: graceful degradation of image quality over packet erasure channels through forward error correction , 2000, IEEE Journal on Selected Areas in Communications.

[30]  Chih-Wei Tang,et al.  Spatiotemporal Visual Considerations for Video Coding , 2007, IEEE Transactions on Multimedia.

[31]  Alan C. Bovik,et al.  Foveation-Based Error Resilience and Unequal Error Protection over Mobile Networks , 2003, J. VLSI Signal Process..

[32]  Harsha Sirisena,et al.  Hybrid error control mechanism for video transmission in the wireless IP networks , 1999, 10th IEEE Workshop on Local and Metropolitan Area Networks. Selected Papers (IEEE Cat. No.99EX512).

[33]  Zhou Wang,et al.  Foveation scalable video coding with automatic fixation selection , 2003, IEEE Trans. Image Process..

[34]  Jungwoo Lee.  Rate-distortion optimization of parametrized quantization matrix for MPEG-2 encoding , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[35]  He Yun,et al.  Macroblock-Level Adaptive Frequency Weighting for Perceptual Video Coding , 2007 .

[36]  Si Wu,et al.  Unequal loss protection for robust transmission of motion compensated video over the internet , 2003, Signal Process. Image Commun..

[37]  Aidong Men,et al.  Perceptually Optimized Error-Resilient H.264 Video Streaming System over the Best-Effort Internet , 2005, Sixth International Conference on Parallel and Distributed Computing Applications and Technologies (PDCAT'05).

[38]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[39]  Ming Yin,et al.  A rate control scheme for H.264 video under low bandwidth channel , 2006 .

[40]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[41]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[42]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[43]  Zhengguo Li,et al.  An unequal packet loss resilience scheme for video over the Internet , 2005, IEEE Transactions on Multimedia.

[44]  K. R. Rao,et al.  Human visual weighted progressive image transmission , 1990, IEEE Trans. Commun..

[45]  Yücel Altunbasak,et al.  Error-resilient image and video transmission over the Internet using unequal error protection , 2003, IEEE Trans. Image Process..

[46]  Ajay Luthra,et al.  Introduction to the special issue on streaming video , 2001, IEEE Trans. Circuits Syst. Video Technol..

[47]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..