Adaptive Frequency Weighting for High-Performance Video Coding

The benefits of utilizing the spatial-frequency sensitivity of the human visual system in video coding have been known for decades and are incorporated into video standards, such as MPEG2, MPEG4, and H.264/AVC, as well as into still image-coding standards, such as JPEG and JPEG-2000. However, all of these standards utilize frequency weighting that is adaptive only at the picture level. In contrast with such picture-level adaptive frequency-weighting algorithms, we describe a coding approach in which a macroblock-level adaptive frequency-weighting (MBAFW) algorithm is used. We show that for the bitrate, quality levels, and resolutions typically associated with high-definition video, MBAFW can lead to overall bitrate reductions typically in the 5%-9% range over the methods used in state-of-the-art video-coding standards.

[1]  Marios S. Pattichis,et al.  Foveated video quality assessment , 2002, IEEE Trans. Multim..

[2]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[3]  Gary Moody,et al.  Eye tracker system for use with head mounted displays , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[4]  Qingxin Meng,et al.  Improved Macroblock-Level Rate Control Algorithm with Visual Properties , 2010, 2010 2nd International Workshop on Intelligent Systems and Applications.

[5]  Henrique S. Malvar,et al.  Low-complexity transform and quantization in H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[6]  K. R. Rao,et al.  Human visual weighted progressive image transmission , 1990, IEEE Trans. Commun..

[7]  J. Robson,et al.  Probability summation and regional variation in contrast sensitivity across the visual field , 1981, Vision Research.

[8]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  James Hu,et al.  DVQ: A digital video quality metric based on human vision , 2001 .

[10]  Bo Yan,et al.  Lagrangian Multiplier Based Joint Three-Layer Rate Control for H.264/AVC , 2009, IEEE Signal Processing Letters.

[11]  T.Q. Nguyen,et al.  Perceptual Video Coding with H.264 , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[12]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[13]  Larry S. Davis,et al.  A survey of edge detection techniques , 1975 .

[14]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[15]  Chih-Wei Tang,et al.  Spatiotemporal Visual Considerations for Video Coding , 2007, IEEE Transactions on Multimedia.

[16]  Peter G. J. Barten,et al.  Evaluation of Subjective Image Quality with the Square Root Integral Method , 1990, Applied Vision.

[17]  Bernd Girod,et al.  Eye Movements And Coding Of Video Sequences , 1988, Other Conferences.

[18]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[19]  Hojin Ha,et al.  Macroblock-Based Frequency Selective Weighting for Visual Scalable Video Coding of H.264/AVC , 2009, IEEE Transactions on Broadcasting.

[20]  Chun-Jen Tsai,et al.  Visual sensitivity guided bit allocation for video coding , 2006, IEEE Transactions on Multimedia.

[21]  Bernd Girod,et al.  What's wrong with mean-squared error? , 1993 .

[22]  Jack Y. B. Lee.  On a unified architecture for video-on-demand services , 2002, IEEE Trans. Multim..

[23]  S J Anderson,et al.  Peripheral spatial vision: limits imposed by optics, photoreceptors, and receptor pooling. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[24]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[25]  Jungwoo Lee.  Rate-distortion optimization of parametrized quantization matrix for MPEG-2 encoding , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[26]  He Yun,et al.  Macroblock-Level Adaptive Frequency Weighting for Perceptual Video Coding , 2007 .