Synthesis of fine details in B picture for dynamic textures

Dynamic textures are characterized with irregular motions that are often challenging for motion compensation as applied in the state of the art video codecs. Due to rapid and randomly evolving nature of such a signal, it is accompanied with very high energy in the residual. As a result, B-pictures as used in HEVC layer are relative expensive to code. This leads to an overall increase in the bitrate. Further, increasing QPoffset worsens the quality by forcing lower rate to these B-pictures, leading to strong blurring and blocking artefacts. In this paper, we exploit Steerable Pyramid (SP) for coding pictures with tid > 2 in a downsampled format. At the decoder side, details are synthesized for these low resolution pictures by adding back the high frequencies using motion compensation from the nearest key picture followed by an inverse SP transform. The paper synthesizes details for the dynamic textures that are expensive to code. Our investigation shows up to 31% saving in bitrate, while visual quality is kept acceptable.

[1]  Fan Zhang,et al.  A Parametric Framework for Video Compression Using Region-Based Texture Models , 2011, IEEE Journal of Selected Topics in Signal Processing.

[2]  Ling Lu,et al.  An Improved Rate Control Algorithm of H.264/AVC Based on Human Visual System , 2012 .

[3]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[4]  Fan Zhang,et al.  Video texture analysis based on HEVC encoding statistics , 2016, 2016 Picture Coding Symposium (PCS).

[5]  Fan Zhang,et al.  An adaptive QP offset determination method for HEVC , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[6]  Mark J. Huiskes,et al.  DynTex: A comprehensive database of dynamic textures , 2010, Pattern Recognit. Lett..

[7]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[8]  Barry G. Haskell,et al.  A texture replacement method at the encoder for bit-rate reduction of compressed video , 2003, IEEE Trans. Circuits Syst. Video Technol..

[9]  Uday Singh Thakur,et al.  Texture analysis and synthesis using steerable pyramid decomposition for video coding , 2015, 2015 International Conference on Systems, Signals and Image Processing (IWSSIP).

[10]  Truong Q. Nguyen,et al.  Subpixel motion estimation without interpolation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Uday Singh Thakur,et al.  Image coding using parametric texture synthesis , 2016, 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP).

[12]  Mathias Wien,et al.  High Efficiency Video Coding: Coding Tools and Specification , 2014 .

[13]  Mathias Wien,et al.  Motion-based analysis and synthesis of dynamic textures , 2016, 2016 Picture Coding Symposium (PCS).

[14]  Wei Zhang,et al.  The SJTU 4K video sequence dataset , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[15]  Mathias Wien,et al.  Dynamic texture synthesis using linear phase shift interpolation , 2016, 2016 Picture Coding Symposium (PCS).

[16]  Jens-Rainer Ohm,et al.  Models for Static and Dynamic Texture Synthesis in Image and Video Compression , 2011, IEEE Journal of Selected Topics in Signal Processing.

[17]  Hua Li,et al.  Perceptually Adaptive Lagrange Multiplier for Rate-Distortion Optimization in H.264 , 2007, Future Generation Communication and Networking (FGCN 2007).

[18]  Fan Zhang,et al.  A video texture database for perceptual compression and quality assessment , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[19]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Heiko Schwarz,et al.  Improved H.264/AVC coding using texture analysis and synthesis , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[21]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Fabien Racapé,et al.  2D+t autoregressive framework for video texture completion , 2014, 2014 IEEE International Conference on Image Processing (ICIP).