Side Information and Noise Learning for Distributed Video Coding Using Optical Flow and Clustering

Distributed video coding (DVC) is a coding paradigm that exploits the source statistics at the decoder side to reduce the complexity at the encoder. The coding efficiency of DVC critically depends on the quality of side information generation and accuracy of noise modeling. This paper considers transform domain Wyner-Ziv (TDWZ) coding and proposes using optical flow to improve side information generation and clustering to improve the noise modeling. The optical flow technique is exploited at the decoder side to compensate for weaknesses of block-based methods, when using motion-compensation to generate side information frames. Clustering is introduced to capture cross band correlation and increase local adaptivity in the noise modeling. This paper also proposes techniques to learn from previously decoded WZ frames. Different techniques are combined by calculating a number of candidate soft side information for low density parity check accumulate decoding. The proposed decoder side techniques for side information and noise learning (SING) are integrated in a TDWZ scheme. On test sequences, the proposed SING codec robustly improves the coding efficiency of TDWZ DVC. For WZ frames using a GOP size of 2, up to 4-dB improvement or an average (Bjøntegaard) bit-rate savings of 37% is achieved compared with DISCOVER.

[1]  Mads Nielsen,et al.  TV-L1 Optical Flow for Vector Valued Images , 2011, EMMCVPR.

[2]  Evan Herbst,et al.  Occlusion Reasoning for Temporal Interpolation using Optical Flow , 2009 .

[3]  Bernd Girod,et al.  Rate-adaptive codes for distributed source coding , 2006, Signal Process..

[4]  Pamela C. Cosman,et al.  Wyner–Ziv Video Coding With Classified Correlation Noise Estimation and Key Frame Coding Mode Selection , 2011, IEEE Transactions on Image Processing.

[5]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[6]  Catarina Brites,et al.  Distributed Video Coding with multiple side information , 2009, 2009 Picture Coding Symposium.

[7]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[8]  Michael Gastpar,et al.  Distributed Source Coding - Theory, Algorithms and Applications , 2009 .

[9]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[10]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[11]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[12]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[13]  Joachim Weickert,et al.  A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion , 2001, International Journal of Computer Vision.

[14]  Kannan Ramchandran,et al.  PRISM: A Video Coding Paradigm With Motion Estimation at the Decoder , 2007, IEEE Transactions on Image Processing.

[15]  James C. Bezdek,et al.  Efficient Implementation of the Fuzzy c-Means Clustering Algorithms , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[17]  Oscar C. Au,et al.  Transform-Domain Adaptive Correlation Estimation (TRACE) for Wyner–Ziv Video Coding , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Catarina Brites,et al.  Refining Side Information for Improved Transform Domain Wyner-Ziv Video Coding , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  Xin Huang,et al.  Adaptive noise model for transform domain Wyner-Ziv video using clustering of DCT blocks , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[21]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Optic Flow in Harmony Optic Flow in Harmony Optic Flow in Harmony , 2022 .

[22]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[23]  Christine Guillemot,et al.  Optimal Reconstruction in Wyner-Ziv Video Coding with Multiple Side Information , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[24]  Bernd Girod,et al.  Distributed Video Coding , 2005, Proceedings of the IEEE.

[25]  Catarina Brites,et al.  Correlation Noise Modeling for Efficient Pixel and Transform Domain Wyner–Ziv Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2005, International Journal of Computer Vision.

[27]  Rik Van de Walle,et al.  Efficient Low-Delay Distributed Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Mads Nielsen,et al.  Multi-hypothesis transform domain Wyner-Ziv video coding including optical flow , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[29]  Shuang Wang,et al.  Adaptive Correlation Estimation With Particle Filtering for Distributed Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Xin Huang,et al.  Cross-band noise model refinement for transform domain Wyner-Ziv video coding , 2012, Signal Process. Image Commun..

[31]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.