Advanced motion vector coding framework for multiview video sequences

The inconsistency between the temporal motion vector and the disparity vector fields causes a drastic loss in coding performance during multiview video coding. In order to resolve this inconsistency problem, we investigate a new approach for motion vector coding. In this paper, degradation in accuracy using a conventional method is experimentally analyzed using mathematical tools. Then, we introduce a new framework for motion vector coding, including modified Inter-mode and Adaptive Direct-mode, in which an advanced motion vector prediction method is employed. The main idea of the proposed motion vector prediction method is to separate each motion vector field from the others. For this purpose, we introduce three alternative motion vectors; a virtual disparity vector, a virtual temporal motion vector and a scaled motion vector. The experimental results show that the proposed method can resolve the inconsistency problem and consequently improve the overall rate-distortion performance. The average bit-rate savings is about 7.3% compared to JMVC 6.0. At maximum, the gain obtained is about 20%, corresponding to a PSNR of 1.42 dB.

[1]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[2]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[3]  Jin Young Lee,et al.  Temporal and inter-view skip modes for multi-view video coding , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[4]  Sehoon Yea,et al.  View Synthesis Prediction for Rate-Overhead Reduction in FTV , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[5]  Haitao Yang,et al.  Fine-Granular Motion Matching for Inter-View Motion Skip Mode in Multiview Video Coding , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Marek Domanski,et al.  Depth-based inter-view prediction of motion vectors for improved multiview video coding , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[7]  S. Cho,et al.  Adaptive Local Illumination Change Compensation Method for H.264/AVC-Based Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Aljoscha Smolic,et al.  Toward a 3D video format for auto-stereoscopic displays , 2008, Optical Engineering + Applications.

[9]  Sehoon Yea,et al.  View synthesis prediction for multiview video coding , 2009, Signal Process. Image Commun..

[10]  Qionghai Dai,et al.  2D-to-3D Conversion Based on Motion and Color Mergence , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[11]  André Kaup,et al.  Histogram-Based Prefiltering for Luminance and Chrominance Compensation of Multiview Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Nam Ik Cho,et al.  A Motion Vector Prediction Method For Multi-view Video Coding , 2008, 2008 International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[13]  Kwanghoon Sohn,et al.  A Stereoscopic Video Generation Method Using Stereoscopic Display Characterization and Motion Analysis , 2008, IEEE Transactions on Broadcasting.

[14]  Aljoscha Smolic,et al.  An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution , 2009, 2009 Picture Coding Symposium.

[15]  Antonio Ortega,et al.  New Coding Tools for Illumination and Focus Mismatch Compensation in Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Toshiaki Fujii,et al.  Multiview Video Coding Using View Interpolation and Color Correction , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Jin Young Lee,et al.  Analysis of Motion Vector Predictor in Multiview Video Coding System , 2011, 2011 IEEE Ninth International Symposium on Parallel and Distributed Processing with Applications Workshops.

[18]  Sang Uk Lee,et al.  Robust bilayer video segmentation by adaptive propagation of global shape and local appearance , 2010, J. Vis. Commun. Image Represent..

[19]  Toshiaki Fujii,et al.  Multi-View Video Coding using View Interpolation and Reference Picture Selection , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[20]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[21]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[22]  Aljoscha Smolic,et al.  Efficient Prediction Structures for Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  B. Cyganek An Introduction to 3D Computer Vision Techniques and Algorithms , 2009 .

[24]  W. Gao,et al.  Inter-View Direct Mode for Multiview Video Coding , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Jin Young Lee,et al.  An independent motion and disparity vector prediction method for multiview video coding , 2012, Electronic Imaging.

[26]  Jin Young Lee,et al.  Adaptive competition for motion vector prediction in multi-view video coding , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).