Stereoscopic video coding and disparity estimation for low bitrate applications based on MPEG-4 multiple auxiliary components

Using an MPEG-4 MAC (multiple auxiliary component) system is a good way to encode stereoscopic video with existing standard CODECs, especially when it comes to low bitrate applications. In this paper, we discuss the properties and problems of MAC systems when encoding stereoscopic video, and propose an MAC-based stereoscopic video coder and disparity estimation scheme to solve those problems. We used a reconstructed disparity map during the disparity compensation process and took that disparity map into account while estimating the base-view sequence motion vectors. Moreover, we proposed a search range finding and illumination imbalance decision system. We also proposed a block-based disparity map regularization process as well as block splitting in the object boundary and occlusion regions (to reduce the number of bits to encode in both the disparity map and the residual image). Last, we compensated for the imbalance between two cameras with a novel system that used MAC characteristics. Experimental results indicate that the proposed MAC system outperformed conventional stereo coding systems by a maximum of 3.5dB in terms of the PSNR and 10-18% in terms of bitsaving, especially in low bitrate applications.

[1]  Soo In Lee,et al.  Disparity-Compensated Stereoscopic Video Coding Using the MAC in MPEG-4 , 2005 .

[2]  S. Cho,et al.  Adaptive Local Illumination Change Compensation Method for H.264/AVC-Based Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[4]  Ajay Luthra,et al.  MPEG-2 multiview profile and its application in 3D TV , 1997, Electronic Imaging.

[5]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[6]  Andrea Giachetti,et al.  Matching techniques to compute image motion , 2000, Image Vis. Comput..

[7]  Jens-Rainer Ohm - 1-Incomplete 3 D Representation of Video Objects for Multiview Applications , 1999 .

[8]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Larry F. Hodges,et al.  Geometric Considerations for Stereoscopic Virtual Environments , 1993, Presence: Teleoperators & Virtual Environments.

[11]  Jens-Rainer Ohm,et al.  Incomplete 3D for Multiview Representation and Synthesis of Video Objects , 1998, ECMAST.

[12]  Mi-Hyun Kim,et al.  Edge-preserving directional regularization technique for disparity estimation of stereoscopic images , 1999, IEEE Trans. Consumer Electron..

[13]  Antonio Ortega,et al.  New Coding Tools for Illumination and Focus Mismatch Compensation in Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.