Object-based depth image-based rendering for a three-dimensional video system by color-correction optimization

Three-dimensional (3-D) video technologies are becoming increasingly popular because they can provide high quality and immersive experience to end users. Depth image-based rendering (DIBR) is a key technology in 3-D video systems due to its low bandwidth cost as well as the arbitrary rendering viewpoint. We propose an object-based DIBR method by color-correction optimization. The proposed method first performs temporal consistent rendering to reduce the rendering complexity. Then, by segmenting the depth map into foreground and background, the object-based scalable rendering is performed to improve the rendering quality and reduce the rendering complexity. Finally, the rendered virtual view is further optimized by color-correction operation. Experimental results show that, compared to the results without the above optimization operations, the proposed method can reduce >40% computational complexity while maintaining high rendering quality.

[1]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[2]  Toshiaki Fujii,et al.  View Generation with 3D Warping Using Depth Information for FTV , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[3]  Qingming Huang,et al.  Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model , 2009, Signal Process. Image Commun..

[4]  Seung-Uk Yoon,et al.  Multiple Color and Depth Video Coding Using a Hierarchical Representation , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Sehoon Yea,et al.  View synthesis prediction for multiview video coding , 2009, Signal Process. Image Commun..

[6]  Yo-Sung Ho,et al.  Virtual view synthesis method and self‐evaluation metrics for free viewpoint television and 3D video , 2010, Int. J. Imaging Syst. Technol..

[7]  CHRISTOPH FEHN,et al.  Interactive 3-DTV-Concepts and Key Technologies , 2006, Proceedings of the IEEE.

[8]  Antonio Ortega,et al.  Improving view rendering quality and coding efficiency by suppressing compression artifacts in depth-image coding , 2009, Electronic Imaging.

[9]  Ken Chen,et al.  Depth perceptual region-of-interest based multiview video coding , 2010, J. Vis. Commun. Image Represent..

[10]  P. J. Narayanan,et al.  Depth Images: Representations and Real-Time Rendering , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[11]  Jiang Gangyi,et al.  Multi-view video color correction using dynamic programming , 2008 .

[12]  Christophe Tillier,et al.  Distance Dependent Depth Filtering in 3D Warping for 3DTV , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[13]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[14]  Peter H. N. de With,et al.  System architecture for free-viewpoint video and 3D-TV , 2008, IEEE Transactions on Consumer Electronics.

[15]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[16]  Warnakulasuriya Anil Chandana Fernando,et al.  3D video assessment with Just Noticeable Difference in Depth evaluation , 2010, 2010 IEEE International Conference on Image Processing.

[17]  Harry Shum,et al.  An Object-Based Approach to Image/Video-Based Synthesis and Processing for 3-D and Multiview Televisions , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Tsuhan Chen,et al.  A survey on image-based rendering - representation, sampling and compression , 2004, Signal Process. Image Commun..

[19]  Liang-Gee Chen,et al.  Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[20]  Zhengyou Zhang,et al.  Camera calibration with one-dimensional objects , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Aljoscha Smolic,et al.  View Synthesis for Advanced 3D Video Systems , 2008, EURASIP J. Image Video Process..

[22]  Mei Yu,et al.  Fast color correction for multi-view video by modeling spatio-temporal variation , 2010, J. Vis. Commun. Image Represent..

[23]  André Kaup,et al.  Histogram-Based Prefiltering for Luminance and Chrominance Compensation of Multiview Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Aljoscha Smolic,et al.  Efficient Prediction Structures for Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  张云.  Object-based depth image-based rendering for a three-dimensional video system by color-correction optimization, Optical Engineering , 2011 .

[26]  Toshiaki Fujii,et al.  Multiview Video Coding Using View Interpolation and Color Correction , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Weisi Lin,et al.  Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation , 2005, IEEE Transactions on Image Processing.