Three-Dimensional Video Contents Exploitation in Depth Camera-Based Hybrid Camera System

Video-plus-depth is an image sequence of synchronized color and depth images. As importance of video-plus-depth increases as an essential part of the next-generation multimedia applications, it is crucial to estimate accurate depth information from a real scene and to find a practical framework to use the immersive video in industry. In this chapter, we introduce a hybrid camera system composed of a stereoscopic camera and a time-of-flight depth camera to generate high-quality and high-resolution video-plus-depth. We also handle a hierarchical decomposition method of depth images to render a dynamic 3D scene represented by video-plus-depth rapidly. Finally, we present a method to generate streamable 3D video contents based on video-plus-depth and computer graphic models in the MPEG-4 multimedia framework. The MPEG-4-based 3D video contents can support a variety of user-friendly interactions, such as free viewpoint changing and free composition with computer graphic images.

[1]  Jongeun Cha,et al.  Depth Video Enhancement for Haptic Interaction Using a Smooth Surface Reconstruction , 2006, IEICE Trans. Inf. Syst..

[2]  Sanjit K. Mitra,et al.  Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard , 1996, IEEE Trans. Circuits Syst. Video Technol..

[3]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[4]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[5]  N. Atzpadin,et al.  Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability , 2007, Signal Process. Image Commun..

[6]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.

[7]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[8]  C. Fehn.  A 3D-TV system based on video plus depth information , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[9]  Yo-Sung Ho,et al.  3D video player system with haptic interaction based on depth image-based representation , 2006, IEEE Transactions on Consumer Electronics.

[10]  Fernando Pereira.  MPEG-4: Why, what, how and when? , 2000, Signal Process. Image Commun..

[11]  Yo-Sung Ho,et al.  Advances in Multimedia Information Processing - PCM 2005, 6th Pacific-Rim Conference on Multimedia, Jeju Island, Korea, November 13-16, 2005, Proceedings, Part I , 2005, PCM.

[12]  P.H.N. de With,et al.  Depth-Image Representation Employing Meshes for Intermediate-View Rendering and Coding , 2007, 2007 3DTV Conference.

[13]  Luc Van Gool,et al.  ATTEST: Advanced Three-dimensional Television System Technologies , 2002 .

[14]  Yo-Sung Ho,et al.  Dynamic 3D human actor generation method using a time-of-flight depth camera , 2008, IEEE Transactions on Consumer Electronics.

[15]  Yo-Sung Ho,et al.  Hierarchical Decomposition of Depth Map Sequences for Representation of Three-Dimensional Dynamic Scenes , 2007, IEICE Trans. Inf. Syst..

[16]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[17]  Yo-Sung Ho,et al.  Three-dimensional Video Generation for Realistic Broadcasting Services , 2008 .

[18]  Yuko Yamanouchi,et al.  High Definition Three-Dimension Camera (HDTV Axi-vision Camera) and its Application for Image Synthesis , 2002 .

[19]  Yo-Sung Ho,et al.  Generation of ROI Enhanced Depth Maps Using Stereoscopic Cameras and a Depth Camera , 2008, IEEE Transactions on Broadcasting.

[20]  In Kyu Park,et al.  Depth image-based representation and compression for static and animated 3-D objects , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Seung-man Kim,et al.  Depth Image Processing Technique for Representing Human Actors in 3DTV using Single Depth Camera , 2007, 2007 3DTV Conference.

[22]  Ian Oakley,et al.  Client System for Realistic Broadcasting: A First Prototype , 2005, PCM.

[23]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  Seung-Uk Yoon,et al.  Multiple Color and Depth Video Coding Using a Hierarchical Representation , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Markus H. Gross,et al.  Point-sampled 3D video of real-world scenes , 2007, Signal Process. Image Commun..

[27]  M. Kawakita,et al.  HDTV Axi-Vision Camera , 2002 .