Rendering-Oriented Decoding for a Distributed Multiview Coding System Using a Coset Code

This paper discusses a system in which multiview images are captured and encoded in a distributed fashion and a viewer synthesizes a novel image from this data. We present an efficient method for such a system that combines decoding and rendering processes in order to directly synthesize the novel image without having to reconstruct all the input images. Our method jointly performs disparity compensation in the decoding process and geometry estimation in the rendering process, because they are essentially equivalent if the camera parameters for the input images are known. Our method keeps both encoder and decoder complexity as low as that of a conventional intracoding method, while attaining better coding performance owing to the interimage decoding. We validate our method by evaluating the coding performance and the processing time for decoding and rendering in experiments.

[1]  Wen Gao,et al.  Distributed Multiview Video Coding , 2005 .

[2]  Toshiaki Fujii,et al.  Multiview Video Coding Using View Interpolation and Color Correction , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[4]  Roberto Rinaldo,et al.  Wavelet Domain Distributed Coding for Video , 2006, 2006 International Conference on Image Processing.

[5]  Tsuhan Chen,et al.  A Self-Reconfigurable Camera Array , 2004, Rendering Techniques.

[6]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[7]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[8]  Narendra Ahuja,et al.  Compression of lightfield rendered images using coset codes , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[9]  Antonio Ortega,et al.  New Coding Tools for Illumination and Focus Mismatch Compensation in Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Takeshi Naemura,et al.  Layered light-field rendering with focus measurement , 2006, Signal Process. Image Commun..

[11]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[12]  Y. Taguchi,et al.  Real-Time All-in-Focus Video-Based Rendering Using A Network Camera Array , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[13]  Toshiaki Fujii,et al.  Multipoint Measuring System for Video and Sound - 100-camera and microphone system , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[14]  Takeshi Naemura,et al.  Real-time video-based rendering for augmented spatial communication , 1998, Electronic Imaging.

[15]  Takeshi Naemura,et al.  Rendering-Oriented Decoding for Distributed Multi-View Coding System , 2007, 2007 IEEE International Conference on Image Processing.

[16]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[17]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[18]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[19]  Hans-Peter Seidel,et al.  On‐the‐Fly Processing of Generalized Lumigraphs , 2001, Comput. Graph. Forum.

[20]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[21]  Bernd Girod,et al.  Wyner-Ziv coding of light fields for random access , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[22]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.