Graph-based representation and coding of 3D images for interactive multiview navigation

Instead of lossily coding depth images resulting in undesirable geometric distortion, graph-based representation (GBR) describes disparity information as a graph with a controllable accuracy. In this paper, we propose a more compact graphical representation called GBR-plus to code both disparity and color information of a target view given a reference view. Specifically, first we differentiate between disocclusion holes (occluded spatial regions in the reference view) and rounding holes (insufficiently sampled regions in the reference view) in the synthesized target view, so that the decoder can optionally complete rounding holes via signal interpolation without coding overhead. Second, we use a compact graphical representation to delimit disparity-shifted boundaries of objects in the target view, which is coded losslessly. Finally, color pixels in disocclusion holes are predicted using adjacent background pixels as predictors, and prediction residuals in a local neighborhood are coded using Graph Fourier Transform (GFT). Experimental results show that GBR-plus outperforms previous GBR, and has comparable performance as HEVC at mid to high bitrates with lower encoder complexity.

[1]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[2]  Antonio Ortega,et al.  Transform domain sparsification of depth maps using iterative quadratic programming , 2011, 2011 18th IEEE International Conference on Image Processing.

[3]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[4]  Ronald Arps,et al.  JBIG2-the ultimate bi-level image coding standard , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[5]  Gene Cheung,et al.  Arbitrarily Shaped Motion Prediction for Depth Video Compression Using Arithmetic Edge Coding , 2014, IEEE Transactions on Image Processing.

[6]  Antonio Ortega,et al.  Sparse representation of depth maps for efficient transform coding , 2010, 28th Picture Coding Symposium.

[7]  Minh N. Do,et al.  Wavelet-Based Joint Estimation and Encoding of Depth-Image-Based Representations for Free-Viewpoint Rendering , 2008, IEEE Transactions on Image Processing.

[8]  Antonio Ortega,et al.  On Dependent Bit Allocation for Multiview Image Coding With Depth-Image-Based Rendering , 2011, IEEE Transactions on Image Processing.

[9]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[10]  Pascal Frossard,et al.  Re-sampling and interpolation of DIBR-synthesized images using graph-signal smoothness prior , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[11]  Oscar C. Au,et al.  Optimal graph laplacian regularization for natural image denoising , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Gene Cheung,et al.  Delay-Cognizant Interactive Streaming of Multiview Video With Free Viewpoint Synthesis , 2012, IEEE Transactions on Multimedia.

[13]  Antonio Ortega,et al.  Depth map coding using graph based transform and transform domain sparsification , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[14]  Antonio Ortega,et al.  Depth map coding with distortion estimation of rendered view , 2010, Electronic Imaging.

[15]  Thomas Maugey,et al.  Graph-Based Representation for Multiview Image Geometry , 2015, IEEE Transactions on Image Processing.

[16]  Oscar C. Au,et al.  Multiresolution Graph Fourier Transform for Compression of Piecewise Smooth Images , 2015, IEEE Transactions on Image Processing.

[17]  Antonio Ortega,et al.  Interactive Streaming of Stored Multiview Video Using Redundant Frame Structures , 2011, IEEE Transactions on Image Processing.

[18]  Jaejoon Lee,et al.  Edge-adaptive transforms for efficient depth map coding , 2010, 28th Picture Coding Symposium.

[19]  Dong Tian,et al.  View synthesis techniques for 3D video , 2009, Optical Engineering + Applications.