Coding of Focused Plenoptic Contents by Displacement Intra Prediction

A light field is commonly described by a two-plane representation with four dimensions. Refocused 3D contents can be rendered from light field images. A method for capturing these images is using cameras with microlens arrays. A dense sampling of the light field results in large amounts of redundant data. Therefore, an efficient compression is vital for a practical use of these data. In this paper, we propose a displacement intra prediction scheme with a maximum of two hypotheses for the compression of plenoptic contents from focused plenoptic cameras. The proposed scheme is further implemented into High Efficiency Video Coding (HEVC). The work is aiming at efficiently coding plenoptic captured contents without knowing underlying camera geometries. In addition, the theoretical analysis of the displacement intra prediction for plenoptic images is explained; the relationship between the compressed captured images and their rendered quality is also analyzed. Evaluation results show that plenoptic contents can be efficiently compressed by the proposed scheme. Bit rate reduction up to 60% over HEVC is obtained for plenoptic images, and more than 30% is achieved for the tested video sequences.

[1]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[2]  Ashok Veeraraghavan,et al.  Axial light field for curved mirrors: Reflect your perspective, widen your view , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Yun Li,et al.  Efficient intra prediction scheme for light field image compression , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Bernd Girod,et al.  Rate-distortion analysis for light field coding and streaming , 2006, Signal Process. Image Commun..

[5]  Bernd Girod,et al.  Light field compression using disparity-compensated lifting and shape adaptation , 2006, IEEE Transactions on Image Processing.

[6]  Marcus A. Magnor,et al.  Model-based coding of multiviewpoint imagery , 2000, Visual Communications and Image Processing.

[7]  Pier Luigi Dragotti,et al.  Distributed Compression of Multi-View Images using a Geometrical Coding Approach , 2007, 2007 IEEE International Conference on Image Processing.

[8]  Yongdong Zhang,et al.  High Efficiency Video Coding: High Efficiency Video Coding , 2014 .

[9]  Luís Ducla Soares,et al.  New HEVC prediction modes for 3D holoscopic video coding , 2012, 2012 19th IEEE International Conference on Image Processing.

[10]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[11]  David L. Neuhoff,et al.  Quantization , 2022, IEEE Trans. Inf. Theory.

[12]  Thomas Wiegand,et al.  Video codec incorporating block-based multihypothesis motion-compensated prediction , 2000, Visual Communications and Image Processing.

[13]  Luís Ducla Soares,et al.  Influence of self-similarity on 3D holoscopic video coding performance , 2012, WebMedia.

[14]  Jeong-Hoon Park,et al.  Block Partitioning Structure in the HEVC Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Luís Ducla Soares,et al.  Acquisition, processing and coding of 3D holoscopic content for immersive video systems , 2013, 2013 3DTV Vision Beyond Depth (3DTV-CON).

[16]  G. Lippmann Epreuves reversibles donnant la sensation du relief , 1908 .

[17]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  F. Bossen,et al.  Common test conditions and software reference configurations , 2010 .

[19]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, CVPR 2004.

[20]  Bernd Girod,et al.  Distributed Compression of Light Fields , 2003 .

[21]  Dan Lelescu,et al.  Representation and coding of light field data , 2004, Graph. Model..

[22]  Qionghai Dai,et al.  Data compression of light field using wavelet packet , 2004, ICME.

[23]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[24]  Marcus A. Magnor,et al.  Data compression for light-field rendering , 2000, IEEE Trans. Circuits Syst. Video Technol..

[25]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[26]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[27]  Zhan Yu,et al.  Lytro camera technology: theory, algorithms, performance analysis , 2013, Electronic Imaging.

[28]  Vladan Velisavljevic,et al.  Multiview image compression using a layer-based representation , 2010, 2010 IEEE International Conference on Image Processing.

[29]  Andrew Lumsdaine,et al.  Focused plenoptic camera and rendering , 2010, J. Electronic Imaging.

[30]  David Flynn,et al.  HEVC Complexity and Implementation Analysis , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Zhan Yu,et al.  Plenoptic depth map in the case of occlusions , 2013, Electronic Imaging.

[32]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[33]  Shinjini Kundu,et al.  Light field compression using homography and 2D warping , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Luís Ducla Soares,et al.  3D Holoscopic Video Coding Based on HEVC with Improved Spatial and Temporal Prediction , 2013 .

[35]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[36]  Bernd Girod,et al.  Efficiency analysis of multihypothesis motion-compensated prediction for video coding , 2000, IEEE Trans. Image Process..

[37]  Katsushi Ikeuchi,et al.  Eigen-texture method: Appearance compression based on 3D model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[38]  Markus Flierl,et al.  Rate-constrained multihypothesis prediction for motion-compensated video compression , 2002, IEEE Trans. Circuits Syst. Video Technol..

[39]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[40]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[41]  Sheng-kui Dai,et al.  An analysis of TZSearch algorithm in JMVC , 2010, The 2010 International Conference on Green Circuits and Systems.