Super-Resolution in Plenoptic Cameras by the Integration of Depth from Focus and Stereo

Depth range cameras like the plenoptic camera are a promising solution for 3DTV. The use of plenoptic cameras for 3DTV has some benefits with respect to 3D capture systems based on camera arrays since there is no need of geometric and color calibration or frame synchronization. This paper presents a new technique to simultaneously recover the 3D depth and all-in-focus images from a plenoptic camera integrating depth from focus and stereo methods. The standard approach to 3D reconstruction using plenoptic cameras presented some drawbacks since its spatio-angular sampling leads to a low spatial resolution. This new technique theoretically increases the spatial resolution of plenoptic cameras to its full sensor size. From a practical point of view, in typical scenes we are able to increase 49 times the spatial resolution of the standard approach and 2 times the spatial resolution of previous super-resolution techniques. Depth resolution is also increased 2 times. The resolution obtained can be used as the input of some autostereoscopic displays on the market and shows the feasibility of a plenoptic approach to 3DTV.

[1]  Gavin C. Cawley,et al.  Gene Selection in Cancer Classification using Sparse Logistic Regression with Bayesian Regularisation , 2006 .

[2]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Peter Lawrence,et al.  An Investigation of Methods for Determining Depth from Focus , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[5]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields , 2006, ECCV.

[6]  José Gil Marichal-Hernández,et al.  The Discrete Focal Stack Transform , 2008, 2008 16th European Signal Processing Conference.

[7]  Zhongliang Jing,et al.  Evaluation of focus measures in multi-focus image fusion , 2007, Pattern Recognit. Lett..

[8]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[9]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[10]  A. Murat Tekalp,et al.  Transport Methods in 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Gilbert Strang,et al.  The Discrete Cosine Transform , 1999, SIAM Rev..

[12]  Thomas P. Minka,et al.  Algorithms for maximum-likelihood logistic regression , 2003 .

[13]  Tom E. Bishop,et al.  Light field superresolution , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[14]  J. P. Luke,et al.  Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[15]  Kim Randall.  The Focus , 1951 .

[16]  Levent Onural,et al.  Television in 3-D: What Are the Prospects? , 2007 .