Consistent Matting for Light Field Images

We present a new image matting algorithm to extract consistent alpha mattes across sub-images of a light field image. Instead of matting each sub-image individually, our approach utilizes the epipolar plane image (EPI) to construct comprehensive foreground and background sample sets across the sub-images without missing a true sample. The sample sets represent all color variation of foreground and background in a light field image, and the optimal alpha matte is obtained by choosing the best combination of foreground and background samples that minimizes the linear composite error subject to the EPI correspondence constraint. To further preserve consistency of the estimated alpha mattes across different sub-images, we impose a smoothness constraint along the EPI of alpha mattes. In experimental evaluations, we have created a dataset where the ground truth alpha mattes of light field images were obtained by using the blue screen technique. A variety of experiments show that our proposed algorithm produces both visually and quantitatively high-quality matting results for light field images.

[1]  Carsten Rother,et al.  Improving Color Modeling for Alpha Matting , 2008, BMVC.

[2]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Sven Wanner,et al.  Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Sven Wanner,et al.  The Variational Structure of Disparity and Regularization of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH '06.

[6]  Ying Wu,et al.  Nonlocal matting , 2011, CVPR 2011.

[7]  Tom E. Bishop,et al.  Plenoptic depth estimation from multiple aliased views , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[8]  Yu-Wing Tai,et al.  Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Yu-Wing Tai,et al.  Video Matting Using Multi-frame Nonlocal Matting Laplacian , 2012, ECCV.

[10]  Jue Wang,et al.  A perceptually motivated online benchmark for image matting , 2009, CVPR.

[11]  Yuanjie Zheng,et al.  Learning based digital matting , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[13]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[14]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[15]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[17]  Jian Sun,et al.  A global sampling method for alpha matting , 2011, CVPR 2011.

[18]  Jitendra Malik,et al.  Depth from Combining Defocus and Correspondence Using Light-Field Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Leonard T. Bruton,et al.  Gradient-based depth estimation from 4D light fields , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[20]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[21]  Chi-Keung Tang,et al.  KNN Matting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Jian Sun,et al.  Fast matting using large kernel matting Laplacian matrices , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[24]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Sven Wanner,et al.  Reconstructing Reflective and Transparent Surfaces from Epipolar Plane Images , 2013, GCPR.

[26]  Deepu Rajan,et al.  Weighted color and texture sample selection for image matting , 2012, CVPR.

[27]  Michael S. Brown,et al.  Motion Regularization for Matting Motion Blurred Objects , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Qinping Zhao,et al.  Image Matting with Local and Nonlocal Smooth Priors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Deepu Rajan,et al.  Improving Image Matting Using Comprehensive Sampling Sets , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Richard Szeliski,et al.  Extracting layers and analyzing their specular properties using epipolar-plane-image analysis , 2005, Comput. Vis. Image Underst..

[31]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .