Light Field Super-Resolution via LFBM5D Sparse Coding

In this paper, we propose a spatial super-resolution method for light fields, which combines the SR-BM3D single image super-resolution filter and the recently introduced LFBM5D light field denoising filter. The proposed algorithm iteratively alternates between an LFBM5D filtering step and a back-projection step. The LFBM5D filter creates disparity compensated 4D patches which are then stacked together with similar 4D patches along a 5th dimension. The 5D patches are then filtered in the 5D transform domain to enforce a sparse coding of the high-resolution light field, which is a powerful prior to solve the ill-posed super-resolution problem. The back-projection step then impose the consistency between the known low-resolution light field and the-high resolution estimate. We further improve this step by using image guided filtering to remove ringing artifacts. Results show that significant improvement can be achieved compared to state-of-the-art methods, for both light fields captured with a lenslet camera or a gantry.

[1]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[2]  Aljoscha Smolic,et al.  Light field denoising by sparse 5D transform domain collaborative filtering , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[3]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Lei Zhang,et al.  Joint Learning of Multiple Regressors for Single Image Super-Resolution , 2016, IEEE Signal Processing Letters.

[5]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[6]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[7]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Pascal Frossard,et al.  Graph-based light field super-resolution , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[9]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[10]  Christine Guillemot,et al.  White lenslet image guided demosaicing for plenoptic cameras , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[11]  Reuben A. Farrugia,et al.  Super Resolution of Light Field Images Using Linear Subspace Projection of Patch-Volumes , 2017, IEEE Journal of Selected Topics in Signal Processing.

[12]  In-So Kweon,et al.  Learning a Deep Convolutional Network for Light-Field Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[13]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[14]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jae-Seok Choi,et al.  Single Image Super-Resolution Using Global Regression Based on Multiple Local Linear Mappings , 2017, IEEE Transactions on Image Processing.

[16]  Ashok Veeraraghavan,et al.  Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[17]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[18]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[19]  Chi-Keung Tang,et al.  Perceptually-Inspired and Edge-Directed Color Image Super-Resolution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Touradj Ebrahimi,et al.  New Light Field Image Dataset , 2016, QoMEX 2016.

[21]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[22]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[23]  Vladimir Katkovnik,et al.  SINGLE IMAGE SUPER-RESOLUTION VIA BM 3 D SPARSE CODING , 2015 .

[24]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[25]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[26]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.