A Nonsmooth Graph-Based Approach to Light Field Super-Resolution

We propose a new super-resolution algorithm tailored for light field cameras, which suffer by design from a limited spatial resolution. In particular, we cast light field super-resolution into an optimization problem, where the particular structure of the light field data is captured by a nonsmooth graph-based regularizer, and where all the light field views are super-resolved jointly. Our experiments show that the proposed method compares favorably to the state-of-the-art light field super-resolution algorithms in terms of PSNR and visual quality. In particular, the nonsmooth graph-based regularizer leads to sharper images while preserving fine details.

[1]  Pascal Frossard,et al.  Geometry-Consistent Light Field Super-Resolution via Graph-Based Regularization , 2017, IEEE Transactions on Image Processing.

[2]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[3]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[4]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[5]  Sven Wanner,et al.  Spatial and Angular Variational Super-Resolution of 4D Light Fields , 2012, ECCV.

[6]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[7]  Sven Wanner,et al.  Datasets and Benchmarks for Densely Sampled 4D Light Fields , 2013, VMV.

[8]  Aline Roumy,et al.  Single-Image Super-Resolution via Linear Mapping of Interpolated Self-Examples , 2014, IEEE Transactions on Image Processing.

[9]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[10]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[12]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[13]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[14]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[15]  Pascal Frossard,et al.  Graph-based light field super-resolution , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[16]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[17]  I. M. Otivation.  Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems , 2018 .

[18]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[19]  Ashok Veeraraghavan,et al.  Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Jean-François Aujol,et al.  Image Zoom Completion , 2016, IEEE Transactions on Image Processing.

[21]  Andreas K. Maier,et al.  Robust Multiframe Super-Resolution Employing Iteratively Re-Weighted Minimization , 2016, IEEE Transactions on Computational Imaging.

[22]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.