Light-Field Image Superresolution Using a Combined Deep CNN Based on EPI

Light-field cameras can capture spatial and angular information of light with a single exposure, but they suffer from low spatial resolution that limits their performance in practical use. Superresolution (SR) methods have been used to improve spatial resolution of light-field images, but most of them do not take full advantages of the particular structure of the light field. In this letter, we present an SR method to obtain light-field images with high quality and geometric consistency via a combined deep convolutional neural networks (CNNs) framework. The spatial resolution of subaperture images is enhanced separately by a single-image superresolution deep CNN. Then, an epipolar plane image enhancement deep CNN is proposed to restore the geometric consistency of these images. Experimental results show that our method achieves the state-of-the-art performance on both quantitative and qualitative evaluations.

[1]  Lijuan Su,et al.  Multiframe super resolution reconstruction method based on light field angular images , 2017 .

[2]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Bastian Goldlücke,et al.  A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields , 2016, ACCV.

[6]  Sven Wanner,et al.  Datasets and Benchmarks for Densely Sampled 4D Light Fields , 2013, VMV.

[7]  Yu-Wing Tai,et al.  Consistent Matting for Light Field Images , 2014, ECCV.

[8]  In-So Kweon,et al.  Light-Field Image Super-Resolution Using Convolutional Neural Network , 2017, IEEE Signal Processing Letters.

[9]  Pascal Frossard,et al.  Geometry-Consistent Light Field Super-Resolution via Graph-Based Regularization , 2017, IEEE Transactions on Image Processing.

[10]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Reuben A. Farrugia,et al.  Light Field Super-Resolution Using a Low-Rank Prior and Deep Convolutional Neural Networks , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Atsushi Shimada,et al.  TransCut: Transparent Object Segmentation from a Light-Field Image , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Touradj Ebrahimi,et al.  New Light Field Image Dataset , 2016, QoMEX 2016.

[16]  Chao Li,et al.  Robust depth estimation for light field via spinning parallelogram operator , 2016, Comput. Vis. Image Underst..

[17]  Christine Guillemot,et al.  Light Field Inpainting Propagation via Low Rank Matrix Completion , 2018, IEEE Transactions on Image Processing.

[18]  Toru Iwane.  Light field camera and integral 3D display: 3D image reconstruction based on lightfield data , 2014, 2014 13th Workshop on Information Optics (WIO).

[19]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Sven Wanner,et al.  Spatial and Angular Variational Super-Resolution of 4D Light Fields , 2012, ECCV.

[21]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[22]  Qionghai Dai,et al.  The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring , 2017, IEEE Transactions on Visualization and Computer Graphics.

[23]  A. Lumsdaine,et al.  Superresolution with Plenoptic Camera 2 . 0 , 2009 .

[24]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Ashok Veeraraghavan,et al.  Improving resolution and depth-of-field of light field cameras using a hybrid imaging system , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[26]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[27]  Ashok Veeraraghavan,et al.  Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[28]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.