Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution

Deep learning algorithms have demonstrated state-of-the-art performance in various tasks of image restoration. This was made possible through the ability of CNNs to learn from large exemplar sets. However, the latter becomes an issue for hyperspectral image processing where datasets commonly consist of just a few images. In this work, we propose a new approach to denoising, inpainting, and super-resolution of hyperspectral image data using intrinsic properties of a CNN without any training. The performance of the given algorithm is shown to be comparable to the performance of trained networks, while its application is not restricted by the availability of training data. This work is an extension of original "deep prior" algorithm to hyperspectral imaging domain and 3D-convolutional networks.

[1]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[2]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[3]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Gozde Bozdagi Akar,et al.  A MAP-Based Approach for Hyperspectral Imagery Super-Resolution , 2018, IEEE Transactions on Image Processing.

[5]  M. F. Baumgardner,et al.  220 Band AVIRIS Hyperspectral Image Data Set: June 12, 1992 Indian Pine Test Site 3 , 2015 .

[6]  Jon Atli Benediktsson,et al.  Advances in Spectral-Spatial Classification of Hyperspectral Images , 2013, Proceedings of the IEEE.

[7]  Salah Bourennane,et al.  Denoising and Dimensionality Reduction Using Multilinear Tools for Hyperspectral Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[8]  Hairong Qi,et al.  Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Jocelyn Chanussot,et al.  Hyperspectral image inpainting based on collaborative total variation , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[10]  Laurent Jacques,et al.  Generalized inpainting method for hyperspectral image acquisition , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[11]  Qiang Zhang,et al.  Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[13]  Liangpei Zhang,et al.  Compressed Sensing-Based Inpainting of Aqua Moderate Resolution Imaging Spectroradiometer Band 6 Using Adaptive Spectrum-Weighted Sparse Bayesian Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[16]  Shen-En Qian,et al.  Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Liangpei Zhang,et al.  Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[18]  M. Körner,et al.  SINGLE-IMAGE SUPER RESOLUTION FOR MULTISPECTRAL REMOTE SENSING DATA USING CONVOLUTIONAL NEURAL NETWORKS , 2016 .

[19]  Lianru Gao,et al.  Hyperspectral image inpainting based on low-rank representation: A case study on Tiangong-1 data , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[20]  Qian Du,et al.  Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network , 2017, Remote. Sens..

[21]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[23]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Shutao Li,et al.  Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[27]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[28]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[29]  Zhenwei Shi,et al.  Super-Resolution for Remote Sensing Images via Local–Global Combined Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[31]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[32]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[33]  A. Bertozzi,et al.  Unconditionally stable schemes for higher order inpainting , 2011 .

[34]  Xiangtao Zheng,et al.  Hyperspectral Image Superresolution by Transfer Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Qian Wang,et al.  Hyperspectral Imagery Denoising Based on Oblique Subspace Projection , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[36]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[37]  Xiai Chen,et al.  Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization , 2017, Remote. Sens..

[38]  José M. Bioucas-Dias,et al.  Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[39]  Frédéric Bousefsaf,et al.  Image completion using multispectral imaging , 2018, IET Image Process..

[40]  Jie Li,et al.  Hyperspectral Image Super-Resolution by Spectral Mixture Analysis and Spatial–Spectral Group Sparsity , 2016, IEEE Geoscience and Remote Sensing Letters.

[41]  Jianhong Shen,et al.  Digital inpainting based on the Mumford–Shah–Euler image model , 2002, European Journal of Applied Mathematics.