An Explicit and Scene-Adapted Definition of Convex Self-Similarity Prior With Application to Unsupervised Sentinel-2 Super-Resolution

Sentinel-2 satellite, launched by the European Space Agency, plays a critical role in various Earth observation missions. However, the spatial resolutions of Sentinel-2 images are different across its spectral bands, including four bands with a resolution of 10 m, six bands with a resolution of 20 m, and three bands with a resolution of 60 m. To facilitate the effectiveness of analyzing these images, super-resolving of the low-/medium-resolution bands to a higher resolution is desired. As in any image restoration inverse problems, we exploit image self-similarity, a commonly observed property in natural images, which underlies the state-of-the-art techniques, e.g., in image denoising. However, the design of self-similarity priors in nondiagonal inverse problems is challenging; often, a denoiser based on self-similarity is plugged into the iterations of an algorithm, without a guarantee of convergence in general. In this article, for the first time, we introduce a convex and scene-adapted regularizer built explicitly on a self-similarity graph directly learned from the Sentinel-2 images. We then develop a fast algorithm, termed Sentinel-2 super-resolution via scene-adapted self-similarity (SSSS). We experimentally show the superiority of SSSS over four commonly observed scenes, indicating the potential usage of our convex self-similarity regularization in other imaging inverse problems.

[1]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[2]  Anand Rangarajan,et al.  Image Denoising Using the Higher Order Singular Value Decomposition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[4]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[5]  John R. Jensen,et al.  Introductory Digital Image Processing: A Remote Sensing Perspective , 1986 .

[6]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[7]  Mario Chica-Olmo,et al.  Image fusion by spatially adaptive filtering using downscaling cokriging , 2011 .

[8]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[10]  Wenzhong Shi,et al.  Fusion of Sentinel-2 images , 2016 .

[11]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[13]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[14]  Andreas T. Ernst,et al.  ICE: a statistical approach to identifying endmembers in hyperspectral images , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[16]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[17]  Simon J. Godsill,et al.  R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation , 2016, IEEE Signal Processing Letters.

[18]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[19]  Lorenzo Bruzzone,et al.  A hierarchical approach to superresolution of multispectral images with different spatial resolutions , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[20]  Shutao Li,et al.  Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization , 2018, IEEE Transactions on Image Processing.

[21]  Phaedon C. Kyriakidis,et al.  Fusion of MODIS Images Using Kriging With External Drift , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[22]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[23]  Mario Chica-Olmo,et al.  Downscaling Cokriging for Super-Resolution Mapping of Continua in Remotely Sensed Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Wei-Chiang Li,et al.  Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications , 2017 .

[25]  José M. Bioucas-Dias,et al.  An Alternating Direction Algorithm for Total Variation Reconstruction of Distributed Parameters , 2012, IEEE Transactions on Image Processing.

[26]  Johannes R. Sveinsson,et al.  Sentinel-2 Image Fusion Using a Deep Residual Network , 2018, Remote. Sens..

[27]  Johannes R. Sveinsson,et al.  Sentinel-2 Sharpening Using a Reduced-Rank Method , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Subhasis Chaudhuri,et al.  Hyperspectral Image Fusion , 2013, Springer New York.

[29]  Eli Shechtman,et al.  Matching Local Self-Similarities across Images and Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Sergey V. Samsonov,et al.  A review of the status of satellite remote sensing and image processing techniques for mapping natural hazards and disasters , 2009 .

[31]  Chia-Hsiang Lin,et al.  A Convex Optimization-Based Coupled Nonnegative Matrix Factorization Algorithm for Hyperspectral and Multispectral Data Fusion , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[33]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[34]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[35]  Wing-Kin Ma,et al.  Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach , 2018, IEEE Transactions on Signal Processing.

[36]  Matthias Drusch,et al.  Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services , 2012 .

[37]  Yue Wang,et al.  Maximum Volume Inscribed Ellipsoid: A New Simplex-Structured Matrix Factorization Framework via Facet Enumeration and Convex Optimization , 2017, SIAM J. Imaging Sci..

[38]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[39]  Marc Niezette,et al.  Sentinel-2 Level 2A Prototype Processor: Architecture, Algorithms And First Results , 2013 .

[40]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[41]  José M. Bioucas-Dias,et al.  Sharpening Hyperspectral Images Using Plug-and-Play Priors , 2017, LVA/ICA.

[42]  Wei-Chiang Li,et al.  Identifiability of the Simplex Volume Minimization Criterion for Blind Hyperspectral Unmixing: The No-Pure-Pixel Case , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Konrad Schindler,et al.  Super-Resolution of Multispectral Multiresolution Images from a Single Sensor , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Yasushi Yamaguchi,et al.  Overview of Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) , 1998, IEEE Trans. Geosci. Remote. Sens..

[45]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[46]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[47]  Konrad Schindler,et al.  Super-Resolution of Sentinel-2 Images: Learning a Globally Applicable Deep Neural Network , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[48]  Wei-Chiang Li,et al.  On the endmember identifiability of Craig's criterion for hyperspectral unmixing: A statistical analysis for three-source case , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  Lorenzo Bruzzone,et al.  A Novel Sharpening Approach for Superresolving Multiresolution Optical Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[51]  Thomas S. Pagano,et al.  Moderate Resolution Imaging Spectroradiometer (MODIS) , 1993, Defense, Security, and Sensing.

[52]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[53]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.