Manifold regularized sparse representation of injected details for pansharpening

ABSTRACT Pansharpening with sparse representation (SR) and details injection (ID) can both produce visually and quantitatively pleasing images. The former constructs pansharpened image by combining the dictionary and estimated sparse coefficients while the later by sharpening the multispectral bands through adding the proper spatial details from panchromatic (Pan) image. The combination of these two methods has been putting forward as the pansharpening method based on sparse representation of injected details (SR-D). Although SR-D has achieved better results both in visual and quantitative parts than many state-of-art methods, it ignores the intrinsic geometric structure connection between the multispectral image (MS) and the corresponding high-resolution MS image. In this paper, we propose a new pansharpening method, called manifold regularized sparse representation of injected details (MR-SR-D) by introducing a manifold regularization (MR) into the former SR-D model. The manifold regularization utilized a graph Laplacian to incorporate the locally geometrical structure of the multispectral data. Experimental results on the IKONOS, QuickBird and WorldView2 data sets show that the proposed method can achieve remarkable spectral and spatial quality on both reduced scale and full scale.

[1]  Henry Leung,et al.  Fusion of Multispectral and Panchromatic Images Using a Restoration-Based Method , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[3]  Manjunath V. Joshi,et al.  MAP Estimation for Multiresolution Fusion in Remotely Sensed Images Using an IGMRF Prior Model , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[5]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[6]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[7]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[8]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[11]  Mohamed-Jalal Fadili,et al.  The Undecimated Wavelet Decomposition and its Reconstruction , 2007, IEEE Transactions on Image Processing.

[12]  A. Mohammadzadeh,et al.  Road extraction based on fuzzy logic and mathematical morphology from pan‐sharpened ikonos images , 2006 .

[13]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[14]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[15]  Ye Zhang,et al.  Combining Component Substitution and Multiresolution Analysis: A Novel Generalized BDSD Pansharpening Algorithm , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Hujun Bao,et al.  Laplacian Regularized Gaussian Mixture Model for Data Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[19]  Guy Flouzat,et al.  Thematic and statistical evaluations of five panchromatic/multispectral fusion methods on simulated PLEIADES-HR images , 2005, Inf. Fusion.

[20]  Lei Wang,et al.  Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients , 2014, Inf. Fusion.

[21]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Thomas M. Breuel,et al.  Performance Evaluation and Benchmarking of Six-Page Segmentation Algorithms , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[24]  Hamid Reza Shahdoosti,et al.  Multimodal image fusion using sparse representation classification in tetrolet domain , 2018, Digit. Signal Process..

[25]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[26]  Lorenzo Bruzzone,et al.  A Model-Based Approach to Multiresolution Fusion in Remotely Sensed Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[28]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[29]  D. Roberts,et al.  Mapping forest degradation in the Eastern Amazon from SPOT 4 through spectral mixture models , 2003 .

[30]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[31]  B. Silverman,et al.  The Stationary Wavelet Transform and some Statistical Applications , 1995 .

[32]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[33]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[34]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Chunxia Zhang,et al.  Manifold regularization for sparse unmixing of hyperspectral images , 2016, SpringerPlus.

[36]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Jocelyn Chanussot,et al.  A Pansharpening Method Based on the Sparse Representation of Injected Details , 2015, IEEE Geoscience and Remote Sensing Letters.

[40]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[41]  Hamid Reza Shahdoosti,et al.  MRI and PET image fusion using structure tensor and dual ripplet-II transform , 2017, Multimedia Tools and Applications.

[42]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[43]  K. Whipple.  Can erosion drive tectonics? , 2014, Science.

[44]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[46]  Mengxi Xu,et al.  Fusion of multispectral and panchromatic images combining IHS and dual-tree complex wavelet , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[47]  Hamid Reza Shahdoosti,et al.  Pansharpening of Clustered MS and Pan Images Considering Mixed Pixels , 2017, IEEE Geoscience and Remote Sensing Letters.

[48]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[49]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[50]  H. S. Seung,et al.  Cognition. The manifold ways of perception. , 2000, Science.

[51]  Liangpei Zhang,et al.  A Practical Compressed Sensing-Based Pan-Sharpening Method , 2012, IEEE Geoscience and Remote Sensing Letters.

[52]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[53]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[54]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[55]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for surveillance images , 2010, Signal Process..

[56]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[57]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.