Image Fusion for Spatial Enhancement of Hyperspectral Image via Pixel Group Based Non-Local Sparse Representation

Restricted by technical and budget constraints, hyperspectral images (HSIs) are usually obtained with low spatial resolution. In order to improve the spatial resolution of a given hyperspectral image, a new spatial and spectral image fusion approach via pixel group based non-local sparse representation is proposed, which exploits the spectral sparsity and spectral non-local self-similarity of the hyperspectral image. The proposed approach fuses the hyperspectral image with a high-spatial-resolution multispectral image of the same scene to obtain a hyperspectral image with high spatial and spectral resolutions. The input hyperspectral image is used to train the spectral dictionary, while the sparse codes of the desired HSI are estimated by jointly encoding the similar pixels in each pixel group extracted from the high-spatial-resolution multispectral image. To improve the accuracy of the pixel group based non-local sparse representation, the similar pixels in a pixel group are selected by utilizing both the spectral and spatial information. The performance of the proposed approach is tested on two remote sensing image datasets. Experimental results suggest that the proposed method outperforms a number of sparse representation based fusion techniques, and can preserve the spectral information while recovering the spatial details under large magnification factors.

[1]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[2]  Guangming Shi,et al.  Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[3]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[4]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[5]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[6]  Peter Reinartz,et al.  Hyperspectral image resolution enhancement based on joint sparsity spectral unmixing , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[7]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Imagery Super-Resolution by Spatial–Spectral Joint Nonlocal Similarity , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Ying Li,et al.  Image fusion via nonlocal sparse K-SVD dictionary learning. , 2016, Applied optics.

[10]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[11]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[12]  John R. Schott,et al.  Evaluation of Two Applications of Spectral Mixing Models to Image Fusion , 2000 .

[13]  Rob Heylen,et al.  Fusion of Hyperspectral and Multispectral Images Using Spectral Unmixing and Sparse Coding , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[14]  Antonio J. Plaza,et al.  Sparse Unmixing of Hyperspectral Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Liangpei Zhang,et al.  An Online Coupled Dictionary Learning Approach for Remote Sensing Image Fusion , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Yi Ma,et al.  A non-negative sparse promoting algorithm for high resolution hyperspectral imaging , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[18]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Peter Bühlmann.  Regression shrinkage and selection via the Lasso: a retrospective (Robert Tibshirani): Comments on the presentation , 2011 .

[20]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[21]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[22]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[23]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Russell C. Hardie,et al.  Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[26]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[27]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[28]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[29]  Ajmal S. Mian,et al.  Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution , 2014, ECCV.

[30]  Roger L. King,et al.  Estimation of the Number of Decomposition Levels for a Wavelet-Based Multiresolution Multisensor Image Fusion , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[32]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[33]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[34]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Murat Simsek,et al.  The effect of dictionary learning algorithms on super-resolution hyperspectral reconstruction , 2015, 2015 XXV International Conference on Information, Communication and Automation Technologies (ICAT).

[36]  S. Macenka,et al.  Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) , 1988 .

[37]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[38]  Qingshan Liu,et al.  Improving the Spatial Resolution of Landsat TM/ETM+ Through Fusion With SPOT5 Images via Learning-Based Super-Resolution , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Da He,et al.  Nonlocal Total Variation Subpixel Mapping for Hyperspectral Remote Sensing Imagery , 2016, Remote. Sens..

[40]  Wallace M. Porter,et al.  The airborne visible/infrared imaging spectrometer (AVIRIS) , 1993 .

[41]  Manjunath V. Joshi,et al.  Super-Resolution of Hyperspectral Images: Use of Optimum Wavelet Filter Coefficients and Sparsity Regularization , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Michael E. Schaepman,et al.  Unmixing-Based Landsat TM and MERIS FR Data Fusion , 2008, IEEE Geoscience and Remote Sensing Letters.

[43]  Zongben Xu,et al.  Spatial and Spectral Image Fusion Using Sparse Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[45]  K. Moffett,et al.  Remote Sens , 2015 .

[46]  Hongyi Liu,et al.  Hyperspectral Imagery Super-Resolution by Compressive Sensing Inspired Dictionary Learning and Spatial-Spectral Regularization , 2015, Sensors.

[47]  Ajmal S. Mian,et al.  Bayesian sparse representation for hyperspectral image super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).