Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery

The volume of data grows with the advent of multiple types of remote sensing sensors and in order to extract the most useful information there is a need to combine the data gathered from the different sources. The widely used panchromatic and multispectral imageries in many applications offer decimetric and metric spatial resolution. However, the spectral resolution of these images is poor. Hyperspectral imaging has unique characteristics of providing very fine spectral resolution in a large number of bands with decametric spatial resolution and found to be highly useful for a wide span of application areas that requires high spectral resolution. The fusion of spectral and spatial information provides an effective way of enhancing the spatial quality of hyperspectral imagery as well as a method for preserving spectral quality. This fusion process is not a trivial task as always there has been a tradeoff between the preservation of spatial and spectral quality information as in the original sources of fusion. In this paper, a review on hyperspectral pansharpening and hyperspectral multispectral fusion based approaches has been reported. The widely adopted quantitative and qualitative performance measures to verify the fusion results are highlighted. In addition, the challenges in existing fusion techniques have also been discussed.

[1]  B. S. Manjunath,et al.  Multi-sensor image fusion using the wavelet transform , 1994, Proceedings of 1st International Conference on Image Processing.

[2]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3]  Dieter Oertel,et al.  Unmixing-based multisensor multiresolution image fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[4]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details , 2000 .

[5]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[6]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[7]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[8]  Su May Hsu,et al.  Multisensor Fusion with Hyperspectral Imaging Data: Detection and Classification , 2003 .

[9]  S. Baronti,et al.  Sharpening of very high resolution images with spectral distortion minimization , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[10]  Yi Shen,et al.  A quantitative method for evaluating the performances of hyperspectral image fusion , 2003, IEEE Trans. Instrum. Meas..

[11]  Yun Zhang,et al.  Understanding image fusion , 2004 .

[12]  Rafael García,et al.  Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Russell C. Hardie,et al.  MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor , 2004, IEEE Transactions on Image Processing.

[14]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[15]  Firooz Sadjadi,et al.  Comparative Image Fusion Analysais , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[16]  Russell C. Hardie,et al.  Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Hugh G. Lewis,et al.  Superresolution mapping using a hopfield neural network with fused images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[19]  V. P. Shah,et al.  On the Performance Evaluation of Pan-Sharpening Techniques , 2007, IEEE Geoscience and Remote Sensing Letters.

[20]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[21]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Ye Zhang,et al.  Integration of Spatial–Spectral Information for Resolution Enhancement in Hyperspectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Julien Radoux,et al.  Bayesian Data Fusion for Adaptable Image Pansharpening , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[24]  J. R. Raol,et al.  Pixel-level Image Fusion using Wavelets and Principal Component Analysis , 2008 .

[25]  Ye Zhang,et al.  Super-Resolution Challenges in Hyperspectral Imagery , 2008 .

[26]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[27]  Ye Zhang,et al.  Spatial-spectral data fusion for resolution enhancement of hyperspectral imagery , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[28]  Ye Zhang,et al.  Optimum Method Selection for Resolution Enhancement of Hyperspectral Imagery , 2009 .

[29]  Todd Wittman,et al.  A variational approach to hyperspectral image fusion , 2009, Defense + Commercial Sensing.

[30]  Subhasis Chaudhuri,et al.  A fast approach for fusion of hyperspectral images through redundancy elimination , 2010, ICVGIP '10.

[31]  F. A. Mianji,et al.  Key Information Retrieval in Hyperspectral Imagery through Spatial-Spectral Data Fusion , 2010 .

[32]  Luciano Alparone,et al.  Hyperspectral image fusion , 2010 .

[33]  Michael Möller,et al.  An Adaptive IHS Pan-Sharpening Method , 2010, IEEE Geoscience and Remote Sensing Letters.

[34]  Lei Zhang,et al.  Band-Subset-Based Clustering and Fusion for Hyperspectral Imagery Classification , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Ankush Khandelwal,et al.  Hyperspectral image enhancement based on sensor simulation and vector decomposition , 2011, 14th International Conference on Information Fusion.

[36]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[37]  Naoto Yokoya,et al.  Hyperspectral, multispectral, and panchromatic data fusion based on coupled non-negative matrix factorization , 2011, 2011 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[38]  Yufeng Zheng,et al.  Image Fusion and Its Applications , 2011 .

[39]  Laércio Massaru Namikawa,et al.  Image Fusion for Remote Sensing Applications , 2011 .

[40]  Jon Atli Benediktsson,et al.  Spectral Unmixing for the Classification of Hyperspectral Images at a Finer Spatial Resolution , 2011, IEEE Journal of Selected Topics in Signal Processing.

[41]  Isin Erer,et al.  Fusion of multispectral and panchromatic images by combining bilateral filter and IHS transform , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[42]  Jocelyn Chanussot,et al.  Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction , 2012, EURASIP J. Adv. Signal Process..

[43]  M. Matsuoka.  THE INFLUENCE OF SPECTRAL WAVELENGTH ON THE QUALITY OF PANSHARPENED IMAGE SIMULATED USING HYPERSPECTRAL DATA , 2012 .

[44]  J. Engels,et al.  Pansharpening of Hyperspectral Images in Urban Areas , 2012 .

[45]  Zhenwei Shi,et al.  Nonnegative matrix factorization-based hyperspectral and panchromatic image fusion , 2012, Neural Computing and Applications.

[46]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for hyperspectral images , 2012, Signal Process..

[47]  P. Reinartz,et al.  HYPERSPECTRAL IMAGE RESOLUTION ENHANCEMENT BASED ON SPECTRAL UNMIXING AND INFORMATION FUSION , 2012 .

[48]  Junwei Han,et al.  Object detection in remote sensing imagery using a discriminatively trained mixture model , 2013 .

[49]  Shahram Latifi,et al.  Performance Analysis of Contourlet-Based Hyperspectral Image Fusion Methods , 2013 .

[50]  Subhasis Chaudhuri,et al.  Hyperspectral Image Fusion , 2013, Springer New York.

[51]  Hina Pande,et al.  High-Resolution and Hyperspectral Data Fusion for Classification , 2013 .

[52]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[53]  Jean-Yves Tourneret,et al.  Bayesian fusion of hyperspectral and multispectral images , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Jocelyn Chanussot,et al.  MultiResolution Analysis and Component Substitution techniques for hyperspectral Pansharpening , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[55]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[56]  Jean-Yves Tourneret,et al.  Bayesian fusion of multispectral and hyperspectral images using a block coordinate descent method , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[57]  Xiao Xiang Zhu,et al.  Dynamic dictionary learning strategies for sparse representation based hyperspectral image enhancement , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[58]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Jocelyn Chanussot,et al.  Impact of hybrid pansharpening approaches applied to hyperspectral images , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[60]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[61]  E. Baltsavias,et al.  ADVANCES IN HYPERSPECTRAL AND MULTISPECTRAL IMAGE FUSION AND SPECTRAL UNMIXING , 2015 .

[62]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Aleksandra Pizurica,et al.  Generalized Graph-Based Fusion of Hyperspectral and LiDAR Data Using Morphological Features , 2015, IEEE Geoscience and Remote Sensing Letters.

[64]  Jon Atli Benediktsson,et al.  A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.