A review of remote sensing image fusion methods

The recent years have been marked by continuous improvements of remote sensors with applications like monitoring and management of the environment, precision agriculture, security and defense. On the one hand, the high spectral resolution is necessary for an accurate class discrimination of land covers. On the other hand, the high spatial resolution is required for an accurate description of the texture and shapes. Practically, each kind of imaging sensor can only focus on a given different operating range and environmental conditions, the reception of all the necessary information for detecting an object or classifying a scene is not possible. So, for the full exploitation of multisource data, advanced analytical or numerical image fusion techniques have been developed. In this paper, we review some popular and state-of-the-art fusion methods in different levels especially at pixel level. In addition to reviewing of different fusion methods, varied approaches and metrics for assessment of fused product are also presented.

[1]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[2]  Wenda Zhao,et al.  Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion , 2016, Inf. Fusion.

[3]  Manjunath V. Joshi,et al.  A learning-based method for image super-resolution from zoomed observations , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Rick S. Blum On multisensor image fusion performance limits from an estimation theory perspective , 2006, Inf. Fusion.

[5]  Jun Zhou,et al.  Object Classification via Feature Fusion Based Marginalized Kernels , 2015, IEEE Geoscience and Remote Sensing Letters.

[6]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[7]  Yifan Zhang,et al.  Noise-Resistant Wavelet-Based Bayesian Fusion of Multispectral and Hyperspectral Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Wenzhong Shi,et al.  Remote Sensing Image Fusion Using Multiscale Mapped LS-SVM , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[9]  A. Venetsanopoulos,et al.  Image coding using vector quantization in LP/sub /spl infin// space , 1998, 1998 IEEE Symposium on Advances in Digital Filtering and Signal Processing. Symposium Proceedings (Cat. No.98EX185).

[10]  Mona Ghassemian,et al.  Multi-sensor image fusion using multirate filter banks , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[11]  Fen Qin,et al.  Fusion of Remote Sensing Images Using Improved ICA Mergers Based on Wavelet Decomposition , 2012 .

[12]  P. Switzer,et al.  A transformation for ordering multispectral data in terms of image quality with implications for noise removal , 1988 .

[13]  B. Silverman,et al.  The Stationary Wavelet Transform and some Statistical Applications , 1995 .

[14]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[15]  David Bull,et al.  Region-Based Image Fusion Using Complex Wavelets , 2004 .

[16]  Lei Yang,et al.  Ripplet: A new transform for image processing , 2010, J. Vis. Commun. Image Represent..

[17]  Farhad Samadzadegan,et al.  Object Recognition Based on the Context Aware Decision-Level Fusion in Multiviews Imagery , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Hassan Ghassemian,et al.  Combining the spectral PCA and spatial PCA fusion methods by an optimal filter , 2016, Inf. Fusion.

[19]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Fang Liu,et al.  Multicontourlet-Based Adaptive Fusion of Infrared and Visible Remote Sensing Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[21]  Ting Liu,et al.  Remote sensing image fusion via wavelet transform and sparse representation , 2015 .

[22]  Jesús García,et al.  Context-based Information Fusion: A survey and discussion , 2015, Inf. Fusion.

[23]  Lucien Wald,et al.  Data fusion: taking into account the modulation transfer function in ARSIS- based pansharpening methods , 2009 .

[24]  W. A. Wright Fast image fusion with a Markov random field , 1999 .

[25]  Lucien Wald,et al.  Analysis of Changes in Quality Assessment with Scale , 2006, 2006 9th International Conference on Information Fusion.

[26]  Hassan Ghassemian,et al.  Nonminutiae-Based Decision-Level Fusion for Fingerprint Verification , 2007, EURASIP J. Adv. Signal Process..

[27]  Zhiguo Cao,et al.  Joint Image Registration and Fusion for Panchromatic and Multispectral Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[28]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[29]  Christine Pohl,et al.  A comparison of geometric models for multisource data fusion , 1995 .

[30]  John van Genderen,et al.  Structuring contemporary remote sensing image fusion , 2015 .

[31]  David Gladstone Review article , 2005, Health Care Analysis.

[32]  Zongben Xu,et al.  Spatial and Spectral Image Fusion Using Sparse Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Kishor P. Upla,et al.  An Edge Preserving Multiresolution Fusion: Use of Contourlet Transform and MRF Prior , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Thierry Ranchin,et al.  A modular platform for fusion of images , 2005 .

[35]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[36]  Belur V. Dasarathy,et al.  Urban remote sensing using multiple data sets: Past, present, and future , 2005, Inf. Fusion.

[37]  Lorenzo Bruzzone,et al.  Image fusion techniques for remote sensing applications , 2002, Inf. Fusion.

[38]  Sabalan Daneshvar,et al.  MRI and PET image fusion by combining IHS and retina-inspired models , 2010, Inf. Fusion.

[39]  Hassan Ghassemian,et al.  Remote-sensing image fusion based on curvelets and ICA , 2015 .

[40]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[41]  Oliver Rockinger Pixel - Level Fusion of Image Sequences using Wavelet Frames , 1996 .

[42]  Hongwei Zhu,et al.  Data fusion for pattern classification via the Dempster-Shafer evidence theory , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[43]  Weile Zhu,et al.  Fusion of IKONOS Satellite Imagery Using IHS Transform and Local Variation , 2008, IEEE Geoscience and Remote Sensing Letters.

[44]  Jean-Yves Tourneret,et al.  Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation , 2015, IEEE Transactions on Image Processing.

[45]  Hassan Ghassemian,et al.  Fusion of MS and PAN Images Preserving Spectral Quality , 2015, IEEE Geoscience and Remote Sensing Letters.

[46]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[47]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[48]  Wei Wu,et al.  A fusion algorithm of remote sensing image based on discrete wavelet packet , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[49]  N. C. Strugnell,et al.  A global albedo data set derived from AVHRR data for use in climate simulations , 2001 .

[50]  H. Ghassemian,et al.  Remote sensing image fusion using combining IHS and Curvelet transform , 2012, 6th International Symposium on Telecommunications (IST).

[51]  Wu Wenbo,et al.  STUDY OF REMOTE SENSING IMAGE FUSION AND ITS APPLICATION IN IMAGE CLASSIFICATION , 2008 .

[52]  Christine Pohl,et al.  Remote sensing image fusion: an update in the context of Digital Earth , 2014, Int. J. Digit. Earth.

[53]  Hui Lin,et al.  Fusion of WorldView-2 Stereo and Multitemporal TerraSAR-X Images for Building Height Extraction in Urban Areas , 2015, IEEE Geoscience and Remote Sensing Letters.

[54]  Bin Luo,et al.  Decision-Based Fusion for Pansharpening of Remote Sensing Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[55]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[56]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[57]  李幼升,et al.  Ph , 1989 .

[58]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.

[59]  Jean-Yves Tourneret,et al.  Bayesian Fusion of Multi-Band Images , 2013, IEEE Journal of Selected Topics in Signal Processing.

[60]  Lucien Wald,et al.  Some terms of reference in data fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[61]  Jon Atli Benediktsson,et al.  Decision Fusion for the Classification of Urban Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Shutao Li,et al.  Performance comparison of different multi-resolution transforms for image fusion , 2011, Inf. Fusion.

[63]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[64]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[66]  K. Nikolakopoulos Comparison of Nine Fusion Techniques for Very High Resolution Data , 2008 .

[67]  Qian Du,et al.  Multi-Modal and Multi-Temporal Data Fusion: Outcome of the 2012 GRSS Data Fusion Contest , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[68]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.

[69]  Hassan Ghassemian,et al.  Remote Sensing Image Fusion Using Ripplet Transform and Compressed Sensing , 2015, IEEE Geoscience and Remote Sensing Letters.

[70]  Hassan Ghassemian,et al.  Integrating Hierarchical Segmentation Maps With MRF Prior for Classification of Hyperspectral Images in a Bayesian Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[71]  Yinghao Li,et al.  Jointly registering and fusing images from multiple sensors , 2016, Inf. Fusion.

[72]  Johannes R. Sveinsson,et al.  Hybrid consensus theoretic classification , 1996, IGARSS '96. 1996 International Geoscience and Remote Sensing Symposium.

[73]  Myeong-Ryong Nam,et al.  Fusion of multispectral and panchromatic Satellite images using the curvelet transform , 2005, IEEE Geoscience and Remote Sensing Letters.

[74]  Qingshan Liu,et al.  Improving the Spatial Resolution of Landsat TM/ETM+ Through Fusion With SPOT5 Images via Learning-Based Super-Resolution , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Lei Cao,et al.  A GIHS-based spectral preservation fusion method for remote sensing images using edge restored spectral modulation , 2014 .

[76]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[77]  Johannes R. Sveinsson,et al.  Model-Based Fusion of Multi- and Hyperspectral Images Using PCA and Wavelets , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[78]  Sabalan Daneshvar,et al.  MRI and PET images fusion based on human retina model , 2007 .

[79]  Luciano Alparone,et al.  Bi-cubic interpolation for shift-free pan-sharpening , 2013 .

[80]  Liangpei Zhang,et al.  An Online Coupled Dictionary Learning Approach for Remote Sensing Image Fusion , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[81]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[82]  J. Moreno,et al.  Seasonal variations of leaf area index of agricultural fields retrieved from Landsat data , 2008 .

[83]  Yan Luo,et al.  FUSION OF REMOTE SENSING IMAGE BASE ON THE PCA + ATROUS WAVELET TRANSFORM , 2008 .

[84]  Ali J. Rashidi,et al.  A New Approach for Multi-System/Sensor Decision Fusion Based on Joint Measures , 2004, Int. J. Inf. Acquis..

[85]  Pramod K. Varshney,et al.  An Image Fusion Approach Based on Markov Random Fields , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[86]  K. Beurs,et al.  Evaluation of Landsat and MODIS data fusion products for analysis of dryland forest phenology , 2012 .

[87]  Luis O. Jimenez,et al.  Classification of hyperdimensional data based on feature and decision fusion approaches using projection pursuit, majority voting, and neural networks , 1999, IEEE Trans. Geosci. Remote. Sens..

[88]  Robert E. Wolfe,et al.  A Landsat surface reflectance dataset for North America, 1990-2000 , 2006, IEEE Geoscience and Remote Sensing Letters.

[89]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[90]  Lucien Wald,et al.  A MTF-Based Distance for the Assessment of Geometrical Quality of Fused Products , 2006, 2006 9th International Conference on Information Fusion.

[91]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[92]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[93]  David A. Landgrebe,et al.  On line object feature extraction for multispectral scene representation , 1988 .

[94]  Jun Liu,et al.  Human visual system consistent quality assessment for remote sensing image fusion , 2015 .

[95]  M. Al-Mualla,et al.  Beyond pan-sharpening: Pixel-level fusion in remote sensing applications , 2012, 2012 International Conference on Innovations in Information Technology (IIT).

[96]  Huinian Zhang,et al.  Remote Sensing Image Fusion Based on Discrete Fractional Random Transform for Modified IHS , 2013 .

[97]  Lorenzo Bruzzone,et al.  A Model-Based Approach to Multiresolution Fusion in Remotely Sensed Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[98]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[99]  Bhaskar Ramamurthi,et al.  Image coding using vector quantization , 1982, ICASSP.

[100]  Hassan Ghassemian,et al.  A Compressed-Sensing-Based Pan-Sharpening Method for Spectral Distortion Reduction , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[101]  Zhenhua Li,et al.  Color transfer based remote sensing image fusion using non-separable wavelet frame transform , 2005, Pattern Recognit. Lett..

[102]  S. Baronti,et al.  Remote Sensing Image Fusion , 2015 .

[103]  Hassan Ghassemian MULTI-SENSOR IMAGE FUSION BY INVERSE SUBBAND CODING , 2000 .

[104]  Subhasis Chaudhuri,et al.  Super-resolution imaging: use of zoom as a cue , 2004, Image Vis. Comput..

[105]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[106]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[107]  Patrick Hostert,et al.  Land cover mapping of large areas using chain classification of neighboring Landsat satellite images , 2009 .

[108]  Deyu Meng,et al.  Spatial and Temporal Image Fusion via Regularized Spatial Unmixing , 2015, IEEE Geoscience and Remote Sensing Letters.

[109]  Hassan Ghassemian A retina based multi-resolution image-fusion , 2001, IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No.01CH37217).

[110]  Hossein Pourghassem,et al.  Combination of Feature and Pixel Level Image Fusion with Feedback Retina and IHS Model , 2022 .

[111]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[112]  F. Nencini,et al.  Fusion of Panchromatic and Multispectral Images by Genetic Algorithms , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[113]  D. Holcomb,et al.  Optimizing the High-Pass Filter Addition Technique for Image Fusion , 2007 .

[114]  Liangpei Zhang,et al.  Quality Assessment of Panchromatic and Multispectral Image Fusion for the ZY-3 Satellite: From an Information Extraction Perspective , 2014, IEEE Geoscience and Remote Sensing Letters.

[115]  Yun Zhang,et al.  Understanding image fusion , 2004 .

[116]  Rafael García,et al.  Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[117]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[118]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[119]  H. Ghassemian,et al.  Improving hyperspectral image classification by combining spectral, texture, and shape features , 2015 .