Multi-source remote sensing data fusion: status and trends

With the fast development of remote sensor technologies, e.g. the appearance of Very High Resolution (VHR) optical sensors, SAR, LiDAR, etc., mounted on either airborne or spaceborne platforms, multi-source remote sensing data fusion techniques are emerging due to the demand for new methods and algorithms. The general fusion techniques have been well developed and applied in various fields ranging from satellite earth observation to computer vision, medical image processing, defence security and so on. Despite the fast development, the techniques remain challenging for multi-source data fusion within varying spatial and temporal resolutions. This article reviews current techniques of multi-source remote sensing data fusion and discusses their future trends and challenges through the concept of hierarchical classification, i.e., pixel/data level, feature level and decision level. This article concentrates on discussing optical panchromatic and multi-spectral data fusing methods. So far, the pixel level fusion methods have mainly focused on optical data fusion; high-level fusion includes feature level and decision level fusion of multi-source data, such as synthetic aperture radar, optical images, LiDAR and other types of data. Finally, this article summarises several trends tending to broaden the application of multi-source data fusion.

[1]  Guoqing Sun,et al.  A three-dimensional radar backscatter model of forest canopies , 1995, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Jon Atli Benediktsson,et al.  Fusion of Support Vector Machines for Classification of Multisensor Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Fabio Remondino,et al.  Photogrammetry and Ground-based Laser Scanning: Assessment of Metric Accuracy of the 3D Model of Pozzoveggiani Church , 2004 .

[5]  Jocelyn Chanussot,et al.  Fuzzy fusion techniques for linear features detection in multitemporal SAR images , 1999, IEEE Trans. Geosci. Remote. Sens..

[6]  Jon Atli Benediktsson,et al.  Decision Fusion for the Classification of Urban Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[7]  David A. Landgrebe,et al.  Decision fusion approach for multitemporal classification , 1999, IEEE Trans. Geosci. Remote. Sens..

[8]  Anil K. Jain,et al.  A Markov random field model for classification of multisource satellite imagery , 1996, IEEE Trans. Geosci. Remote. Sens..

[9]  Isabelle Bloch,et al.  Application of Dempster-Shafer evidence theory to unsupervised classification in multisource remote sensing , 1997, IEEE Trans. Geosci. Remote. Sens..

[10]  Yang Guijun GUIDED AUTOMATIC REMOTE SENSING DETECTION OF LAND USE CHANGES BASED ON THEMATIC KNOWLEDGE , 2005 .

[11]  Jinghui Yang,et al.  Block-regression based fusion of optical and SAR imagery for feature enhancement , 2010 .

[12]  Xiangyun Hu,et al.  Automatic Road Extraction from Dense Urban Area by Integrated Processing of High Resolution Imagery and Lidar Data , 2004 .

[13]  R. Milne,et al.  Integrating remote sensing datasets into ecological modelling: a Bayesian approach , 2008 .

[14]  Xavier Otazu,et al.  A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Y. Hu,et al.  Mapping canopy height using a combination of digital stereo‐photogrammetry and lidar , 2008 .

[16]  Manfred Ehlers,et al.  FFT-enhanced IHS transform method for fusing high-resolution satellite images , 2007 .

[17]  Nelson D. A. Mascarenhas,et al.  Multispectral image data fusion under a Bayesian approach , 1996 .

[18]  Guoqing Sun,et al.  Modeling lidar returns from forest canopies , 2000, IEEE Trans. Geosci. Remote. Sens..

[19]  C. Ginzler,et al.  Arial images and LIDAR fusion applied in forest boundary detection , 2007 .

[20]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[21]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Uwe Stilla,et al.  Region refinement and parametric reconstruction of building roofs by integration of image and height data , 2005 .

[23]  Kurt Kubik,et al.  Fusing airborne laser scanner data and aerial imagery for the automatic extraction of buildings in densely built-up areas , 2004 .

[24]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[25]  Yahya Alshawabkeh,et al.  Integration of Digital Photogrammetry and Laser Scanning for Heritage Documentation , 2004 .

[26]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[27]  Xiaoyong Chen,et al.  Extraction of City Roads Through Shadow Path Reconstruction Using Laser Data , 2004 .

[28]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[29]  D. Maktav,et al.  The role of the integration of remote sensing and GIS in land use/land cover analysis after an earthquake , 2009 .

[30]  N. Koutsias,et al.  The use of intensity-hue-saturation transformation of Landsat-5 thematic mapper data for burned land mapping. , 2000 .

[31]  D. Holcomb,et al.  Optimizing the High-Pass Filter Addition Technique for Image Fusion , 2007 .

[32]  W. Wagner,et al.  Gaussian decomposition and calibration of a novel small-footprint full-waveform digitising airborne laser scanner , 2006 .

[33]  Luciano Alparone,et al.  Image fusion—the ARSIS concept and some successful implementation schemes , 2003 .

[34]  Johannes R. Sveinsson,et al.  Model-Based Satellite Image Fusion , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Y. Chibani,et al.  The joint use of IHS transform and redundant wavelet decomposition for fusing multispectral and panchromatic images , 2002 .

[36]  Franz Rottensteiner,et al.  Automatic Generation of High-Quality Building Models from Lidar Data , 2003, IEEE Computer Graphics and Applications.

[37]  Tong Lee,et al.  Probabilistic and Evidential Approaches for Multisource Data Analysis , 1987, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Jon Atli Benediktsson,et al.  Neural Network Approaches Versus Statistical Methods in Classification of Multisource Remote Sensing Data , 1989, 12th Canadian Symposium on Remote Sensing Geoscience and Remote Sensing Symposium,.

[39]  D. Amarsaikhan,et al.  Data fusion and multisource image classification , 2004 .

[40]  Krištof Oštir,et al.  High-resolution image fusion : Methods to preserve spectral and spatial resolution , 2006 .

[41]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[42]  P. Dherete,et al.  Data fusion applications: classification and mapping , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[43]  Sönke Müller,et al.  A framework for GIS and imagery data fusion in support of cartographic updating , 2005, Inf. Fusion.

[44]  Hai Wei,et al.  Fusion of LIDAR and aerial imagery for accurate building footprint extraction , 2009, Electronic Imaging.

[45]  Tom Duckett,et al.  Fusion of aerial images and sensor data from a ground vehicle for improved semantic mapping , 2008, Robotics Auton. Syst..

[46]  Andrea Garzelli,et al.  Interband structure modeling for Pan-sharpening of very high-resolution multispectral images , 2005, Inf. Fusion.

[47]  K. Huemmrich The GeoSail model: a simple addition to the SAIL model to describe discontinuous canopy reflectance , 2001 .

[48]  D. Fofi,et al.  Front Matter: Volume 7251 , 2009 .

[49]  M. Hofton,et al.  LAND SURFACE CHARACTERIZATION USING LIDAR REMOTE SENSING , 2001 .

[50]  Gabriele Guidi,et al.  Fusion of range camera and photogrammetry: A systematic procedure for improving 3-D models metric accuracy , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[51]  Jon Atli Benediktsson,et al.  Classification of multisource and hyperspectral data based on decision fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[52]  Jan G. P. W. Clevers,et al.  Mapping of aggregated floodplain plant communities using image fusion of CASI and LiDAR data , 2009, Int. J. Appl. Earth Obs. Geoinformation.

[53]  P. J. Williams,et al.  Data fusion on a distributed heterogeneous sensor network , 2006, SPIE Defense + Commercial Sensing.

[54]  J. Vrabel,et al.  Multispectral Imagery Advanced Band Sharpening Study , 2000 .

[55]  D. Yocky Multiresolution wavelet decomposition image merger of landsat thematic mapper and SPOT panchromatic data , 1996 .

[56]  Björn Waske,et al.  Classifying Multilevel Imagery From SAR and Optical Sensors by Decision Fusion , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[58]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[59]  J.-Angelo Beraldin,et al.  INTEGRATION OF LASER SCANNING AND CLOSE-RANGE PHOTOGRAMMETRY – THE LAST DECADE AND BEYOND , 2004 .

[60]  Guoqing Sun,et al.  The Potential of Combined Lidar and SAR Data in Retrieving Forest Parameters using Model Analysis , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[61]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[62]  Yun Zhang,et al.  An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images , 2005, Inf. Fusion.

[63]  Manfred Ehlers Spectral characteristics preserving image fusion based on Fourier domain filtering , 2004, SPIE Remote Sensing.

[64]  Pascal Vasseur,et al.  Introduction to Multisensor Data Fusion , 2005, The Industrial Information Technology Handbook.

[65]  Guoqing Sun,et al.  Inversion of a lidar waveform model for forest biophysical parameter estimation , 2006, IEEE Geoscience and Remote Sensing Letters.

[66]  A. H. J. M. Pellemans,et al.  MERGING MULTISPECTRAL AND PANCHROMATIC SPOT IMAGES WITH RESPECT TO THE RADIOMETRIC PROPERTIES OF THE SENSOR , 1993 .

[67]  J. Shan,et al.  Building boundary tracing and regularization from airborne lidar point clouds , 2007 .

[68]  Isabelle Bloch,et al.  A first step toward automatic interpretation of SAR images using evidential fusion of several structure detectors , 1999, IEEE Trans. Geosci. Remote. Sens..

[69]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[70]  Alfred Stein,et al.  Use of single- and multi-source image fusion for statistical decision-making , 2005 .

[71]  Lorenzo Bruzzone,et al.  A neural-statistical approach to multitemporal and multisource remote-sensing image classification , 1999, IEEE Trans. Geosci. Remote. Sens..

[72]  M. Scaioni,et al.  Fusion of aerial and satellite imagery over the city of Venezia , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[73]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[74]  Anil K. Jain,et al.  Multisource classification of remotely sensed data: fusion of Landsat TM and SAR images , 1994, IEEE Trans. Geosci. Remote. Sens..

[75]  Rafael García,et al.  Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[76]  K. Itten,et al.  Fusion of imaging spectrometer and LIDAR data over combined radiative transfer models for forest canopy characterization , 2007 .

[77]  Ali Mohammad-Djafari,et al.  A Bayesian Approach for Data and Image Fusion , 2003 .

[78]  Z. Liu,et al.  Bidimensional Empirical Mode Decomposition for the fusion of multispectral and panchromatic images , 2007 .

[79]  G. Sithole,et al.  Segmentation and classification of airborne laser scanner data , 2005 .

[80]  F. Nencini,et al.  Fusion of Panchromatic and Multispectral Images by Genetic Algorithms , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[81]  Qihao Weng,et al.  A survey of image classification methods and techniques for improving classification performance , 2007 .

[82]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[83]  Frederico T. Fonseca,et al.  Semantic Granularity in Ontology-Driven Geographic Information Systems , 2002, Annals of Mathematics and Artificial Intelligence.

[84]  J. Chassery,et al.  The use of multiresolution analysis and wavelets transform for merging SPOT panchromatic and multisp , 1996 .

[85]  Y. Zhang,et al.  A new merging method and its spectral and spatial effects , 1999 .

[86]  K. Itten,et al.  LIDAR-based geometric reconstruction of boreal type forest stands at single tree level for forest and wildland fire management , 2004 .

[87]  Jocelyn Chanussot,et al.  Foreword to the Special Issue on Data Fusion , 2008, IEEE Trans. Geosci. Remote. Sens..

[88]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[89]  George Vosselman,et al.  Accuracy of 3D city models: EuroSDR comparison , 2005 .

[90]  M. Wulder,et al.  Forest inventory height update through the integration of lidar data with segmented Landsat imagery , 2003 .

[91]  Yun Zhang,et al.  Wavelet based image fusion techniques — An introduction, review and comparison , 2007 .

[92]  Deyi Li,et al.  LAND USE CLASSIFICATION OF REMOTE SENSING IMAGE WITH GIS DATA BASED ON SPATIAL DATA MINING TECHNIQUES , 2000 .