Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions

Satellite time series with high spatial resolution is critical for monitoring land surface dynamics in heterogeneous landscapes. Although remote sensing technologies have experienced rapid development in recent years, data acquired from a single satellite sensor are often unable to satisfy our demand. As a result, integrated use of data from different sensors has become increasingly popular in the past decade. Many spatiotemporal data fusion methods have been developed to produce synthesized images with both high spatial and temporal resolutions from two types of satellite images, frequent coarse-resolution images, and sparse fine-resolution images. These methods were designed based on different principles and strategies, and therefore show different strengths and limitations. This diversity brings difficulties for users to choose an appropriate method for their specific applications and data sets. To this end, this review paper investigates literature on current spatiotemporal data fusion methods, categorizes existing methods, discusses the principal laws underlying these methods, summarizes their potential applications, and proposes possible directions for future studies in this field.

[1]  Jindi Wang,et al.  Bayesian Method for Building Frequent Landsat-Like NDVI Datasets by Integrating MODIS and Landsat NDVI , 2016, Remote. Sens..

[2]  Thomas Hilker,et al.  An Improved Image Fusion Approach Based on Enhanced Spatial and Temporal the Adaptive Reflectance Fusion Model , 2013, Remote. Sens..

[3]  Quazi K. Hassan,et al.  Spatiotemporal image-fusion model for enhancing the temporal resolution of Landsat-8 surface reflectance images using MODIS images , 2015 .

[4]  Rebekka R. E. Artz,et al.  Potential for using remote sensing to estimate carbon fluxes across northern peatlands - A review. , 2018, The Science of the total environment.

[5]  Albert Y. Zomaya,et al.  Spatiotemporal Fusion of MODIS and Landsat-7 Reflectance Images via Compressed Sensing , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Bo Wu,et al.  Improving spatiotemporal reflectance fusion using image inpainting and steering kernel regression techniques , 2017 .

[7]  Devendra Singh,et al.  Generation and evaluation of gross primary productivity using Landsat data through blending with MODIS data , 2011, Int. J. Appl. Earth Obs. Geoinformation.

[8]  Peter M. Atkinson,et al.  Spatio-temporal fusion for daily Sentinel-2 images , 2018 .

[9]  Xiaolin Zhu,et al.  An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions , 2010 .

[10]  Yanhong Tang,et al.  Influences of temperature and precipitation before the growing season on spring phenology in grasslands of the central and eastern Qinghai-Tibetan Plateau , 2011 .

[11]  Bo Huang,et al.  Generating High Spatiotemporal Resolution Land Surface Temperature for Urban Heat Island Monitoring , 2013, IEEE Geoscience and Remote Sensing Letters.

[12]  Matthew F. McCabe,et al.  A Spatio-Temporal Enhancement Method for medium resolution LAI (STEM-LAI) , 2016, Int. J. Appl. Earth Obs. Geoinformation.

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  A. Senthil Kumar,et al.  Fast spatiotemporal data fusion: merging LISS III with AWiFS sensor data , 2014 .

[15]  Alex J. Cannon,et al.  Crop yield forecasting on the Canadian Prairies by remotely sensed vegetation indices and machine learning methods , 2016 .

[16]  Tim R. McVicar,et al.  Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection , 2013 .

[17]  Cornelius Senf,et al.  Mapping land cover in complex Mediterranean landscapes using Landsat: Improved classification accuracies from integrating multi-seasonal and synthetic imagery , 2015 .

[18]  Qingshan Liu,et al.  Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Peijuan Wang,et al.  Fusing Landsat and MODIS Data for Vegetation Monitoring , 2015, IEEE Geoscience and Remote Sensing Magazine.

[20]  David P. Roy,et al.  A note on the temporary misregistration of Landsat-8 Operational Land Imager (OLI) and Sentinel-2 Multi Spectral Instrument (MSI) imagery , 2016 .

[21]  F. Maselli,et al.  Integration of LAC and GAC NDVI data to improve vegetation monitoring in semi-arid environments , 2002 .

[22]  Liangpei Zhang,et al.  An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Yanchen Bo,et al.  Blending multi-resolution satellite sea surface temperature (SST) products using Bayesian maximum entropy method , 2013 .

[24]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[25]  Jin Chen,et al.  An Improved Method for Producing High Spatial-Resolution NDVI Time Series Datasets with Multi-Temporal MODIS NDVI Data and Landsat TM/ETM+ Images , 2015, Remote. Sens..

[26]  Yuhan Rao,et al.  Land cover change detection by integrating object-based data blending model of Landsat and MODIS , 2016 .

[27]  H. Nagendra,et al.  Remote sensing for conservation monitoring: Assessing protected areas, habitat extent, habitat condition, species diversity, and threats , 2013 .

[28]  Jungho Im,et al.  Downscaling of MODIS One Kilometer Evapotranspiration Using Landsat-8 Data and Machine Learning Approaches , 2016, Remote. Sens..

[29]  Robert Frouin,et al.  Upscale integration of normalized difference vegetation index: the problem of spatial heterogeneity , 1992, IEEE Trans. Geosci. Remote. Sens..

[30]  Dieter Oertel,et al.  Unmixing-based multisensor multiresolution image fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[31]  D. Roy,et al.  Multi-temporal MODIS-Landsat data fusion for relative radiometric normalization, gap filling, and prediction of Landsat data , 2008 .

[32]  Michael E. Schaepman,et al.  Unmixing-Based Landsat TM and MERIS FR Data Fusion , 2008, IEEE Geoscience and Remote Sensing Letters.

[33]  Wei Zhang,et al.  An Enhanced Spatial and Temporal Data Fusion Model for Fusing Landsat and MODIS Surface Reflectance to Generate High Temporal Landsat-Like Data , 2013, Remote. Sens..

[34]  F. Gao,et al.  Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data , 2014 .

[35]  Peter M. Atkinson,et al.  Fusion of Landsat 8 OLI and Sentinel-2 MSI Data , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Mingquan Wu,et al.  Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model , 2012 .

[37]  Bin Chen,et al.  Comparison of Spatiotemporal Fusion Models: A Review , 2015, Remote. Sens..

[38]  G. Foody,et al.  Generating a series of fine spatial and temporal resolution land cover maps by fusing coarse spatial resolution remotely sensed images and fine spatial resolution land cover maps , 2017 .

[39]  Shilong Piao,et al.  Delayed autumn phenology in the Northern Hemisphere is related to change in both climate and spring phenology , 2016, Global change biology.

[40]  Sebastian T Meyer,et al.  Towards a standardized Rapid Ecosystem Function Assessment (REFA). , 2015, Trends in ecology & evolution.

[41]  B. Kleinschmit,et al.  The benefit of synthetically generated RapidEye and Landsat 8 data fusion time series for riparian forest disturbance monitoring , 2016 .

[42]  Conghe Song,et al.  Downscaling real-time vegetation dynamics by fusing multi-temporal MODIS and Landsat NDVI in topographically complex terrain , 2011 .

[43]  Bo Huang,et al.  Unified fusion of remote-sensing imagery: generating simultaneously high-resolution synthetic spatial–temporal–spectral earth observations , 2013 .

[44]  Xiang Li,et al.  Land Cover Classification Based on Fused Data from GF-1 and MODIS NDVI Time Series , 2016, Remote. Sens..

[45]  Mingquan Wu,et al.  Generating Daily Synthetic Landsat Imagery by Combining Landsat and MODIS Data , 2015, Sensors.

[46]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Peijun Du,et al.  Disaggregation of remotely sensed land surface temperature: A simple yet flexible index (SIFI) to assess method performances , 2017 .

[48]  V. Moosavi,et al.  A wavelet-artificial intelligence fusion approach (WAIFA) for blending Landsat and MODIS surface temperature , 2015 .

[49]  W. Tobler.  A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[50]  Anatoly A. Gitelson,et al.  Remote estimation of gross primary productivity in crops using MODIS 250m data , 2013 .

[51]  H. Kerdiles,et al.  NOAA-AVHRR NDVI decomposition and subpixel classification using linear mixing in the Argentinean Pampa , 1995 .

[52]  B. Lakshmi,et al.  Spatiotemporal Data Fusion Using Temporal High-Pass Modulation and Edge Primitives , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Annemarie Schneider,et al.  Monitoring land cover change in urban and peri-urban areas using dense time stacks of Landsat satellite data and a data mining approach , 2012 .

[54]  Peijuan Wang,et al.  Operational Data Fusion Framework for Building Frequent Landsat-Like Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[55]  T. Ohta,et al.  Development and evaluation of a lookup-table-based approach to data fusion for seasonal wetlands monitoring: An integrated use of AMSR series, MODIS, and Landsat , 2017 .

[56]  David B. Lobell,et al.  The use of satellite data for crop yield gap analysis , 2013 .

[57]  K. Beurs,et al.  Dryland vegetation phenology across an elevation gradient in Arizona, USA, investigated with fused MODIS and Landsat data , 2014 .

[58]  N. Ramankutty,et al.  Closing yield gaps through nutrient and water management , 2012, Nature.

[59]  N. Pettorelli,et al.  Satellite remote sensing of ecosystem functions: opportunities, challenges and way forward , 2018 .

[60]  Bin Chen,et al.  Multi-source remotely sensed data fusion for improving land cover classification , 2017 .

[61]  Mohammad Kakooei,et al.  Fusion of satellite, aircraft, and UAV data for automatic disaster damage assessment , 2017 .

[62]  Chunhua Liao,et al.  A Spatio-Temporal Data Fusion Model for Generating NDVI Time Series in Heterogeneous Regions , 2017, Remote. Sens..

[63]  Chengquan Huang,et al.  Improving Satellite Estimates of the Fraction of Absorbed Photosynthetically Active Radiation Through Data Integration: Methodology and Validation , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[64]  Chi-Farn Chen,et al.  A logistic-based method for rice monitoring from multitemporal MODIS-Landsat fusion data , 2016 .

[65]  Bo Huang,et al.  A Rigorously-Weighted Spatiotemporal Fusion Model with Uncertainty Analysis , 2017, Remote. Sens..

[66]  Chunlin Huang,et al.  Mapping daily evapotranspiration based on spatiotemporal fusion of ASTER and MODIS images over irrigated agricultural areas in the Heihe River Basin, Northwest China , 2017 .

[67]  Martha C. Anderson,et al.  Toward mapping crop progress at field scales through fusion of Landsat and MODIS imagery , 2017 .

[68]  S. Carpenter,et al.  Solutions for a cultivated planet , 2011, Nature.

[69]  Lindi J. Quackenbush,et al.  A simple Landsat–MODIS fusion approach for monitoring seasonal evapotranspiration at 30 m spatial resolution , 2015 .

[70]  Ayse Irmak,et al.  Satellite‐based ET estimation in agriculture using SEBAL and METRIC , 2011 .

[71]  Bo Huang,et al.  A generalization of spatial and temporal fusion methods for remotely sensed surface parameters , 2015 .

[72]  Baojun Zhao,et al.  Fast and Accurate Spatiotemporal Fusion Based Upon Extreme Learning Machine , 2016, IEEE Geoscience and Remote Sensing Letters.

[73]  Yee Leung,et al.  A Bayesian Data Fusion Approach to Spatio-Temporal Fusion of Remotely Sensed Images , 2017, Remote. Sens..

[74]  Bruce K. Wylie,et al.  Fusing MODIS with Landsat 8 data to downscale weekly normalized difference vegetation index estimates for central Great Basin rangelands, USA , 2018 .

[75]  Hongli Liu,et al.  An Improved STARFM with Help of an Unmixing-Based Method to Generate High Spatial and Temporal Resolution Remote Sensing Data in Complex Heterogeneous Regions , 2016, Sensors.

[76]  Luis Alonso,et al.  Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[77]  Desheng Liu,et al.  Blending MODIS and Landsat images for urban flood mapping , 2014 .

[78]  Yang Zheng,et al.  Crop Phenology Detection Using High Spatio-Temporal Resolution Data Fused from SPOT5 and MODIS Products , 2016, Sensors.

[79]  Yunyan Du,et al.  An integrated model for generating hourly Landsat-like land surface temperatures over heterogeneous landscapes , 2018 .

[80]  Bo Huang,et al.  Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[81]  Liangpei Zhang,et al.  An Error-Bound-Regularized Sparse Coding for Spatiotemporal Reflectance Fusion , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[82]  F. Javier García-Haro,et al.  A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion , 2015 .

[83]  Deyu Meng,et al.  Spatial and Temporal Image Fusion via Regularized Spatial Unmixing , 2015, IEEE Geoscience and Remote Sensing Letters.

[84]  Michael A. Lefsky,et al.  A flexible spatiotemporal method for fusing satellite images with different resolutions , 2016 .

[85]  Lizhe Wang,et al.  Spatiotemporal Fusion of Remote Sensing Images with Structural Sparsity and Semi-Coupled Dictionary Learning , 2016, Remote. Sens..

[86]  Mathew R. Schwaller,et al.  On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[87]  Yuyu Zhou,et al.  Response of vegetation phenology to urbanization in the conterminous United States , 2017, Global change biology.

[88]  Bo Huang,et al.  Spatiotemporal Reflectance Fusion via Sparse Representation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[89]  Tinghua Ai,et al.  A spatial and temporal reflectance fusion model considering sensor observation differences , 2013 .

[90]  Xin Du,et al.  Generation of high spatial and temporal resolution NDVI and its application in crop biomass estimation , 2013, Int. J. Digit. Earth.

[91]  Zhiliang Zhu,et al.  Downscaling of Surface Soil Moisture Retrieval by Combining MODIS/Landsat and In Situ Measurements , 2018, Remote. Sens..

[92]  Xudong Guan,et al.  An Object-Based Linear Weight Assignment Fusion Scheme to Improve Classification Accuracy Using Landsat and MODIS Data at the Decision Level , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[93]  Joanne C. White,et al.  A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS , 2009 .

[94]  Peter M. Atkinson,et al.  Enhancing Spatio-Temporal Fusion of MODIS and Landsat Data by Incorporating 250 m MODIS Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[95]  Mingquan Wu,et al.  High-resolution Leaf Area Index estimation from synthetic Landsat data generated by a spatial and temporal data fusion model , 2015, Comput. Electron. Agric..

[96]  Xiaoxia Wang,et al.  Estimating Fractional Vegetation Cover From Landsat-7 ETM+ Reflectance Data Based on a Coupled Radiative Transfer and Crop Growth Model , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[97]  Li Zhang,et al.  Application of Synthetic NDVI Time Series Blended from Landsat and MODIS Data for Grassland Biomass Estimation , 2015, Remote. Sens..

[98]  Jin Chen,et al.  A new geostatistical approach for filling gaps in Landsat ETM+ SLC-off images , 2012 .

[99]  Kirsten M. de Beurs,et al.  Monitoring forest disturbances in Southeast Oklahoma using Landsat and MODIS images , 2016, Int. J. Appl. Earth Obs. Geoinformation.

[100]  Le Yu,et al.  High Resolution Mapping of Cropping Cycles by Fusion of Landsat and MODIS Data , 2017, Remote. Sens..

[101]  Liang-pei Zhang,et al.  Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature , 2015 .

[102]  D. Roy,et al.  The availability of cloud-free Landsat ETM+ data over the conterminous United States and globally , 2008 .

[103]  William J. Emery,et al.  An Integrated Spatio-Spectral–Temporal Sparse Representation Method for Fusing Remote-Sensing Images With Different Resolutions , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[104]  N. Pettorelli,et al.  Satellite remote sensing for applied ecologists: opportunities and challenges , 2014 .