SFSDAF: An enhanced FSDAF that incorporates sub-pixel class fraction change information for spatio-temporal image fusion

Abstract Spatio-temporal image fusion methods have become a popular means to produce remotely sensed data sets that have both fine spatial and temporal resolution. Accurate prediction of reflectance change is difficult, especially when the change is caused by both phenological change and land cover class changes. Although several spatio-temporal fusion methods such as the Flexible Spatiotemporal DAta Fusion (FSDAF) directly derive land cover phenological change information (such as endmember change) at different dates, the direct derivation of land cover class change information is challenging. In this paper, an enhanced FSDAF that incorporates sub-pixel class fraction change information (SFSDAF) is proposed. By directly deriving the sub-pixel land cover class fraction change information the proposed method allows accurate prediction even for heterogeneous regions that undergo a land cover class change. In particular, SFSDAF directly derives fine spatial resolution endmember change and class fraction change at the date of the observed image pair and the date of prediction, which can help identify image reflectance change resulting from different sources. SFSDAF predicts a fine resolution image at the time of acquisition of coarse resolution images using only one prior coarse and fine resolution image pair, and accommodates variations in reflectance due to both natural fluctuations in class spectral response (e.g. due to phenology) and land cover class change. The method is illustrated using degraded and real images and compared against three established spatio-temporal methods. The results show that the SFSDAF produced the least blurred images and the most accurate predictions of fine resolution reflectance values, especially for regions of heterogeneous landscape and regions that undergo some land cover class change. Consequently, the SFSDAF has considerable potential in monitoring Earth surface dynamics.

[1]  Qihao Weng,et al.  An evaluation of monthly impervious surface dynamics by fusing Landsat and MODIS time series in the Pearl River Delta, China, from 2000 to 2015 , 2017 .

[2]  Xiaolin Zhu,et al.  An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions , 2010 .

[3]  Bo Huang,et al.  Spatiotemporal Reflectance Fusion via Sparse Representation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[4]  H. Mooney,et al.  Human Domination of Earth’s Ecosystems , 1997, Renewable Energy.

[5]  O. Dubrule.  Comparing splines and kriging , 1984 .

[6]  Qingshan Liu,et al.  Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[7]  M. Schaepman,et al.  Downscaling time series of MERIS full resolution data to monitor vegetation seasonal dynamics , 2009 .

[8]  Joanne C. White,et al.  A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS , 2009 .

[9]  Dieter Oertel,et al.  Unmixing-based multisensor multiresolution image fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[10]  Bo Huang,et al.  Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Liangpei Zhang,et al.  An Error-Bound-Regularized Sparse Coding for Spatiotemporal Reflectance Fusion , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Yong Xu,et al.  A Spatio–Temporal Pixel-Swapping Algorithm for Subpixel Land Cover Mapping , 2014, IEEE Geoscience and Remote Sensing Letters.

[13]  F. Javier García-Haro,et al.  A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion , 2015 .

[14]  John F. Mustard,et al.  Spectral unmixing , 2002, IEEE Signal Process. Mag..

[15]  Huihui Song,et al.  A robust adaptive spatial and temporal image fusion model for complex land surface changes , 2018 .

[16]  Peter M. Atkinson,et al.  Spatio-temporal fusion for daily Sentinel-2 images , 2018 .

[17]  Mingquan Wu,et al.  Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model , 2012 .

[18]  Feng Ling,et al.  Sub-pixel-scale land cover map updating by integrating change detection and sub-pixel mapping. , 2015 .

[19]  G. Foody,et al.  Generating a series of fine spatial and temporal resolution land cover maps by fusing coarse spatial resolution remotely sensed images and fine spatial resolution land cover maps , 2017 .

[20]  Mathew R. Schwaller,et al.  On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Bo Huang,et al.  A Rigorously-Weighted Spatiotemporal Fusion Model with Uncertainty Analysis , 2017, Remote. Sens..

[22]  Wenzhong Shi,et al.  Spatiotemporal Subpixel Mapping of Time-Series Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Meng Liu,et al.  An Improved Flexible Spatiotemporal DAta Fusion (IFSDAF) method for producing high spatiotemporal resolution normalized difference vegetation index time series , 2019, Remote Sensing of Environment.

[24]  Alfred Stein,et al.  Spatiotemporal Image Fusion in Remote Sensing , 2019, Remote. Sens..

[25]  D. Roy,et al.  The availability of cloud-free Landsat ETM+ data over the conterminous United States and globally , 2008 .

[26]  Tim R. McVicar,et al.  Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection , 2013 .

[27]  Giles M. Foody,et al.  Spatial-temporal fraction map fusion with multi-scale remotely sensed images , 2018, Remote Sensing of Environment.

[28]  Baojun Zhao,et al.  StfNet: A Two-Stream Convolutional Neural Network for Spatiotemporal Image Fusion , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Fuqun Zhou,et al.  Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions , 2019, Remote. Sens..

[30]  Hankui K. Zhang,et al.  Spatio-temporal reflectance fusion via unmixing: accounting for both phenological and land-cover changes , 2014 .

[31]  Xiaodong Li,et al.  Land Cover Change Mapping at the Subpixel Scale With Different Spatial-Resolution Remotely Sensed Imagery , 2011, IEEE Geoscience and Remote Sensing Letters.

[32]  Thomas Hilker,et al.  An Improved Image Fusion Approach Based on Enhanced Spatial and Temporal the Adaptive Reflectance Fusion Model , 2013, Remote. Sens..

[33]  Martha C. Anderson,et al.  Toward mapping crop progress at field scales through fusion of Landsat and MODIS imagery , 2017 .

[34]  Fernando Pérez-Cabello,et al.  Fusing Landsat and MODIS data to retrieve multispectral information from fire-affected areas over tropical savannah environments in the Brazilian Amazon , 2018, International Journal of Remote Sensing.

[35]  Wenzhong Shi,et al.  A spatio-temporal fusion method for remote sensing data Using a linear injection model and local neighbourhood information , 2018, International Journal of Remote Sensing.

[36]  F. Chapin,et al.  Consequences of changing biodiversity , 2000, Nature.

[37]  Hanqiu Xu.  Modification of normalised difference water index (NDWI) to enhance open water features in remotely sensed imagery , 2006 .

[38]  K. Beurs,et al.  Evaluation of Landsat and MODIS data fusion products for analysis of dryland forest phenology , 2012 .

[39]  Xiaolin Zhu,et al.  Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions , 2018, Remote. Sens..

[40]  Chunhua Liao,et al.  A Spatio-Temporal Data Fusion Model for Generating NDVI Time Series in Heterogeneous Regions , 2017, Remote. Sens..

[41]  Ethem Alpaydin,et al.  Soft vector quantization and the EM algorithm , 1998, Neural Networks.

[42]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[43]  Jan Verbesselt,et al.  Multi-resolution time series imagery for forest disturbance and regrowth monitoring in Queensland, Australia , 2015 .

[44]  Peter M. Atkinson,et al.  Enhancing Spatio-Temporal Fusion of MODIS and Landsat Data by Incorporating 250 m MODIS Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[45]  Luis Alonso,et al.  Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[46]  Bin Chen,et al.  A hierarchical spatiotemporal adaptive fusion model using one image pair , 2017, Int. J. Digit. Earth.

[47]  Michael E. Schaepman,et al.  Unmixing-Based Landsat TM and MERIS FR Data Fusion , 2008, IEEE Geoscience and Remote Sensing Letters.

[48]  Michael A. Lefsky,et al.  A flexible spatiotemporal method for fusing satellite images with different resolutions , 2016 .

[49]  B. Kleinschmit,et al.  The benefit of synthetically generated RapidEye and Landsat 8 data fusion time series for riparian forest disturbance monitoring , 2016 .

[50]  Giles M. Foody,et al.  A Superresolution Land-Cover Change Detection Method Using Remotely Sensed Images With Different Spatial Resolutions , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Marta Chiesi,et al.  A New Method to Enhance the Spatial Features of Multitemporal NDVI Image Series , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[52]  S. Carpenter,et al.  Global Consequences of Land Use , 2005, Science.

[53]  Lifan Chen,et al.  Dynamic monitoring of the Poyang Lake wetland by integrating Landsat and MODIS observations , 2018 .