HISTIF: A New Spatiotemporal Image Fusion Method for High-Resolution Monitoring of Crops at the Subfield Level

Satellite-based time-series crop monitoring at the subfield level is essential to the efficient implementation of precision crop management. Existing spatiotemporal image fusion techniques can be helpful, but they were often proposed to generate medium-resolution images. This study proposed a high-resolution spatiotemporal image fusion method (HISTIF) consisting of filtering for cross-scale spatial matching (FCSM) and multiplicative modulation of temporal change (MMTC). In FCSM, we considered both point spread function effect and geo-registration errors between fine and coarse resolution images. Subsequently, MMTC used pixel-based multiplicative factors to estimate the temporal change between reference and prediction dates without image classification. The performance of HISTIF was evaluated using both simulated and real datasets with one from real Gaofen-1 (GF-1) and simulated Landsat-like/Sentinel-like images, and the other from real GF-1 and real Landsat/Sentinel-2 data on two sites. HISTIF was compared with the existing methods spatial and temporal adaptive reflectance fusion model (STARFM), FSDAF, and Fit-FC. The results demonstrated that HISTIF produced substantial reduction in the fusion error from cross-scale spatial mismatch and accurate reconstruction in spatial details within fields, regardless of simulated or real data. The images predicted by STARFM exhibited pronounced blocky artifacts. While the images predicted by HISTIF and Fit-FC both showed clear within-field variability patterns, HISTIF was able to reduce the spectral distortion more significantly than Fit-FC. Furthermore, HISTIF exhibited the most stable performance across sensors. The findings suggest that HISTIF could be beneficial for the frequent and detailed monitoring of crop growth at the subfield level.

[1]  Peijuan Wang,et al.  Operational Data Fusion Framework for Building Frequent Landsat-Like Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Peter M. Atkinson,et al.  Enhancing Spatio-Temporal Fusion of MODIS and Landsat Data by Incorporating 250 m MODIS Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  David B. Lobell,et al.  Mapping Smallholder Wheat Yields and Sowing Dates Using Micro-Satellite Data , 2016, Remote. Sens..

[4]  Song Qin,et al.  High-Resolution Satellite Observations of a New Hazard of Golden Tides Caused by Floating Sargassum in Winter in the Yellow Sea , 2017, IEEE Geoscience and Remote Sensing Letters.

[5]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[6]  A. Cracknell Review article Synergy in remote sensing-what's in a pixel? , 1998 .

[7]  B. Markham The Landsat Sensors' Spatial Responses , 1985, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Lei Luo,et al.  Identifying Linear Traces of the Han Dynasty Great Wall in Dunhuang Using Gaofen-1 Satellite Remote Sensing Imagery and the Hough Transform , 2019, Remote. Sens..

[9]  F. Baret,et al.  Crop specific green area index retrieval from MODIS data at regional scale by controlling pixel-target adequacy , 2011 .

[10]  V. Demarez,et al.  Combining hectometric and decametric satellite observations to provide near real time decametric FAPAR product , 2017 .

[11]  Bo Huang,et al.  A generalization of spatial and temporal fusion methods for remotely sensed surface parameters , 2015 .

[12]  Wei Cai,et al.  A Novel Pixel-Level and Feature-Level Combined Multisensor Image Fusion Scheme , 2008, ISNN.

[13]  Luis Alonso,et al.  Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[14]  Chunlin Huang,et al.  Mapping daily evapotranspiration based on spatiotemporal fusion of ASTER and MODIS images over irrigated agricultural areas in the Heihe River Basin, Northwest China , 2017 .

[15]  Martha C. Anderson,et al.  Toward mapping crop progress at field scales through fusion of Landsat and MODIS imagery , 2017 .

[16]  Bin Chen,et al.  A hierarchical spatiotemporal adaptive fusion model using one image pair , 2017, Int. J. Digit. Earth.

[17]  Zbigniew Michalewicz,et al.  Particle Swarm Optimization for Single Objective Continuous Space Problems: A Review , 2017, Evolutionary Computation.

[18]  Dieter Oertel,et al.  Unmixing-based multisensor multiresolution image fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[19]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[20]  F. J. García-Haro,et al.  Multitemporal and multiresolution leaf area index retrieval for operational local rice crop monitoring , 2016 .

[21]  Martha C. Anderson,et al.  Field-scale mapping of evaporative stress indicators of crop yield: An application over Mead, NE, USA , 2018, Remote Sensing of Environment.

[22]  Qiming Qin,et al.  Scaling Correction of Remotely Sensed Leaf Area Index for Farmland Landscape Pattern With Multitype Spatial Heterogeneities Using Fractal Dimension and Contextural Parameters , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  D. Roy,et al.  Multi-temporal MODIS-Landsat data fusion for relative radiometric normalization, gap filling, and prediction of Landsat data , 2008 .

[24]  Michael E. Schaepman,et al.  Unmixing-Based Landsat TM and MERIS FR Data Fusion , 2008, IEEE Geoscience and Remote Sensing Letters.

[25]  F. Gao,et al.  Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data , 2014 .

[26]  Huihui Song,et al.  A robust adaptive spatial and temporal image fusion model for complex land surface changes , 2018 .

[27]  Robert A. Schowengerdt,et al.  Remote sensing, models, and methods for image processing , 1997 .

[28]  Meng Liu,et al.  An Improved Flexible Spatiotemporal DAta Fusion (IFSDAF) method for producing high spatiotemporal resolution normalized difference vegetation index time series , 2019, Remote Sensing of Environment.

[29]  José A. Sobrino,et al.  Improved land surface emissivities over agricultural areas using ASTER NDVI , 2006 .

[30]  Xiaolin Zhu,et al.  Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions , 2018, Remote. Sens..

[31]  Yuzhong Shen,et al.  Assessment of Spatiotemporal Fusion Algorithms for Planet and Worldview Images , 2018, Sensors.

[32]  O. Hagolle,et al.  The MODIS (collection V006) BRDF/albedo product MCD43D: Temporal course evaluated over agricultural landscape , 2015 .

[33]  Mathew R. Schwaller,et al.  On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[35]  Bo Huang,et al.  Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Chiman Kwan,et al.  A Super-Resolution and Fusion Approach to Enhancing Hyperspectral Images , 2018, Remote. Sens..

[37]  F. Javier García-Haro,et al.  A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion , 2015 .

[38]  J. Townshend,et al.  Impact of sensor's point spread function on land cover characterization: Assessment and deconvolution , 2002 .

[39]  Xiaolin Zhu,et al.  An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions , 2010 .

[40]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[41]  Xiaolin Zhu,et al.  "Blend-then-Index" or "Index-then-Blend": A Theoretical Analysis for Generating High-resolution NDVI Time Series by STARFM , 2018 .

[42]  Michael A. Lefsky,et al.  A flexible spatiotemporal method for fusing satellite images with different resolutions , 2016 .

[43]  Peter M. Atkinson,et al.  Spatio-temporal fusion for daily Sentinel-2 images , 2018 .

[44]  Lei Luo,et al.  Auto-Extraction of Linear Archaeological Traces of Tuntian Irrigation Canals in Miran Site (China) from Gaofen-1 Satellite Imagery , 2018, Remote. Sens..

[45]  Bo Huang,et al.  Spatiotemporal Reflectance Fusion via Sparse Representation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Xiangnan Liu,et al.  A Modified Spatiotemporal Fusion Algorithm Using Phenological Information for Predicting Reflectance of Paddy Rice in Southern China , 2018, Remote. Sens..

[47]  D. Lobell,et al.  Satellite-based assessment of yield variation and its determinants in smallholder African systems , 2017, Proceedings of the National Academy of Sciences.

[48]  Joanne C. White,et al.  A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS , 2009 .

[49]  Jose Gomez-Dans,et al.  A sensor-invariant atmospheric correction method: application to Sentinel-2/MSI and Landsat 8/OLI , 2019 .

[50]  Weixing Cao,et al.  Evaluation of Three Techniques for Correcting the Spatial Scaling Bias of Leaf Area Index , 2018, Remote. Sens..

[51]  Huajun Tang,et al.  Perspective of Chinese GF-1 high-resolution satellite data in agricultural remote sensing monitoring , 2017 .

[52]  Pierre Defourny,et al.  Retrieval of the canopy chlorophyll content from Sentinel-2 spectral bands to estimate nitrogen uptake in intensive winter wheat cropping systems , 2018, Remote Sensing of Environment.

[53]  Yang Hong,et al.  A lake data set for the Tibetan Plateau from the 1960s, 2005, and 2014 , 2016, Scientific Data.

[54]  Bo Wu,et al.  Improving spatiotemporal reflectance fusion using image inpainting and steering kernel regression techniques , 2017 .

[55]  Shaowen Wang,et al.  A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach , 2018, Remote Sensing of Environment.

[56]  Feng Gao,et al.  A Hybrid Color Mapping Approach to Fusing MODIS and Landsat Images for Forward Prediction , 2018, Remote. Sens..

[57]  Chunhua Liao,et al.  A Spatio-Temporal Data Fusion Model for Generating NDVI Time Series in Heterogeneous Regions , 2017, Remote. Sens..

[58]  Jessica L. McCarty,et al.  Extracting smallholder cropped area in Tigray, Ethiopia with wall-to-wall sub-meter WorldView and moderate resolution Landsat 8 imagery , 2017 .