Deep learning models to map an agricultural expansion area with MODIS and Sentinel-2 time series images

Abstract. Mapping changing land use and land cover (LULC) is important for land management and environment analysis. We tried to build deep learning models to classify LULC over time at an agricultural expansion area in Matopiba region, Brazil with MCD43A4 V006 moderate resolution imaging spectroradiometer (MODIS), and Sentinel-2 multispectral instrument (MSI) time series data. We collected time series MODIS data and Sentinel-2 A/B MSI data from 2015 to 2020 and prepared small patches with containing blue, green, red, near-infrared, and shortwave-infrared-1 bands as features. Then the both datasets were used to build and train the convolutional neural network (CNN) model, the CNN gate recurrent unit (CNN-GRU) model, and the CNN long short-term memory (CNN-LSTM) model, respectively. We evaluated these three trained models with ground truth data, and the CNN-LSTM model (overall accuracy: 91.29% from MODIS data and 89.47% from Sentinel-2 data) was better than the CNN-GRU model (overall accuracy: 89.19% from MODIS data and 88.61% from Sentinel-2 data) and the CNN model (overall accuracy: 89.17% from MODIS data and 86.02% from Sentinel-2 data). Our results also showed that the accuracy from cropland and savanna classes were higher than grassland and forest classes in all three models. These two classes generated from the CNN-LSTM model performed better than the other two deep learning models. The results from these two datasets indicated that the methods were reliable for both coarse and medium spatial resolution satellite images and time series remote sensing images worked better than single image for classification problems when considering LULC change over time. The results also provided an alternative way to prepare input data from satellite images for deep learning models. Furthermore, the classification results of the whole agricultural expansion area were reasonable and it can be used as an additional dataset for further environmental analysis at a regional scale.

[1]  P. Brando,et al.  Climatic limit for agriculture in Brazil , 2021, Nature Climate Change.

[2]  Adugna G. Mullissa,et al.  Spatial and temporal deep learning methods for deriving land-use following deforestation: A pan-tropical case study using Landsat time series , 2021 .

[3]  Wadii Boulila,et al.  A Novel CNN-LSTM-based Approach to Predict Urban Expansion , 2021, Ecol. Informatics.

[4]  Jan Dirk Wegner,et al.  Crop mapping from image time series: deep learning with multi-scale label hierarchies , 2021, ArXiv.

[5]  Edson Eyji Sano,et al.  Mapping Three Decades of Changes in the Brazilian Savanna Native Vegetation Using Landsat Data Processed in the Google Earth Engine Platform , 2020, Remote. Sens..

[6]  Dong Luo,et al.  Spatial-Temporal Analysis of Land Cover Change at the Bento Rodrigues Dam Disaster Area Using Machine Learning Techniques , 2019, Remote. Sens..

[7]  Jie Sun,et al.  County-Level Soybean Yield Prediction Using Deep CNN-LSTM Model , 2019, Sensors.

[8]  Simone A. Ludwig,et al.  Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM and GRU , 2019, J. Artif. Intell. Soft Comput. Res..

[9]  M. Bustamante,et al.  Cerrado ecoregions: A spatial framework to assess and prioritize Brazilian savanna environmental diversity for conservation. , 2019, Journal of environmental management.

[10]  Xin Pan,et al.  Joint Deep Learning for land cover and land use classification , 2019, Remote Sensing of Environment.

[11]  E. Sano,et al.  Spatiotemporal dynamics of soybean crop in the Matopiba region, Brazil (1990–2015) , 2019, Land Use Policy.

[12]  Liping Di,et al.  Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series , 2018, International Journal of Remote Sensing.

[13]  Bo Huang,et al.  Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery , 2018, Remote Sensing of Environment.

[14]  Emile Ndikumana,et al.  Deep Recurrent Neural Network for Agricultural Classification using multitemporal SAR Sentinel-1 for Camargue, France , 2018, Remote. Sens..

[15]  Chen Li,et al.  Spatial Sequential Recurrent Neural Network for Hyperspectral Image Classification , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Cheng Shi,et al.  Multi-scale hierarchical recurrent neural networks for hyperspectral image classification , 2018, Neurocomputing.

[17]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[18]  C. Schaaf,et al.  Capturing rapid land surface dynamics with Collection V006 MODIS BRDF/NBAR/Albedo (MCD43) products , 2018 .

[19]  Yun Shi,et al.  3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images , 2018, Remote. Sens..

[20]  Antonio Plaza,et al.  A new deep convolutional neural network for fast hyperspectral image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[21]  Xin Pan,et al.  A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[22]  J. I. B. D. Brito,et al.  Trend of annual temperature and frequency of extreme events in the MATOPIBA region of Brazil , 2018, Theoretical and Applied Climatology.

[23]  Xiuwen Liu,et al.  A patch-based convolutional neural network for remote sensing image classification , 2017, Neural Networks.

[24]  Xiao Xiang Zhu,et al.  Deep learning in remote sensing: a review , 2017, ArXiv.

[25]  B. Strassburg,et al.  Hard times for the Brazilian environment , 2017, Nature Ecology & Evolution.

[26]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[27]  Dino Ienco,et al.  Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[28]  Jianxin Li,et al.  Incrementally Learning the Hierarchical Softmax Function for Neural Language Models , 2017, AAAI.

[29]  Fathi M. Salem,et al.  Gate-variants of Gated Recurrent Unit (GRU) neural networks , 2017, 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS).

[30]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[31]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Chengquan Huang,et al.  Forest carbon emissions from cropland expansion in the Brazilian Cerrado biome , 2017 .

[33]  Antonio J. Plaza,et al.  Adaptive Deep Pyramid Matching for Remote Sensing Scene Classification , 2016, ArXiv.

[34]  Xiangzheng Deng,et al.  A review on historical trajectories and spatially explicit scenarios of land-use and land-cover changes in China , 2016 .

[35]  Enrico Cadau,et al.  SENTINEL-2 SEN2COR: L2A Processor for Users , 2016 .

[36]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Pierre Alliez,et al.  Fully convolutional neural networks for remote sensing image classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[38]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[39]  Mariana Belgiu,et al.  Random forest in remote sensing: A review of applications and future directions , 2016 .

[40]  Shihong Du,et al.  Learning multiscale and deep representations for classifying remotely sensed imagery , 2016 .

[41]  Dong Liang,et al.  Evaluation of the Consistency of MODIS Land Cover Product (MCD12Q1) Based on Chinese 30 m GlobeLand30 Datasets: A Case Study in Anhui Province, China , 2015, ISPRS Int. J. Geo Inf..

[42]  Suming Jin,et al.  Completion of the 2011 National Land Cover Database for the Conterminous United States – Representing a Decade of Land Cover Change Information , 2015 .

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  M. Watanabe,et al.  Fertilizer Consumption in the Region MATOPIBA and Their Reflections on Brazilian Soybean Production , 2015 .

[45]  Antonio Roberto Formaggio,et al.  Land use and land cover changes in the Brazilian Cerrado: A multidisciplinary approach to assess the impacts of agricultural expansion , 2014 .

[46]  Victor F. Rodriguez-Galiano,et al.  Evaluation of different machine learning methods for land cover mapping of a Mediterranean area using multi-seasonal Landsat images and Digital Terrain Models , 2014, Int. J. Digit. Earth.

[47]  Eric F. Lambin,et al.  Globalization of land use: distant drivers of land change and geographic displacement of land use , 2013 .

[48]  Nicholas E. Rada.  Assessing Brazil’s Cerrado agricultural miracle , 2013 .

[49]  Yang Shao,et al.  Comparison of support vector machine, neural network, and CART algorithms for the land-cover classification using limited training data points , 2012 .

[50]  Mario Chica-Olmo,et al.  An assessment of the effectiveness of a random forest classifier for land-cover classification , 2012 .

[51]  Stefan W. Maier,et al.  Comparing object-based and pixel-based classifications for mapping savannas , 2011, Int. J. Appl. Earth Obs. Geoinformation.

[52]  Jungho Im,et al.  ISPRS Journal of Photogrammetry and Remote Sensing , 2022 .

[53]  Edson E. Sano,et al.  Land cover mapping of the tropical savanna region in Brazil , 2010, Environmental monitoring and assessment.

[54]  S. Robinson,et al.  Food Security: The Challenge of Feeding 9 Billion People , 2010, Science.

[55]  Thomas Blaschke,et al.  Land cover change assessment using decision trees, support vector machines and maximum likelihood classification algorithms , 2010, Int. J. Appl. Earth Obs. Geoinformation.

[56]  Taskin Kavzoglu,et al.  A kernel functions analysis for support vector machines for land cover classification , 2009, Int. J. Appl. Earth Obs. Geoinformation.

[57]  Janet Franklin,et al.  Mapping land-cover modifications over large areas: A comparison of machine learning algorithms , 2008 .

[58]  Sotiris B. Kotsiantis,et al.  Machine learning: a review of classification and combining techniques , 2006, Artificial Intelligence Review.

[59]  Pramod K. Varshney,et al.  Decision tree regression for soft classification of remote sensing data , 2005 .

[60]  Paul M. Mather,et al.  Support vector machines for classification in remote sensing , 2005 .

[61]  Paul M. Mather,et al.  An assessment of the effectiveness of decision tree methods for land cover classification , 2003 .

[62]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .

[63]  L. S. Davis,et al.  An assessment of support vector machines for land cover classi(cid:142) cation , 2002 .

[64]  J. A. Ratter,et al.  The Brazilian Cerrado Vegetation and Threats to its Biodiversity , 1997 .