Integrating the Continuous Wavelet Transform and a Convolutional Neural Network to Identify Vineyard Using Time Series Satellite Images

Grape is an economic crop of great importance and is widely cultivated in China. With the development of remote sensing, abundant data sources strongly guarantee that researchers can identify crop types and map their spatial distributions. However, to date, only a few studies have been conducted to identify vineyards using satellite image data. In this study, a vineyard is identified using satellite images, and a new approach is proposed that integrates the continuous wavelet transform (CWT) and a convolutional neural network (CNN). Specifically, the original time series of the normalized difference vegetation index (NDVI), enhanced vegetation index (EVI), and green chlorophyll vegetation index (GCVI) are reconstructed by applying an iterated Savitzky-Golay (S-G) method to form a daily time series for a full year; then, the CWT is applied to three reconstructed time series to generate corresponding scalograms; and finally, CNN technology is used to identify vineyards based on the stacked scalograms. In addition to our approach, a traditional and common approach that uses a random forest (RF) to identify crop types based on multi-temporal images is selected as the control group. The experimental results demonstrated the following: (i) the proposed approach was comprehensively superior to the RF approach; it improved the overall accuracy by 9.87% (up to 89.66%); (ii) the CWT had a stable and effective influence on the reconstructed time series, and the scalograms fully represented the unique time-related frequency pattern of each of the planting conditions; and (iii) the convolution and max pooling processing of the CNN captured the unique and subtle distribution patterns of the scalograms to distinguish vineyards from other crops. Additionally, the proposed approach is considered as able to be applied to other practical scenarios, such as using time series data to identify crop types, map landcover/land use, and is recommended to be tested in future practical applications.

[1]  Jin Chen,et al.  A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky-Golay filter , 2004 .

[2]  Xiao Xiang Zhu,et al.  Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Hugo Carrão,et al.  A Nonlinear Harmonic Model for Fitting Satellite Image Time Series: Analysis and Prediction of Land Cover Dynamics , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[5]  A. Huete,et al.  Overview of the radiometric and biophysical performance of the MODIS vegetation indices , 2002 .

[6]  Xiang Zhao,et al.  Long Time Series Land Cover Classification in China from 1982 to 2015 Based on Bi-LSTM Deep Learning , 2019, Remote. Sens..

[7]  R. Nemani,et al.  Mapping vineyard leaf area with multispectral satellite imagery , 2003 .

[8]  Geoffrey I. Webb,et al.  Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series , 2018, Remote. Sens..

[9]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[10]  Frédéric Baret,et al.  Vineyard identification and description of spatial crop structure by per-field frequency analysis , 2002 .

[11]  Tom Milliman,et al.  Detection of Large-Scale Forest Canopy Change in Pan-Tropical Humid Forests 2000–2009 With the SeaWinds Ku-Band Scatterometer , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Joanne C. White,et al.  Optical remotely sensed time series data for land cover classification: A review , 2016 .

[13]  Gaohuan Liu,et al.  Evaluation and Comparison of Random Forest and A-LSTM Networks for Large-scale Winter Wheat Identification , 2019, Remote. Sens..

[14]  F. M. Lacar,et al.  Use of hyperspectral imagery for mapping grape varieties in the Barossa Valley, South Australia , 2001, IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No.01CH37217).

[15]  Maoguo Gong,et al.  Feature learning and change feature classification based on deep learning for ternary change detection in SAR images , 2017 .

[16]  Conghe Song,et al.  Long-term land cover dynamics by multi-temporal classification across the Landsat-5 record , 2013 .

[17]  Clement Atzberger,et al.  How much does multi-temporal Sentinel-2 data improve crop type classification? , 2018, Int. J. Appl. Earth Obs. Geoinformation.

[18]  Carreño Carreño,et al.  Evaluación de la diversidad taxonómica y funcional de la comunidad microbiana relacionada con el ciclo del nitrógeno en suelos de cultivo de arroz con diferentes manejos del tamo , 2020 .

[19]  María-Paz Diago,et al.  Using RPAS Multi-Spectral Imagery to Characterise Vigour, Leaf Development, Yield Components and Berry Composition Variability within a Vineyard , 2015, Remote. Sens..

[20]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  André R. S. Marçal,et al.  Very early prediction of wine yield based on satellite data from VEGETATION , 2010 .

[22]  Elif Sertel,et al.  Vineyard parcel identification from Worldview-2 images using object-based classification model , 2014 .

[23]  John R. Miller,et al.  Assessing vineyard condition with hyperspectral indices: Leaf and canopy reflectance simulation in a row-structured discontinuous canopy , 2005 .

[24]  Giorgos Mountrakis,et al.  A meta-analysis of remote sensing research on supervised pixel-based land-cover image classification processes: General guidelines for practitioners and future research , 2016 .

[25]  David Morin,et al.  Operational High Resolution Land Cover Map Production at the Country Scale Using Satellite Image Time Series , 2017, Remote. Sens..

[26]  Ying Li,et al.  Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network , 2017, Remote. Sens..

[27]  T. Sakamoto,et al.  A crop phenology detection method using time-series MODIS data , 2005 .

[28]  Qi Li,et al.  Hyperspectral Imagery Classification Using Sparse Representations of Convolutional Neural Network Features , 2016, Remote. Sens..

[29]  Dino Ienco,et al.  Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Konstantinos Karantzalos,et al.  Vineyard Detection and Vine Variety Discrimination from Very High Resolution Satellite Data , 2016, Remote. Sens..

[31]  Pablo J. Zarco-Tejada,et al.  Grape quality assessment in vineyards affected by iron deficiency chlorosis using narrow-band physiological remote sensing indices , 2010 .

[32]  Matthew Bardeen,et al.  Detection and Segmentation of Vine Canopy in Ultra-High Spatial Resolution RGB Imagery Obtained from Unmanned Aerial Vehicle (UAV): A Case Study in a Commercial Vineyard , 2017, Remote. Sens..

[33]  Feng Gao,et al.  Daily Mapping of 30 m LAI and NDVI for Grape Yield Prediction in California Vineyards , 2017, Remote. Sens..

[34]  I. Daubechies.  Ten Lectures on Wavelets , 1992 .

[35]  Gang Fu,et al.  Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network , 2017, Remote. Sens..

[36]  P. Couteron,et al.  Textural approaches for vineyard detection and characterization using very high spatial resolution remote sensing data , 2008 .

[37]  Pablo J. Zarco-Tejada,et al.  Detecting water stress effects on fruit quality in orchards with time-series PRI airborne imagery , 2010 .

[38]  Amy Loutfi,et al.  Classification and Segmentation of Satellite Orthoimagery Using Convolutional Neural Networks , 2016, Remote. Sens..

[39]  Imed Riadh Farah,et al.  Wavelet Transform Application for/in Non-Stationary Time-Series Analysis: A Review , 2019, Applied Sciences.

[40]  Liping Di,et al.  Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series , 2018, International Journal of Remote Sensing.

[41]  Yun Shi,et al.  3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images , 2018, Remote. Sens..

[42]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[43]  D. Lamb,et al.  Using remote sensing to predict grape phenolics and colour at harvest in a Cabernet Sauvignon vineyard: Timing observations against vine phenology and optimising image resolution , 2008 .

[44]  Andrea Berton,et al.  Estimation of Water Stress in Grapevines Using Proximal and Remote Sensing Methods , 2018, Remote. Sens..

[45]  A. Viña,et al.  Remote estimation of leaf area index and green leaf biomass in maize canopies , 2003 .

[46]  Carole Delenne,et al.  An Automatized Frequency Analysis for Vine Plot Detection and Delineation in Remote Sensing , 2008, IEEE Geoscience and Remote Sensing Letters.

[47]  Aiman Soliman,et al.  Remote Sensing of Soil Moisture in Vineyards Using Airborne and Ground-Based Thermal Inertia Data , 2013, Remote. Sens..

[48]  Nataliia Kussul,et al.  Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[49]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Marcello Chiaberge,et al.  Comparison of Satellite and UAV-Based Multispectral Imagery for Vineyard Variability Assessment , 2019, Remote. Sens..

[51]  Peijuan Wang,et al.  Fusing Landsat and MODIS Data for Vegetation Monitoring , 2015, IEEE Geoscience and Remote Sensing Magazine.

[52]  Jayantrao Mohite,et al.  Hybrid classification-clustering approach for export-non export grape area mapping and health estimation using Sentinel-2 satellite data , 2017, 2017 6th International Conference on Agro-Geoinformatics.

[53]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Aaron O'Leary,et al.  PyWavelets: A Python package for wavelet analysis , 2019, J. Open Source Softw..

[55]  Alessandro Matese,et al.  Airborne high‐resolution images for grape classification: changes in correlation between technological and late maturity in a Sangiovese vineyard in Central Italy , 2012 .

[56]  Gerald Kaiser,et al.  A Friendly Guide to Wavelets , 1994 .

[57]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[58]  C. Tucker.  Red and photographic infrared linear combinations for monitoring vegetation , 1979 .

[59]  Paolo Cinat,et al.  Comparison of Unsupervised Algorithms for Vineyard Canopy Segmentation from UAV Multispectral Images , 2019, Remote. Sens..

[60]  Qing Wang,et al.  Change detection based on Faster R-CNN for high-resolution remote sensing images , 2018, Remote Sensing Letters.

[61]  Matthias Drusch,et al.  Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services , 2012 .

[62]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[63]  Clement Atzberger,et al.  A time series for monitoring vegetation activity and phenology at 10-daily time steps covering large parts of South America , 2011, Int. J. Digit. Earth.

[64]  A. Mathur,et al.  Denoising and wavelet-based feature extraction of MODIS multi-temporal vegetation signatures , 2005, International Workshop on the Analysis of Multi-Temporal Remote Sensing Images, 2005..

[65]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[66]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[67]  Feng Gao,et al.  Determining a robust indirect measurement of leaf area index in California vineyards for validating remote sensing-based retrievals , 2018, Irrigation Science.

[68]  Susan L. Ustin,et al.  Remote estimation of vine canopy density in vertically shoot‐positioned vineyards: determining optimal vegetation indices , 2002 .

[69]  Nitesh K. Poona,et al.  Modelling Water Stress in a Shiraz Vineyard Using Hyperspectral Imaging and Machine Learning , 2018, Remote. Sens..