Detection and classification of soybean pests using deep learning with UAV images

Abstract This paper presents the results of the evaluation of five deep learning architectures for the classification of soybean pest images. The performance of Inception-v3, Resnet-50, VGG-16, VGG-19 and Xception was evaluated for different fine-tuning and transfer learning strategies over a dataset of 5,000 images captured in real field conditions. The experimental results showed that the deep learning architectures trained with a fine-tuning can lead to higher classification rates in comparison to other approaches, reaching accuracies of up to 93.82%. In addition, deep learning architectures outperformed traditional feature extraction methods, such as SIFT and SURF with Bag-of-Visual Words approach, the semi-supervised learning method OPFSEMImst, and supervised learning methods used to classify images, for example, SVM, k-NN and Random Forest. The results indicate that architectures evaluated can support specialists and farmers in the pest control management in soybean fields.

[1]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Jayme Garcia Arnal Barbedo,et al.  Using digital image processing for counting whiteflies on soybean leaves , 2014 .

[3]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Ricardo Dias Peruca,et al.  Impacts of soybean-induced defenses on Spodoptera frugiperda (Lepidoptera: Noctuidae) development , 2018, Arthropod-Plant Interactions.

[5]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[6]  Won Suk Lee,et al.  Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees , 2013 .

[7]  Pablo J. Zarco-Tejada,et al.  Early Detection and Quantification of Verticillium Wilt in Olive Using Hyperspectral and Thermal Imagery over Large Areas , 2015, Remote. Sens..

[8]  João Paulo Papa,et al.  Improving semi-supervised learning through optimum connectivity , 2016, Pattern Recognit..

[9]  J. Gai,et al.  Association analysis of vegetable soybean quality traits with SSR markers , 2011 .

[10]  Jafar Massah,et al.  Performance evaluation of a machine vision system for insect pests identification of field crops using artificial neural networks , 2013 .

[11]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[12]  Hemerson Pistori,et al.  Automatic Recognition of Soybean Leaf Diseases Using UAV Images and Deep Convolutional Neural Networks , 2020, IEEE Geoscience and Remote Sensing Letters.

[13]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[15]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[16]  Xanthoula Eirini Pantazi,et al.  Evaluation of hierarchical self-organising maps for weed mapping using UAS multispectral imagery , 2017, Comput. Electron. Agric..

[17]  Reza Ehsani,et al.  Detection of multi-tomato leaf diseases (late blight, target and bacterial spots) in different stages by using a spectral-based sensor , 2018, Scientific Reports.

[18]  João Paulo Papa,et al.  Semi-supervised learning with convolutional neural networks for UAV images automatic recognition , 2019, Comput. Electron. Agric..

[19]  Hemerson Pistori,et al.  Identification of Soybean Foliar Diseases Using Unmanned Aerial Vehicle Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[20]  G. Pate,et al.  Using Unmanned Aircraft Systems for Early Detection of Soybean Diseases , 2017 .

[21]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Anne-Katrin Mahlein,et al.  Recent advances in sensing plant diseases for precision crop protection , 2012, European Journal of Plant Pathology.

[23]  Panmanas Sirisomboon,et al.  Study on non-destructive evaluation methods for defect pods for green soybean processing by near-infrared spectroscopy. , 2009 .

[24]  S. M. Al-Saqer,et al.  Artificial Neural Networks Based Red Palm Weevil (Rynchophorus Ferrugineous, Olivier) Recognition System , 2011 .

[25]  Hemerson Pistori,et al.  Weed detection in soybean crops using ConvNets , 2017, Comput. Electron. Agric..

[26]  Qibing Zhu,et al.  Automatic threshold method and optimal wavelength selection for insect-damaged vegetable soybean detection using hyperspectral images , 2014 .

[27]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[28]  Sarinder Kaur Dhillon,et al.  Automated identification of copepods using digital image processing and artificial neural network , 2015, BMC Bioinformatics.

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[31]  Huajian Liu,et al.  A review of recent sensing technologies to detect invertebrates on crops , 2017, Precision Agriculture.

[32]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Sreekala G. Bajwa,et al.  Detection of soybean aphids in a greenhouse using an image processing technique , 2017, Comput. Electron. Agric..

[34]  V. Chelladurai,et al.  Detection of Callosobruchus maculatus (F.) infestation in soybean using soft X-ray and NIR hyperspectral imaging techniques , 2014 .

[35]  Hemerson Pistori,et al.  BioLeaf: A professional mobile application to measure foliar damage caused by insect herbivory , 2016, Comput. Electron. Agric..

[36]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[37]  J. Chahl,et al.  An evaluation of the contribution of ultraviolet in fused multispectral images for invertebrate detection on green leaves , 2017, Precision Agriculture.

[38]  Romain Raveaux,et al.  A survey on image-based insect classification , 2017, Pattern Recognit..

[39]  Hemerson Pistori,et al.  A Deep-Learning Approach for Automatic Counting of Soybean Insect Pests , 2020, IEEE Geoscience and Remote Sensing Letters.

[40]  Lakhmi C. Jain,et al.  An insect classification analysis based on shape features using quality threshold ARTMAP and moment invariant , 2011, Applied Intelligence.

[41]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Sang Cheol Kim,et al.  A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition , 2017, Sensors.

[43]  Felipe Leno da Silva,et al.  Evaluating classification and feature selection techniques for honeybee subspecies identification using wing images , 2015, Comput. Electron. Agric..

[44]  Miklós Dombos,et al.  An Opto-Electronic Sensor for Detecting Soil Microarthropods and Estimating Their Size in Field Conditions , 2017, Sensors.

[45]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[47]  U. Steiner,et al.  Journal of Experimental Botany Advance Access published May 19, 2006 Journal of Experimental Botany, Page 1 of 12 , 2022 .

[48]  Jiangning Wang,et al.  A new automatic identification system of insect images at the order level , 2012, Knowl. Based Syst..

[49]  R.M. Haralick,et al.  Statistical and structural approaches to texture , 1979, Proceedings of the IEEE.

[50]  Chenglu Wen,et al.  Pose estimation-dependent identification method for field moth images using deep learning architecture , 2015 .