Detecting and Classifying Pests in Crops Using Proximal Images and Machine Learning: A Review

Pest management is among the most important activities in a farm. Monitoring all different species visually may not be effective, especially in large properties. Accordingly, considerable research effort has been spent towards the development of effective ways to remotely monitor potential infestations. A growing number of solutions combine proximal digital images with machine learning techniques, but since species and conditions associated to each study vary considerably, it is difficult to draw a realistic picture of the actual state of the art on the subject. In this context, the objectives of this article are (1) to briefly describe some of the most relevant investigations on the subject of automatic pest detection using proximal digital images and machine learning; (2) to provide a unified overview of the research carried out so far, with special emphasis to research gaps that still linger; (3) to propose some possible targets for future research.

[1]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[2]  Sumio Kawano,et al.  Automatic image analysis and spot classification for detection of fruit fly infestation in hyperspectral images of mangoes , 2013 .

[3]  Yongxin Yang,et al.  Frankenstein: Learning Deep Face Representations Using Small Data , 2016, IEEE Transactions on Image Processing.

[4]  Eldert J. van Henten,et al.  Optimising realism of synthetic images using cycle generative adversarial networks for improved part segmentation , 2020, Comput. Electron. Agric..

[5]  Romain Raveaux,et al.  A survey on image-based insect classification , 2017, Pattern Recognit..

[6]  Xi Cheng,et al.  Pest identification via deep residual learning in complex background , 2017, Comput. Electron. Agric..

[7]  Jayme Garcia Arnal Barbedo,et al.  A review on the main challenges in automatic plant disease identification based on visible range images , 2016 .

[8]  Noel D.G. White,et al.  Detection techniques for stored-product insects in grain , 2007 .

[9]  Jayme Garcia Arnal Barbedo,et al.  Detection of nutrition deficiencies in plants using proximal images and machine learning: A review , 2019, Comput. Electron. Agric..

[10]  Zhongzhi Han,et al.  Research on insect pest image detection and recognition based on bio-inspired methods , 2018 .

[11]  V. Nalam,et al.  Plant defense against aphids, the pest extraordinaire. , 2019, Plant science : an international journal of experimental plant biology.

[12]  J. Yen,et al.  Evaluating the effectiveness of five sampling methods for detection of the tomato potato psyllid, Bactericera cockerelli (Šulc) (Hemiptera: Psylloidea: Triozidae) , 2013 .

[13]  Marcel Salathé,et al.  Using Deep Learning for Image-Based Plant Disease Detection , 2016, Front. Plant Sci..

[14]  Vincent Martin,et al.  A cognitive vision approach to early pest detection in greenhouse crops , 2008 .

[15]  Yao Zhou,et al.  A Vision-Based Counting and Recognition System for Flying Insects in Intelligent Agriculture , 2018, Sensors.

[16]  Saleh Mufleh Al-Saqer,et al.  Red Palm Weevil (Rynchophorus Ferrugineous, Olivier) Recognition by Image Processing Techniques , 2011 .

[17]  Tao Wang,et al.  Fast Detection of Striped Stem-Borer (Chilo suppressalis Walker) Infested Rice Seedling Based on Visible/Near-Infrared Hyperspectral Imaging System , 2017, Sensors.

[18]  Tae-Soo Chon,et al.  In situ detection of small-size insect pests sampled on traps using multifractal analysis , 2012 .

[19]  Yu Sun,et al.  Automatic in-trap pest detection using deep learning for pheromone-based Dendroctonus valens monitoring , 2018, Biosystems Engineering.

[20]  Yong He,et al.  Feasibility Study on a Portable Field Pest Classification System Design Based on DSP and 3G Wireless Communication Technology , 2012, Sensors.

[21]  Dmitry Bratanov,et al.  A Novel Methodology for Improving Plant Pest Surveillance in Vineyards and Crops Using UAV-Based Hyperspectral and Spatial Data , 2018, Sensors.

[22]  Chenglu Wen,et al.  Local feature-based identification and classification for orchard insects , 2009 .

[23]  Wang Xiaofeng,et al.  A Cognitive Vision Method for Insect Pest Image Segmentation , 2018 .

[24]  A. Urbaneja,et al.  Biological control using invertebrates and microorganisms: plenty of new opportunities , 2018, BioControl.

[25]  Clive H. Bock,et al.  Plant Disease Severity Estimated Visually, by Digital Photography and Image Analysis, and by Hyperspectral Imaging , 2010 .

[26]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[27]  Jafar Massah,et al.  Performance evaluation of a machine vision system for insect pests identification of field crops using artificial neural networks , 2013 .

[28]  Saeid Minaei,et al.  Vision-based pest detection based on SVM classification method , 2017, Comput. Electron. Agric..

[29]  Di Zhang,et al.  Deep recursive super resolution network with Laplacian Pyramid for better agricultural pest surveillance and detection , 2018, Comput. Electron. Agric..

[30]  Rodrigo Castañeda-Miranda,et al.  Original paper: Scale invariant feature approach for insect monitoring , 2011 .

[31]  Konstantinos P. Ferentinos,et al.  Deep learning models for plant disease detection and diagnosis , 2018, Comput. Electron. Agric..

[32]  Po Yang,et al.  Fusing multi-scale context-aware information representation for automatic in-field pest detection and recognition , 2020, Comput. Electron. Agric..

[33]  Haiyang Zhou,et al.  A smart-vision algorithm for counting whiteflies and thrips on sticky traps using two-dimensional Fourier transform spectrum , 2017 .

[34]  Jayme Garcia Arnal Barbedo,et al.  Factors influencing the use of deep learning for plant disease recognition , 2018, Biosystems Engineering.

[35]  Sang-Heon Lee,et al.  A Multispectral 3-D Vision System for Invertebrate Detection on Crops , 2017, IEEE Sensors Journal.

[36]  Alain Clément,et al.  A new colour vision system to quantify automatically foliar discolouration caused by insect pests feeding on leaf cells , 2015 .

[37]  Jacob Goldberger,et al.  Training deep neural-networks based on unreliable labels , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  E. Oerke.  Crop losses to pests , 2005, The Journal of Agricultural Science.

[39]  Jun Zhang,et al.  Insect Detection and Classification Based on an Improved Convolutional Neural Network , 2018, Sensors.

[40]  Alejandro López,et al.  Combination of image processing and artificial neural networks as a novel approach for the identification of Bemisia tabaci and Frankliniella occidentalis on sticky traps in greenhouse agriculture , 2016, Comput. Electron. Agric..

[41]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[42]  Jang-myung Lee,et al.  Detection of small-sized insect pest in greenhouses based on multifractal analysis , 2015 .

[43]  Qibing Zhu,et al.  Automatic threshold method and optimal wavelength selection for insect-damaged vegetable soybean detection using hyperspectral images , 2014 .

[44]  Huan Zhang,et al.  Localization and Classification of Paddy Field Pests using a Saliency Map and Deep Convolutional Neural Network , 2016, Scientific Reports.

[45]  Jayme Garcia Arnal Barbedo,et al.  Using digital image processing for counting whiteflies on soybean leaves , 2014 .

[46]  Chenglu Wen,et al.  Image-based orchard insect automated identification and classification method , 2012 .

[47]  Deng Limiao,et al.  Recognition pest by image-based transfer learning. , 2019, Journal of the science of food and agriculture.

[48]  Jian Tang,et al.  Automated Counting of Rice Planthoppers in Paddy Fields Based on Image Processing , 2014 .

[49]  Renfu Lu,et al.  Detection of fruit fly infestation in pickling cucumbers using a hyperspectral reflectance/transmittance imaging system , 2013 .

[50]  Yufeng Shen,et al.  Detection of stored-grain insects using deep learning , 2018, Comput. Electron. Agric..

[51]  Sreekala G. Bajwa,et al.  Detection of soybean aphids in a greenhouse using an image processing technique , 2017, Comput. Electron. Agric..

[52]  Karen Lucero Roldán-Serrato,et al.  Automatic pest detection on bean and potato crops by applying neural classifiers , 2018, Engineering in Agriculture, Environment and Food.

[53]  Feng Yang,et al.  Mobile smart device-based vegetable disease and insect pest recognition method , 2013, Intell. Autom. Soft Comput..

[54]  Chenglu Wen,et al.  Pose estimation-dependent identification method for field moth images using deep learning architecture , 2015 .

[55]  Amirthalingam Ramanan,et al.  Image Classification of Paddy Field Insect Pests Using Gradient-Based Features , 2014 .

[56]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[57]  Thomas G. Dietterich,et al.  Segmentation of touching insects based on optical flow and NCuts , 2013 .

[58]  Ta-Te Lin,et al.  An Online Unsupervised Deep Learning Approach for an Automated Pest Insect Monitoring System , 2019, 2019 Boston, Massachusetts July 7- July 10, 2019.

[59]  Wei Wu,et al.  Detection of aphids in wheat fields using a computer vision technique , 2016 .

[60]  Jun Lv,et al.  An Insect Imaging System to Automate Rice Light-Trap Pest Identification , 2012 .

[61]  Huajian Liu,et al.  A multispectral machine vision system for invertebrate detection on green leaves , 2018, Comput. Electron. Agric..

[62]  Huajian Liu,et al.  A review of recent sensing technologies to detect invertebrates on crops , 2017, Precision Agriculture.

[63]  Jayme Garcia Arnal Barbedo,et al.  Influence of image quality on the identification of psyllids using convolutional neural networks , 2019, Biosystems Engineering.

[64]  Chengjun Xie,et al.  AF-RCNN: An anchor-free convolutional neural network for multi-categories agricultural pest detection , 2020, Comput. Electron. Agric..

[65]  Fangyuan Wang,et al.  PestNet: An End-to-End Deep Learning Approach for Large-Scale Multi-Class Pest Detection and Classification , 2019, IEEE Access.

[66]  Tae-Soo Chon,et al.  Automatic identification and counting of small size pests in greenhouse conditions with low computational cost , 2015, Ecol. Informatics.

[67]  Kamil Dimililer,et al.  ICSPI: intelligent classification system of pest insects based on image processing and neural arbitration. , 2017 .

[68]  Torsten Prinz,et al.  Analysis of Unmanned Aerial System-Based CIR Images in Forestry—A New Perspective to Monitor Pest Infestation Levels , 2015 .

[69]  Patrizia Busato,et al.  Machine Learning in Agriculture: A Review , 2018, Sensors.

[70]  Joris IJsselmuiden,et al.  Monitoring and mapping with robot swarms for agricultural applications , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[71]  Min Zhang,et al.  Detection of insect-damaged vegetable soybeans using hyperspectral transmittance image , 2013 .

[72]  M. A. Shah,et al.  Imaging techniques for the detection of stored product pests , 2014, Applied Entomology and Zoology.

[73]  Tae-Soo Chon,et al.  Density estimation of Bemisia tabaci (Hemiptera: Aleyrodidae) in a greenhouse using sticky traps in conjunction with an image processing system , 2008 .

[74]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..