Localization and Classification of Paddy Field Pests using a Saliency Map and Deep Convolutional Neural Network

We present a pipeline for the visual localization and classification of agricultural pest insects by computing a saliency map and applying deep convolutional neural network (DCNN) learning. First, we used a global contrast region-based approach to compute a saliency map for localizing pest insect objects. Bounding squares containing targets were then extracted, resized to a fixed size, and used to construct a large standard database called Pest ID. This database was then utilized for self-learning of local image features which were, in turn, used for classification by DCNN. DCNN learning optimized the critical parameters, including size, number and convolutional stride of local receptive fields, dropout ratio and the final loss function. To demonstrate the practical utility of using DCNN, we explored different architectures by shrinking depth and width, and found effective sizes that can act as alternatives for practical applications. On the test set of paddy field images, our architectures achieved a mean Accuracy Precision (mAP) of 0.951, a significant improvement over previous methods.

[1]  Jun Lv,et al.  An Insect Imaging System to Automate Rice Light-Trap Pest Identification , 2012 .

[2]  Xin Wang,et al.  Research on insect identification based on pattern recognition technology , 2010, 2010 Sixth International Conference on Natural Computation.

[3]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[4]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[5]  M. Koziel,et al.  Transgenic plants: An emerging approach to pest control , 1997, Nature Biotechnology.

[6]  Thomas G. Dietterich,et al.  A Hierarchical Object Recognition System Based on Multi-scale Principal Curvature Regions , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[8]  Rodrigo Castañeda-Miranda,et al.  Original paper: Scale invariant feature approach for insect monitoring , 2011 .

[9]  Thomas G. Dietterich,et al.  An aquatic insect imaging system to automate insect classification , 2008 .

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Jason Weston,et al.  Large-scale kernel machines , 2007 .

[14]  Zaw Zaw Htike,et al.  VISION BASED ENTOMOLOGY : A SURVEY , 2014 .

[15]  Henk Corporaal,et al.  Speed sign detection and recognition by convolutional neural networks , 2011 .

[16]  Amirthalingam Ramanan,et al.  Image Classification of Paddy Field Insect Pests Using Gradient-Based Features , 2014 .

[17]  Thomas G. Dietterich,et al.  Automated insect identification through concatenated histograms of local appearance features: feature vector generation and region detection for deformable objects , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[18]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[19]  S. Morse,et al.  Obstacles to integrated pest management adoption in developing countries , 2014, Proceedings of the National Academy of Sciences.

[20]  King Ngi Ngan,et al.  Unsupervised extraction of visual attention objects in color images , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Soo-Young Lee,et al.  Adaptive learning algorithms to incorporate additional functional constraints into neural networks , 1998, Neurocomputing.

[22]  John K. Tsotsos,et al.  50 Years of object recognition: Directions forward , 2013, Comput. Vis. Image Underst..

[23]  Sabine Moisan,et al.  Robust Insect Classification Applied to Real Time Greenhouse Infestation Monitoring , 2010 .

[24]  Hongtao Zhang,et al.  Feature Selection for the Stored-grain Insects Based on PSO and SVM , 2009, 2009 Second International Workshop on Knowledge Discovery and Data Mining.

[25]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[26]  Thomas G. Dietterich,et al.  Automated processing and identification of benthic invertebrate samples , 2010, Journal of the North American Benthological Society.

[27]  Rodrigo Castañeda-Miranda,et al.  Machine vision algorithm for whiteflies (Bemisia tabaci Genn.) scouting under greenhouse environment , 2009 .

[28]  Yongxin Yang,et al.  Deep Neural Networks for Sketch Recognition , 2015, ArXiv.

[29]  Jian Tang,et al.  Automated Counting of Rice Planthoppers in Paddy Fields Based on Image Processing , 2014 .

[30]  Zhen Zhang,et al.  Auto-classification of insect images based on color histogram and GLCM , 2010, 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery.

[31]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[32]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Vincent Martin,et al.  A cognitive vision approach to early pest detection in greenhouse crops , 2008 .

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[36]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[37]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Tae-Soo Chon,et al.  In situ detection of small-size insect pests sampled on traps using multifractal analysis , 2012 .

[39]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  J. Cid-Sueiro,et al.  Neural architectures for parametric estimation of a posteriori probabilities by constrained conditional density functions , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[41]  Xiaolin Chen,et al.  Insect species recognition using discriminative local soft coding , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[42]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[43]  Keith C. Norris,et al.  A test of a pattern recognition system for identification of spiders , 1999 .

[44]  Seyed Mahmood Hosseini,et al.  Farmers' competence and training needs on pest management practices: Participation in extension workshops , 2009 .

[45]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[46]  Stefan Schröder,et al.  Biodiversity Informatics in Action: Identification and Monitoring of Bee Species using ABIS , 2001 .

[47]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[48]  Robert L. Metcalf,et al.  Introduction to insect pest management , 1975 .

[49]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  Chenglu Wen,et al.  Image-based orchard insect automated identification and classification method , 2012 .

[51]  Byoung Chul Ko,et al.  Object-of-interest image segmentation based on human attention and semantic region clustering. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.