Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images

In recent years, weeds have been responsible for most agricultural yield losses. To deal with this threat, farmers resort to spraying the fields uniformly with herbicides. This method not only requires huge quantities of herbicides but impacts the environment and human health. One way to reduce the cost and environmental impact is to allocate the right doses of herbicide to the right place and at the right time (precision agriculture). Nowadays, unmanned aerial vehicles (UAVs) are becoming an interesting acquisition system for weed localization and management due to their ability to obtain images of the entire agricultural field with a very high spatial resolution and at a low cost. However, despite significant advances in UAV acquisition systems, the automatic detection of weeds remains a challenging problem because of their strong similarity to the crops. Recently, a deep learning approach has shown impressive results in different complex classification problems. However, this approach needs a certain amount of training data, and creating large agricultural datasets with pixel-level annotations by an expert is an extremely time-consuming task. In this paper, we propose a novel fully automatic learning method using convolutional neuronal networks (CNNs) with an unsupervised training dataset collection for weed detection from UAV images. The proposed method comprises three main phases. First, we automatically detect the crop rows and use them to identify the inter-row weeds. In the second phase, inter-row weeds are used to constitute the training dataset. Finally, we perform CNNs on this dataset to build a model able to detect the crop and the weeds in the images. The results obtained are comparable to those of traditional supervised training data labeling, with differences in accuracy of 1.5% in the spinach field and 6% in the bean field.

[1]  Dongqing,et al.  Weed Recognition Using Image-Processing Technique Based on Leaf Parameters , 2012 .

[2]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[3]  S. Kinast,et al.  Ground-level hyperspectral imagery for detecting weeds in wheat fields , 2013, Precision Agriculture.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Abdolabbas Jafari,et al.  Evaluation of support vector machine and artificial neural networks in weed detection using shape features , 2018, Comput. Electron. Agric..

[6]  Latha,et al.  Image Processing in Agriculture , 2014 .

[7]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[8]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[9]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[10]  F. J. Pierce,et al.  ASPECTS OF PRECISION AGRICULTURE , 1999 .

[11]  Wilson S. Geisler,et al.  Multichannel Texture Analysis Using Localized Spatial Filters , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Nur Nabilah Abu Mangshor,et al.  Leaf Recognition using Texture Features for Herbal Plant Identification , 2018 .

[13]  Faisal Ahmed,et al.  Classification of crops and weeds from digital images: A support vector machine approach , 2012 .

[14]  Edward Jones,et al.  A survey of image processing techniques for plant extraction and segmentation in the field , 2016, Comput. Electron. Agric..

[15]  Ciro Potena,et al.  Automatic model based dataset generation for fast and accurate crop and weeds detection , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Lei Tian,et al.  CLASSIFICATION OF BROADLEAF AND GRASS WEEDS USING GABOR WAVELETS AND AN ARTIFICIAL NEURAL NETWORK , 2003 .

[17]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Sylvain Villette,et al.  Unsupervised Classification Algorithm for Early Weed Detection in Row-Crops by Combining Spatial and Spectral Information , 2018, Remote. Sens..

[19]  Johan Trygg,et al.  Haralick texture features from apparent diffusion coefficient (ADC) MRI images depend on imaging and pre-processing parameters , 2017, Scientific Reports.

[20]  Adel Bakhshipour,et al.  Weed segmentation using texture features extracted from wavelet sub-images , 2017 .

[21]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[22]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[23]  Lirong Song,et al.  Effects of arsenate on the growth and microcystin production of Microcystis aeruginosa isolated from Taiwan as influenced by extracellular phosphate , 2009, Journal of Applied Phycology.

[24]  María Pérez-Ortiz,et al.  Selecting patterns and features for between- and within- crop-row weed mapping using UAV-imagery , 2016, Expert Syst. Appl..

[25]  Cyrill Stachniss,et al.  UAV-based crop and weed classification for smart farming , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Ying Yang,et al.  Special object extraction from medieval books using superpixels and bag-of-features , 2017, J. Electronic Imaging.

[27]  Leonid P. Yaroslavsky,et al.  Weed detection in multi-spectral images of cotton fields , 2005 .

[28]  Aleksandra Pizurica,et al.  Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery , 2018, Int. J. Appl. Earth Obs. Geoinformation.

[29]  Adel Hafiane,et al.  Deep Learning Based Classification System for Identifying Weeds Using High-Resolution UAV Imagery , 2018, Advances in Intelligent Systems and Computing.

[30]  F. López-Granados,et al.  Weed Mapping in Early-Season Maize Fields Using Object-Based Analysis of Unmanned Aerial Vehicle (UAV) Images , 2013, PloS one.

[31]  Hemerson Pistori,et al.  Weed detection in soybean crops using ConvNets , 2017, Comput. Electron. Agric..

[32]  Adel Hafiane,et al.  Weeds detection in UAV imagery using SLIC and the hough transform , 2017, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA).

[33]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[35]  Gonzalo Pajares,et al.  Automatic detection of crop rows in maize fields with high weeds pressure , 2012, Expert Syst. Appl..

[36]  F. Truchetet,et al.  Crop/weed discrimination in perspective agronomic images , 2008 .

[37]  G. Meyer,et al.  Color indices for weed identification under various soil, residue, and lighting conditions , 1994 .

[38]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[39]  E. Oerke Crop losses to pests , 2005, The Journal of Agricultural Science.

[40]  Xiao-Feng Wang,et al.  HOG-Based Approach for Leaf Classification , 2010, ICIC.

[41]  J. Bouma,et al.  Future Directions of Precision Agriculture , 2005, Precision Agriculture.

[42]  Hans Jørgen Andersen,et al.  Exploiting affine invariant regions and leaf edge shapes for weed detection , 2015, Comput. Electron. Agric..

[43]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[44]  D. Mulla Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps , 2013 .

[45]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[46]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[47]  Jorge Torres-Sánchez,et al.  An Automatic Random Forest-OBIA Algorithm for Early Weed Mapping between and within Crop Rows Using UAV Imagery , 2018, Remote. Sens..

[48]  Heping Zhu,et al.  Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination , 2011, Sensors.

[49]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[50]  Chunhua Zhang,et al.  The application of small unmanned aerial systems for precision agriculture: a review , 2012, Precision Agriculture.

[51]  R. Gerhards,et al.  Precision farming for weed management: techniques , 2008, Gesunde Pflanzen.

[52]  Patrick M. Pilarski,et al.  First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning , 2014 .

[53]  Anders Krogh Mortensen,et al.  Semantic Segmentation of Mixed Crops using Deep Convolutional Neural Network , 2016 .

[54]  Zhe Xu,et al.  Feature Learning Based Approach for Weed Classification Using High Resolution Aerial Images from a Digital Camera Mounted on a UAV , 2014, Remote. Sens..

[55]  Zongxu Pan,et al.  Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data , 2017, Remote. Sens..

[56]  C. Tucker Red and photographic infrared linear combinations for monitoring vegetation , 1979 .

[57]  F. Truchetet,et al.  Modelling agronomic images for weed detection and comparison of crop/weed discrimination algorithm performance , 2009, Precision Agriculture.

[58]  Jorge Torres-Sánchez,et al.  An automatic object-based method for optimal thresholding in UAV images: Application for vegetation detection in herbaceous crops , 2015, Comput. Electron. Agric..

[59]  Jian Wang,et al.  Texture analysis using Gabor wavelets , 1996, Electronic Imaging.

[60]  Leen-Kiat Soh,et al.  Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices , 1999, IEEE Trans. Geosci. Remote. Sens..

[61]  Cyrill Stachniss,et al.  REAL-TIME BLOB-WISE SUGAR BEETS VS WEEDS CLASSIFICATION FOR MONITORING FIELDS USING CONVOLUTIONAL NEURAL NETWORKS , 2017 .