Review of Machine-Vision-Based Plant Detection Technologies for Robotic Weeding

Controlling weeds with reduced reliance on herbicides is one of the main challenges to move toward a more sustainable agriculture. Robotic weeding is a thought to be a viable way to reduce the environmental loading of agrochemicals while keeping the operation efficiency high. One of the key technologies for performing robotic weeding is automatic detection of crops and weeds in fields. This paper presents an overview on various methods for detecting plants based on machine vision, mainly concentrating on two main challenges: dealing with changing light and crop/weed discrimination. To overcome the first challenge, both physical and algorithmic methods have been proposed. Physical methods can result in a more cumbersome machine while algorithmic methods are less robust. For crop/weed discrimination, deep-learning-based methods have shown obvious advantages over traditional methods based on hand-crafted features. However, traditional methods still hold some merits that can be leveraged to deep-learning-based methods. With the fast development of hardware technologies, researchers should take full advantage of advanced hardware to ease the algorithm design. In the future, the identification of crops and weeds can be more accurate and fine-grained with the support of online databases and computing resources based on the advances in artificial intelligence and communication technologies.

[1]  Gonzalo Pajares,et al.  Automatic expert system based on images for accuracy crop row detection in maize fields , 2013, Expert Syst. Appl..

[2]  M. Nørremark,et al.  Evaluation of an autonomous GPS-based system for intra-row weed control by assessing the tilled area , 2011, Precision Agriculture.

[3]  Wei Li,et al.  A Method for Broccoli Seedling Recognition in Natural Environment Based on Binocular Stereo Vision and Gaussian Mixture Model , 2019, Sensors.

[4]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[5]  Wolfram Burgard,et al.  Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields , 2017, Int. J. Robotics Res..

[6]  Frédéric Lebeau,et al.  Selection of the most efficient wavelength bands for discriminating weeds from crop , 2008 .

[7]  Lei Tian,et al.  CLASSIFICATION OF BROADLEAF AND GRASS WEEDS USING GABOR WAVELETS AND AN ARTIFICIAL NEURAL NETWORK , 2003 .

[8]  David C. Slaughter,et al.  X-ray based stem detection in an automatic tomato weeding system , 2009 .

[9]  Brian L. Steward,et al.  Plant Recognition through the Fusion of 2D and 3D Images for Robotic Weeding , 2015 .

[10]  Cyrill Stachniss,et al.  Effective Vision‐based Classification for Separating Sugar Beets and Weeds for Precision Farming , 2017, J. Field Robotics.

[11]  Abdolabbas Jafari,et al.  Evaluation of support vector machine and artificial neural networks in weed detection using shape features , 2018, Comput. Electron. Agric..

[12]  Ole Green,et al.  Weed identification using an automated active shape matching (AASM) technique , 2011 .

[13]  Alexander Wendel,et al.  Self-supervised weed detection in vegetable crops using ground based hyperspectral imaging , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Tony E Grift,et al.  Design and testing of an intra-row mechanical weeding machine for corn , 2011 .

[15]  Daniele Nardi,et al.  Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture , 2016, IAS.

[16]  Alexandre Escolà,et al.  Discriminating Crop, Weeds and Soil Surface with a Terrestrial LIDAR Sensor , 2013, Sensors.

[17]  J. Marchant,et al.  Shadow-invariant classification for scenes illuminated by daylight. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Adel Bakhshipour,et al.  Weed segmentation using texture features extracted from wavelet sub-images , 2017 .

[19]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Li Nan,et al.  Crop positioning for robotic intra-row weeding based on machine vision , 2015 .

[21]  Min Huang,et al.  Maize and weed classification using color indices with support vector data description in outdoor fields , 2017, Comput. Electron. Agric..

[22]  T. Carlson,et al.  On the relation between NDVI, fractional vegetation cover, and leaf area index , 1997 .

[23]  Cyrill Stachniss,et al.  Semi-supervised online visual crop and weed classification in precision farming exploiting plant arrangement , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Tony E Grift,et al.  Image processing for crop/weed discrimination in fields with high weed pressure. , 2016 .

[25]  Tristan Perez,et al.  Robot for weed species plant‐specific management , 2017, J. Field Robotics.

[26]  Lie Tang,et al.  Crop recognition under weedy conditions based on 3D imaging for robotic weed control , 2018, J. Field Robotics.

[27]  N. D. Tillett,et al.  DEALING WITH COLOR CHANGES CAUSED BY NATURAL ILLUMINATION IN OUTDOOR MACHINE VISION , 2004, Cybern. Syst..

[28]  Henrik Skov Midtiby,et al.  Plant species classification using deep convolutional neural network , 2016 .

[29]  Qian Wang,et al.  Mean-shift-based color segmentation of images containing green vegetation , 2009 .

[30]  G. Meyer,et al.  Color indices for weed identification under various soil, residue, and lighting conditions , 1994 .

[31]  Cyrill Stachniss,et al.  Fully Convolutional Networks With Sequential Information for Robust Crop and Weed Detection in Precision Farming , 2018, IEEE Robotics and Automation Letters.

[32]  Hao Sun Automatic GPS-Based Intra-Row Weed Control System for Transplanted Row Crops , 2012 .

[33]  J. Hemming,et al.  PA—Precision Agriculture: Computer-Vision-based Weed Identification under Field Conditions using Controlled Lighting , 2001 .

[34]  Jin-Young Jeong,et al.  AE—Automation and Emerging Technologies: Weed–plant Discrimination by Machine Vision and Artificial Neural Network , 2002 .

[35]  Gonzalo Pajares,et al.  Crop rows and weeds detection in maize fields applying a computer vision system based on geometry , 2017, Comput. Electron. Agric..

[36]  Heping Zhu,et al.  Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination , 2011, Sensors.

[37]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Edward Jones,et al.  Automatic crop detection under field conditions using the HSV colour space and morphological operations , 2017, Comput. Electron. Agric..

[39]  Xiaoguang Zhang,et al.  Real-Time Crop Recognition in Transplanted Fields With Prominent Weed Growth: A Visual-Attention-Based Approach , 2019, IEEE Access.

[40]  Wu Lanlan,et al.  Weed/corn seedling recognition by support vector machine using texture features , 2009 .

[41]  A. Samal,et al.  Plant species identification using Elliptic Fourier leaf shape analysis , 2006 .

[42]  G.W.A.M. van der Heijden,et al.  The role of textures to improve the detection accuracy of Rumex obtusifolius in robotic systems , 2012 .

[43]  Jun Chen,et al.  Intra-row weed recognition using plant spacing information in stereo images , 2013 .

[44]  Zhenghong Yu,et al.  Crop feature extraction from images with probabilistic superpixel Markov random field , 2015, Comput. Electron. Agric..

[45]  Farrah Wong,et al.  Genetic Algorithm Feature Selection and Classifier Optimization Using Moment Invariants and Shape Features , 2013, 2013 1st International Conference on Artificial Intelligence, Modelling and Simulation.

[46]  John A. Marchant,et al.  An Autonomous Crop Treatment Robot: Part I. A Kalman Filter Model for Localization and Crop/Weed Classification , 2002, Int. J. Robotics Res..

[47]  Cyrill Stachniss,et al.  Real-Time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[48]  T. Kataoka,et al.  Crop growth estimation system using machine vision , 2003, Proceedings 2003 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2003).

[49]  Tristan Perez,et al.  A Rapidly Deployable Classification System using Visual Data for the Application of Precision Weed Management , 2018, Comput. Electron. Agric..