Plant Localization and Discrimination using 2D+3D Computer Vision for Robotic Intra-row Weed Control

Abstract. Weed management is vitally important in crop production systems. However, conventional herbicide based weed control can lead to negative environmental impacts. Manual weed control is laborious and impractical for large scale production. Robotic weed control offers a possibility of controlling weeds precisely, particularly for weeds growing near or within crop rows. A computer vision system was developed based on Kinect V2 sensor, using the fusion of two-dimensional textural data and three-dimensional spatial data to recognize and localized crop plants different growth stages. Images were acquired of different plant species such as broccoli, lettuce and corn at different growth stages. A database system was developed to organize these images. Several feature extraction algorithms were developed which addressed the problems of canopy occlusion and damaged leaves. With our proposed algorithms, different features were extracted and used to train plant and background classifiers. Finally, the efficiency and accuracy of the proposed classification methods were demonstrated and validated by experiments.

[1]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[2]  F. J. Pierce,et al.  Automation: The Future of Weed Control in Cropping Systems , 2013, Springer Netherlands.

[3]  Lie Tang,et al.  Automatic inter-plant spacing sensing at early growth stages using a 3D vision sensor , 2012 .

[4]  J. Alex Thomasson,et al.  Ground-based sensing system for weed mapping in cotton , 2008 .

[5]  Sergi Foix,et al.  Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison , 2014 .

[6]  Roland Siegwart,et al.  Kinect v2 for mobile robot navigation: Evaluation and modeling , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[7]  Xiaofeng Wang,et al.  Leaf shape based plant species recognition , 2007, Appl. Math. Comput..

[8]  Joachim Hertzberg,et al.  An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments , 2003, Robotics Auton. Syst..

[9]  Remo Sala,et al.  A metrological characterization of the Kinect V2 time-of-flight camera , 2016, Robotics Auton. Syst..

[10]  Robert L. Zimdahl,et al.  Fundamentals of Weed Science , 1993 .

[11]  Reyer Zwiggelaar,et al.  A review of spectral properties of plants and their potential use for crop/weed discrimination in row-crops , 1998 .

[12]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.

[13]  Thomas Rath,et al.  Improving plant discrimination in image processing by use of different colour space transformations , 2002 .

[14]  Roland Siegwart,et al.  Conference Presentation Slides on Kinect v2 for Mobile Robot Navigation: Evaluation and Modeling , 2015 .

[15]  Ji Li 3D machine vision system for robotic weeding and plant phenotyping , 2014 .

[16]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..

[17]  Blatt,et al.  Superparamagnetic clustering of data. , 1998, Physical review letters.

[18]  Carme Torras,et al.  Robotic leaf probing via segmentation of range data into surface patches , 2012 .

[19]  I. M. Scotford,et al.  Applications of Spectral Reflectance Techniques in Northern European Cereal Production: A Review , 2005 .

[20]  Jian Jin,et al.  Corn plant sensing using real-time stereo vision , 2009 .

[21]  Hans Jørgen Andersen,et al.  Detecting leaf features for automatic weed control using trinocular stereo vision. , 2004 .

[22]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.