Plants detection, localization and discrimination using 3D machine vision for robotic intra-row weed control

Weed management is vitally important in crop production systems. However, conventional herbicide-based weed control can lead to negative environmental impacts. Manual weed control is laborious and impractical for large scale production. Robotic weeding offers a possibility of controlling weeds precisely, particularly for weeds growing close to or within crop rows. The fusion of two-dimensional textural images and threedimensional spatial images to recognize and localize crop plants at different growth stages were investigated. Images of different crop plants at different growth stages with weeds were acquired. Feature extraction algorithms were developed, and different features were extracted and used to train plant and background classifiers, which also addressed the problems of canopy occlusion and leaf damage. Then, the efficacy and accuracy of the proposed methods in classification were demonstrated by experiments. Currently, the algorithms were only developed and tested for broccoli and lettuce. For broccoli plants, the crop plants detection true positive rate was 93.1%, and the false discover rate was 1.1%, with the average cropplant-localization error of 15.9 mm. For lettuce plants, the crop plants detection true positive rate was 92.3%, and the false discover rate was 4.0%, with the average crop-plantlocalization error of 8.5 mm. The results have demonstrated that 3D imaging based plant recognition algorithms are effective and reliable for crop/weed differentiation.

[1]  David C. Slaughter,et al.  Autonomous robotic weed control systems: A review , 2008 .

[2]  Ji Li.  3D machine vision system for robotic weeding and plant phenotyping , 2014 .

[3]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..

[4]  Dong Yu,et al.  Conversational Speech Transcription Using Context-Dependent Deep Neural Networks , 2012, ICML.

[5]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jian Jin,et al.  Corn plant sensing using real-time stereo vision , 2009 .

[7]  Yuxuan Wang,et al.  A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[8]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[9]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[10]  J.M. Alvarez,et al.  Illuminant-invariant model-based road segmentation , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[11]  Reyer Zwiggelaar,et al.  A review of spectral properties of plants and their potential use for crop/weed discrimination in row-crops , 1998 .

[12]  Hans Jørgen Andersen,et al.  Detecting leaf features for automatic weed control using trinocular stereo vision. , 2004 .

[13]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[14]  Thomas Rath,et al.  Improving plant discrimination in image processing by use of different colour space transformations , 2002 .

[15]  Lie Tang,et al.  Automatic inter-plant spacing sensing at early growth stages using a 3D vision sensor , 2012 .

[16]  Sergi Foix,et al.  Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison , 2014 .

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2001, Springer Series in Statistics.

[18]  Carme Torras,et al.  Robotic leaf probing via segmentation of range data into surface patches , 2012 .

[19]  I. M. Scotford,et al.  Applications of Spectral Reflectance Techniques in Northern European Cereal Production: A Review , 2005 .

[20]  Craig S. T. Daughtry,et al.  Acquisition of NIR-Green-Blue Digital Photographs from Unmanned Aircraft for Crop Monitoring , 2010, Remote. Sens..

[21]  J. Alex Thomasson,et al.  Ground-based sensing system for weed mapping in cotton , 2008 .

[22]  Xiaofeng Wang,et al.  Leaf shape based plant species recognition , 2007, Appl. Math. Comput..

[23]  Joachim Hertzberg,et al.  An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments , 2003, Robotics Auton. Syst..

[24]  Lin Kunhui,et al.  Feature extraction and automatic recognition of plant leaf using artificial neural network , 2007 .

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Roland Siegwart,et al.  Kinect v2 for mobile robot navigation: Evaluation and modeling , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[27]  Andreas Zell,et al.  Plant Species Classification Using a 3D LIDAR Sensor and Machine Learning , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[28]  Larry Li,et al.  Time-of-Flight Camera - An Introduction , 2014 .

[29]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.