Can you pick a broccoli? 3D-vision based detection and localisation of broccoli heads in the field

This paper presents a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier and a temporal filter to track the detected heads results in a system that detects broccoli heads with 95.2% precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field.

[1]  Jan Faigl,et al.  Simple yet stable bearing-only navigation , 2010 .

[2]  Steven R. Raine,et al.  Applied machine vision of plants – a review with implications for field deployment in automated farming operations 1 , 2010 .

[3]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Libor Preucil,et al.  Simple yet stable bearing-only navigation , 2010, J. Field Robotics.

[5]  Rachael Angela Ramirez,et al.  Computer Vision Based Analysis of Broccoli for Application in a Selective Autonomous Harvester , 2006 .

[6]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[7]  W. S. Lee,et al.  Machine Vision for Green Citrus Detection in Tree Images , 2010 .

[8]  Ying Zhang,et al.  Automatic recognition vision system guided for apple harvesting robot , 2012, Comput. Electr. Eng..

[9]  Fei-Fei Li,et al.  Object discovery in 3D scenes via shape analysis , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Kenta Shigematsu,et al.  Evaluation of a strawberry-harvesting robot in a field test , 2010 .

[11]  Jose L Pons,et al.  A SURVEY OF COMPUTER VISION METHODS FOR LOCATING FRUIT ON TREES , 2000 .

[12]  Nico Blodow,et al.  CAD-model recognition and 6DOF pose estimation using 3D cues , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[13]  Tristan Perez,et al.  On Visual Detection of Highly-occluded Objects for Harvesting Automation in Horticulture , 2015, ICRA 2015.

[14]  Roland Siegwart,et al.  Kinect v2 for mobile robot navigation: Evaluation and modeling , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[15]  J. Bontsema,et al.  An Autonomous Robot for Harvesting Cucumbers in Greenhouses , 2002, Auton. Robots.

[16]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..

[17]  Brian L. Steward,et al.  Plant Recognition through the Fusion of 2D and 3D Images for Robotic Weeding , 2015 .

[18]  Ohad Ben-Shahar,et al.  Colour-agnostic shape-based 3D fruit detection for crop harvesting robots , 2016 .

[19]  Hongwen Li,et al.  A Study of Broccoli Grading System Based on Machine Vision and Neural Networks , 2007, 2007 International Conference on Mechatronics and Automation.

[20]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Yael Edan,et al.  Computer vision for fruit harvesting robots - state of the art and challenges ahead , 2012, Int. J. Comput. Vis. Robotics.

[22]  V. Alchanatis,et al.  Low and high-level visual feature-based apple detection from multi-modal images , 2010, Precision Agriculture.

[23]  Jörn Ostermann,et al.  Plant classification system for crop /weed discrimination without segmentation , 2014, IEEE Winter Conference on Applications of Computer Vision.

[24]  Josse De Baerdemaeker,et al.  Apple detection algorithm for robotic harvesting using a RGB-D camera , 2014 .