Sweet pepper pose detection and grasping for automated crop harvesting

This paper presents a method for estimating the 6DOF pose of sweet-pepper (capsicum) crops for autonomous harvesting via a robotic manipulator. The method uses the Kinect Fusion algorithm to robustly fuse RGB-D data from an eye-in-hand camera combined with a colour segmentation and clustering step to extract an accurate representation of the crop. The 6DOF pose of the sweet peppers is then estimated via a nonlinear least squares optimisation by fitting a superellipsoid to the segmented sweet pepper. The performance of the method is demonstrated on a real 6DOF manipulator with a custom gripper. The method is shown to estimate the 6DOF pose successfully enabling the manipulator to grasp sweet peppers for a range of different orientations. The results obtained improve largely on the performance of grasping when compared to a naive approach, which does not estimate the orientation of the crop.

[1]  Kenta Shigematsu,et al.  Evaluation of a strawberry-harvesting robot in a field test , 2010 .

[2]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using orthonormal matrices , 1988 .

[4]  Atilla Baskurt,et al.  Segmentation and Superquadric Modeling of 3D Objects , 2003, WSCG.

[5]  Shih-Fu Chang,et al.  Real-time pose estimation of deformable objects using a volumetric approach , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Gary R. Bradski,et al.  Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor , 2012, Robotics: Science and Systems.

[7]  Marcus Strand,et al.  Segmentation and approximation of objects in pointclouds using superquadrics , 2009, 2009 International Conference on Information and Automation.

[8]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[9]  近藤 直,et al.  Agricultural robots : mechanisms and practice , 2011 .

[10]  Maarten Weyn,et al.  A Survey of Rigid 3D Pointcloud Registration Algorithms , 2014 .

[11]  Zhao Dean,et al.  System Design and Control of an Apple Harvesting Robot , 2020, ArXiv.

[12]  Siddhartha S. Mehta,et al.  Vision-based control of robotic manipulator for citrus harvesting , 2014 .

[13]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[14]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[15]  E. J. Pekkeriet,et al.  A robot for harvesting sweet-pepper in greenhouses , 2014 .

[16]  Josse De Baerdemaeker,et al.  Apple detection algorithm for robotic harvesting using a RGB-D camera , 2014 .

[17]  Yael Edan,et al.  Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead , 2014, J. Field Robotics.

[18]  Takashi Kataoka,et al.  Fruit detection system and an end effector for robotic harvesting of Fuji apples , 2010 .

[19]  Martin Mellado,et al.  Review. Technologies for robot grippers in pick and place operations for fresh fruits and vegetables , 2011 .

[20]  Sudeep Sarkar,et al.  Multi-scale superquadric fitting for efficient shape and pose recovery of unknown objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  Sang-Heon Lee,et al.  Review on fruit harvesting method for potential use of automatic fruit harvesting systems , 2011 .

[22]  Yael Edan,et al.  CROPS : high tech agricultural robots , 2014 .

[23]  J. Bontsema,et al.  An Autonomous Robot for Harvesting Cucumbers in Greenhouses , 2002, Auton. Robots.

[24]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[25]  Yoav Sarig,et al.  Robotics of Fruit Harvesting: A State-of-the-art Review , 1993 .

[26]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.