Planning the sequence of tasks for harvesting robots

Abstract A methodology for planning the sequence of tasks for a harvesting robot is presented. The fruit targets are situated at unknown locations and must be detected by the robot through a sequence of sensing tasks. Once the targets are detected, the robot must execute a harvest action at each target location. The traveling salesman paradigm (TSP) is used to plan the sequence of sensing and harvesting tasks taking into account the costs of the sensing and harvesting actions and the traveling times. Sensing is planned online. The methodology is validated and evaluated in both laboratory and greenhouse conditions for a case study of a sweet pepper harvesting robot. The results indicate that planning the sequence of tasks for a sweet pepper harvesting robot results in 12% cost reduction. Incorporating the sensing operation in the planning sequence for fruit harvesting is a new approach in fruit harvesting robots and is important for cycle time reduction. Furthermore, the sequence is re-planned as sensory information becomes available and the costs of these new sensing operations are also considered in the planning.

[1]  Moshe P. Mann,et al.  Minimum Time Kinematic Motions of a Cartesian Mobile Manipulator for a Fruit Harvesting Robot , 2014 .

[2]  Tateshi Fujiura,et al.  Cherry-harvesting robot , 2008 .

[3]  Anders la Cour-Harbo,et al.  Side-to-side 3D coverage path planning approach for agricultural robots to minimize skip/overlap areas between swaths , 2016, Robotics Auton. Syst..

[4]  Jan Faigl,et al.  Online planning for multi-robot active perception with self-organising maps , 2018, Auton. Robots.

[5]  Oliver Brock,et al.  Sampling-Based Motion Planning With Sensing Uncertainty , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[6]  Eldert J. van Henten,et al.  Design of an eye-in-hand sensing and servo control framework for harvesting robotics in dense vegetation , 2016 .

[7]  G. Laporte.  The traveling salesman problem: An overview of exact and approximate algorithms , 1992 .

[8]  J. Bontsema,et al.  An Autonomous Robot for Harvesting Cucumbers in Greenhouses , 2002, Auton. Robots.

[9]  Jochen Hemming,et al.  Fruit Detectability Analysis for Different Camera Positions in Sweet-Pepper † , 2014, Sensors.

[10]  Yael Edan,et al.  CROPS : high tech agricultural robots , 2014 .

[11]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..

[12]  Phillip J. McKerrow,et al.  Recognising plants with ultrasonic sensing for mobile robot navigation , 2001, Robotics Auton. Syst..

[13]  Dionysis Bochtis,et al.  Coverage planning for capacitated field operations, part II: Optimisation , 2015 .

[14]  Stavros Vougioukas,et al.  A distributed control framework for motion coordination of teams of autonomous agricultural vehicles , 2012 .

[15]  Yael Edan,et al.  Simulation Analysis of Sweet Pepper Harvesting Operations , 2014, APMS.

[16]  Kurt D. Krebsbach,et al.  An Analysis of Sensor-Based Task Planning , 2007 .

[17]  Moshe P. Mann,et al.  Harvest-order planning for a multiarm robotic harvester , 2014 .

[18]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[19]  Tran Cao Son,et al.  Planning with Sensing Actions and Incomplete Information Using Logic Programming , 2004, LPNMR.

[20]  Yael Edan,et al.  Near-minimum-time task planning for fruit-picking robots , 1991, IEEE Trans. Robotics Autom..

[21]  Yael Edan.  Food and Agriculture Robotics , 2007 .

[22]  Yael Edan,et al.  Computer vision for fruit harvesting robots - state of the art and challenges ahead , 2012, Int. J. Comput. Vis. Robotics.

[23]  Keld Helsgaun,et al.  An effective implementation of the Lin-Kernighan traveling salesman heuristic , 2000, Eur. J. Oper. Res..

[24]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[25]  Steven M. LaValle,et al.  An Objective-Based Framework for Motion Planning under Sensing and Control Uncertainties , 1998, Int. J. Robotics Res..

[26]  Yael Edan,et al.  Robotics in protected cultivation , 2013 .

[27]  E. J. van Henten,et al.  Field Test of an Autonomous Cucumber Picking Robot , 2003 .

[28]  Yael Edan,et al.  Automatic Parameter Tuning for Adaptive Thresholding in Fruit Detection , 2019, Sensors.

[29]  Yael Edan,et al.  Spatial and temporal distribution of ripe muskmelon fruit , 1997 .

[30]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[31]  Shigehiko Hayashi,et al.  Robotic Harvesting System for Eggplants , 2002 .

[32]  Andrés Villa-Henriksen,et al.  Route planning evaluation of a prototype optimised infield route planner for neutral material flow agricultural operations , 2017 .

[33]  Victor Alchanatis,et al.  FRUIT VISIBILITY ANALYSIS FOR ROBOTIC CITRUS HARVESTING , 2009 .

[34]  Kenta Shigematsu,et al.  Evaluation of a strawberry-harvesting robot in a field test , 2010 .

[35]  Yael Edan,et al.  Dynamic thresholding algorithm for robotic apple detection , 2017, 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[36]  Daniel J. Rosenkrantz,et al.  An analysis of several heuristics for the traveling salesman problem , 2013, Fundamental Problems in Computing.

[37]  Eugene L. Lawler,et al.  The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization , 1985 .

[38]  Yael Edan,et al.  Statistical models for fruit detectability: spatial and temporal analyses of sweet peppers , 2018, Biosystems Engineering.

[39]  Giuseppe De Giacomo,et al.  Planning with Sensing for a Mobile Robot , 1997, ECP.

[40]  D. Van Oudheusden,et al.  Infield logistics planning for crop-harvesting operations , 2009 .

[41]  Jochen Hemming,et al.  Performance Evaluation of a Harvesting Robot for Sweet Pepper , 2017, J. Field Robotics.

[42]  Claus G. Sørensen,et al.  The vehicle routing problem in field logistics: Part II , 2009 .

[43]  Lydia E. Kavraki,et al.  The Open Motion Planning Library , 2012, IEEE Robotics & Automation Magazine.

[44]  Jean-Claude Latombe,et al.  Robot Motion Planning with Uncertainty in Control and Sensing , 1991, Artif. Intell..

[45]  Yael Edan,et al.  Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting , 2019, Sensors.

[46]  K. Zhou,et al.  Route planning for orchard operations , 2015, Comput. Electron. Agric..

[47]  Marko Hočevar,et al.  CROPS : Clever robots for crops , 2015 .

[48]  Yael Edan,et al.  Development of a sweet pepper harvesting robot , 2020, J. Field Robotics.

[49]  Yael Edan,et al.  Robotic melon harvesting , 2000, IEEE Trans. Robotics Autom..

[50]  Moshe P. Mann,et al.  Combinatorial Optimization and Performance Analysis of a Multi-arm Cartesian Robotic Fruit Harvester—Extensions of Graph Coloring , 2016, J. Intell. Robotic Syst..

[51]  Yael Edan,et al.  Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead , 2014, J. Field Robotics.