Automated Detection of Branch Shaking Locations for Robotic Cherry Harvesting Using Machine Vision

Automation in cherry harvesting is essential to reduce the demand for seasonal labor for cherry picking and reduce the cost of production. The mechanical shaking of tree branches is one of the widely studied and used techniques for harvesting small tree fruit crops like cherries. To automate the branch shaking operation, different methods of detecting branches and cherries in full foliage canopies of the cherry tree have been developed previously. The next step in this process is the localization of shaking positions in the detected tree branches for mechanical shaking. In this study, a method of locating shaking positions for automated cherry harvesting was developed based on branch and cherry pixel locations determined using RGB images and 3D camera images. First, branch and cherry regions were located in 2D RGB images. Depth information provided by a 3D camera was then mapped on to the RGB images using a standard stereo calibration method. The overall root mean square error in estimating the distance to desired shaking points was 0.064 m. Cherry trees trained in two different canopy architectures, Y-trellis and vertical trellis systems, were used in this study. Harvesting testing was carried out by shaking tree branches at the locations selected by the algorithm. For the Y-trellis system, the maximum fruit removal efficiency of 92.9% was achieved using up to five shaking events per branch. However, maximum fruit removal efficiency for the vertical trellis system was 86.6% with up to four shakings per branch. However, it was found that only three shakings per branch would achieve a fruit removal percentage of 92.3% and 86.4% in Y and vertical trellis systems respectively.

[1]  R. B. Fridley,et al.  Forced Vibration of a Tree Limb , 1965 .

[2]  Manoj Karkee,et al.  Detection of cherry tree branches for automated shake-and-catch harvesting , 2015 .

[3]  Jianfeng Zhou,et al.  Evaluation of the Influence of Shaking Frequency and Duration in Mechanical Harvesting of Sweet Cherry , 2013 .

[4]  Jianfeng Zhou,et al.  Effect of excitation position of a handheld shaker on fruit removal efficiency and damage in mechanical harvesting of sweet cherry , 2014 .

[5]  D. L. Peterson,et al.  FRESH–MARKET QUALITY TREE FRUIT HARVESTER PART II: APPLES , 2003 .

[6]  E. Dursun,et al.  Mechanical Harvesting of Apricots , 2003 .

[7]  Zhao Dean,et al.  System Design and Control of an Apple Harvesting Robot , 2020, ArXiv.

[8]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[9]  D. L. Peterson,et al.  A SYSTEMS APPROACH TO ROBOTIC BULK HARVESTING OF APPLES , 1999 .

[10]  Jose L Pons,et al.  Design and implementation of an aided fruit‐harvesting robot (Agribot) , 1998 .

[11]  D. L. Peterson,et al.  MECHANICAL HARVESTER FOR FRESH MARKET QUALITY STEMLESS SWEET CHERRIES , 2001 .

[12]  Q. Zhang,et al.  Apple crop-load estimation with over-the-row machine vision system , 2016, Comput. Electron. Agric..

[13]  D. L. Peterson,et al.  FRESH–MARKET QUALITY TREE FRUIT HARVESTER PART I: SWEET CHERRY , 2003 .

[14]  Qin Zhang,et al.  Performance Evaluation of Mechanical Cherry Harvesters for Fresh Market Grade Fruits , 2012 .

[15]  Qin Zhang,et al.  Mechanical harvesting of UFO Cherry: Investigation of tree plant dynamics , 2011 .

[16]  J. D. Whitney.  Design and Performance of an Air Shaker for Citrus Fruit Removal , 1977 .

[17]  Eric Claesen,et al.  Autonomous Fruit Picking Machine: A Robotic Apple Harvester , 2007, FSR.

[18]  James L. Halderson.  Fundamental Factors in Mechanical Cherry Harvesting , 1966 .

[19]  Qin Zhang,et al.  Detection of cherry tree branches with full foliage in planar architecture for automated sweet-cherry harvesting , 2016 .

[20]  Manoj Karkee,et al.  Effects of Orchard Characteristics and Operator Performance on Harvesting Rate of a Mechanical Sweet Cherry Harvester , 2014 .

[21]  Manoj Karkee,et al.  Integration of visible branch sections and cherry clusters for detecting cherry tree branches in dense foliage canopies , 2016 .

[22]  Qin Zhang,et al.  Modification and Field Evaluation of an Experimental Mechanical Sweet Cherry Harvester , 2015 .

[23]  M. D. Whiting,et al.  COMPARING THE ECONOMICS OF MECHANICAL AND TRADITIONAL SWEET CHERRY HARVEST , 2011 .

[24]  Tateshi Fujiura,et al.  Cherry-harvesting robot , 2008 .

[25]  Gregorio L. Blanco-Roldán,et al.  Effects of Trunk Shaker Duration and Repetitions on Removal Efficiency for the Harvesting of Oil Olives , 2009 .