Technology progress in mechanical harvest of fresh market apples

Abstract This article reviews the research and development progress of mechanical harvest technologies for fresh market apples over the past decades with a focus on the predominant technologies of shake-and-catch, robots, and harvest-assist platform methods. In addition, based on the review it points out the bottlenecks and future trends of these three technology categories. Major progress in the shake-and-catch method is related to theoretical studies on the effective removal of apples and catching mechanisms to minimize bruising. The unacceptable bruising conditions hinder the shake-and-catch method from commercial application. Two startups of apple harvesting robots are in the stage of commercializing their products based on vacuum and three-finger end-effectors, respectively. Economic benefits, as well as technology reliability and robustness of both robots, are pending for validation before they are on the market. In addition, a key obstacle faced by both robots before commercial use is to find a solution to pick apples grown in clusters. Harvest-assist platforms are gradually adopted by apple growers, but at a very low rate due to their doubts on economic benefits. Validation of harvest-assist platforms’ economic benefits and incorporation with more functions (e.g., sorting) would enhance their adoption. With the rapid development of sensing and automation technologies, such as novel sensors, embedded systems, and machine learning algorithms, and the progress in new tree canopy structures that are friendlier for fruit visibility and accessibility, it is believed the robots for fresh market apple harvest would be realized and commercialized in the near future. Currently, more efforts should be invested in analyzing and validating the economic benefits of harvest-assist platforms, as well as adding more functions to the harvest-assist platforms, to increase their application rate for the benefit of the apple industry.

[1]  T. F. Burks,et al.  GEOMETRIC PERFORMANCE INDICES FOR ANALYSIS AND SYNTHESIS OF MANIPULATORS FOR ROBOTIC HARVESTING , 2006 .

[2]  D. Stajnko,et al.  Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging , 2004 .

[3]  E. Dursun,et al.  Mechanical Harvesting of Apricots , 2003 .

[4]  Rui Hai Liu,et al.  Apple phytochemicals and their health benefits , 2004, Nutrition journal.

[5]  Rui Li,et al.  Improved Kiwifruit Detection Using Pre-Trained VGG16 With RGB and NIR Information Fusion , 2020, IEEE Access.

[6]  Q. Zhang,et al.  Apple crop-load estimation with over-the-row machine vision system , 2016, Comput. Electron. Agric..

[7]  Qin Zhang,et al.  Air Suspension-based Catching Mechanism for Mechanical Harvesting of Apples , 2016 .

[8]  James J. Mehlschau,et al.  Harvesting Valencia Oranges with Flexible Curved Fingers , 1982 .

[9]  C. E. Schertz,et al.  Determining Fruit-Bearing Zones in Citrus , 1966 .

[10]  Renfu Lu,et al.  Design Features and Bruise Evaluation of an Apple Harvest and In-Field Presorting Machine , 2018 .

[11]  Yoav Sarig,et al.  Robotics of Fruit Harvesting: A State-of-the-art Review , 1993 .

[12]  Z. Zhang,et al.  Brush mechanism for distributing apples in a low-cost apple harvest-assist unit , 2017 .

[13]  Fadi A Fathallah,et al.  Musculoskeletal disorders in labor-intensive agriculture. , 2010, Applied ergonomics.

[14]  Paul Heinz Heinemann,et al.  Economic analysis of a low-cost apple harvest-assist unit , 2017 .

[15]  Renfu Lu,et al.  An image segmentation method for apple sorting and grading using support vector machine and Otsu's method , 2013 .

[16]  Raphael Linker,et al.  Determination of the number of green apples in RGB images recorded in orchards , 2012 .

[17]  David C. Slaughter,et al.  Discriminating Fruit for Robotic Harvest Using Color in Natural Outdoor Scenes , 1989 .

[18]  Zhao Dean,et al.  System Design and Control of an Apple Harvesting Robot , 2020, ArXiv.

[19]  Renfu Lu,et al.  Development and Preliminary Evaluation of a New Bin Filler for Apple Harvesting and In-Field Sorting Machine , 2017 .

[20]  Yuzhen Lu,et al.  Detection of Surface and Subsurface Defects of Apples Using Structured- Illumination Reflectance Imaging with Machine Learning Algorithms , 2018 .

[21]  Victor Alchanatis,et al.  A Multispectral Imaging Analysis for Enhancing Citrus Fruit Detection , 2010 .

[22]  Clément Gosselin,et al.  Underactuation in robotic grasping hands , 2002 .

[23]  Renfu Lu,et al.  Economic Evaluation of Apple Harvest and In-Field Sorting Technology , 2017 .

[24]  Marco Ceccarelli,et al.  Designing a robotic gripper for harvesting horticulture products , 2000, Robotica.

[25]  D. L. Peterson,et al.  A SYSTEMS APPROACH TO ROBOTIC BULK HARVESTING OF APPLES , 1999 .

[26]  John H. Miller,et al.  Design and Field Evaluation of a Robotic Apple Harvesting System with a 3D-Printed Soft-Robotic End-Effector , 2019, Transactions of the ASABE.

[27]  Chris McCarthy,et al.  Deep learning - Method overview and review of use for fruit detection and yield estimation , 2019, Comput. Electron. Agric..

[28]  G. E. Rehkugler,et al.  Apple Sorting with Machine Vision , 1985 .

[29]  David C. Slaughter,et al.  Color Vision in Robotic Fruit Harvesting , 1987 .

[30]  Juan Feng,et al.  Location of apples in trees using stereoscopic vision , 2015, Comput. Electron. Agric..

[31]  A. G. Berlage,et al.  Harvesting Apples with Straddle-Frame Trunk Shaker , 1974 .

[32]  Bayram Cetisli,et al.  Design of an automatic apple sorting system using machine vision , 2016, Comput. Electron. Agric..

[33]  Jun Li,et al.  Characterizing apple picking patterns for robotic harvesting , 2016, Comput. Electron. Agric..

[34]  D. L. Peterson,et al.  FRESH–MARKET QUALITY TREE FRUIT HARVESTER PART II: APPLES , 2003 .

[35]  Qin Zhang,et al.  Effect of Fruit Location on Apple Detachment with Mechanical Shaking , 2017 .

[36]  Renfu Lu,et al.  Improvements and Evaluation of an In-Field Bin Filler for Apple Bruising and Distribution , 2019 .

[37]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[38]  Shaohua Wan,et al.  Faster R-CNN for multi-class fruit detection using a robotic vision system , 2020, Comput. Networks.

[39]  Jude Liu,et al.  DESIGN AND FIELD TEST OF A LOW-COST APPLE HARVEST-ASSIST UNIT , 2016 .

[40]  Zhao Zhang,et al.  Ergonomic and efficiency analysis of conventional apple harvest process , 2019, International Journal of Agricultural and Biological Engineering.

[41]  Qin Zhang,et al.  Field evaluation of a mechanical-assist cherry harvesting system , 2016 .

[42]  R. E. Pitt,et al.  Models for the Rheology and Statistical Strength of Uniformly Stressed Vegetative Tissue , 1982 .

[43]  Manoj Karkee,et al.  APPLE IDENTIFICATION IN FIELD ENVIRONMENT WITH OVER THE ROW MACHINE VISION SYSTEM , 2014 .

[44]  D. Stajnko,et al.  Modelling apple fruit yield using image analysis for fruit colour, shape and texture. , 2009 .

[45]  Victor Alchanatis,et al.  Image fusion of visible and thermal images for fruit detection. , 2009 .

[46]  Verónica Vilaplana,et al.  Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities , 2019, Comput. Electron. Agric..

[47]  Antonio J. Torregrosa,et al.  Mechanical harvesting of oranges and mandarins in Spain , 2009 .

[48]  Minzan Li,et al.  A Positioning Method for Apple Fruits Based on Image Processing and Information Fusion , 2018 .

[49]  Bernard Gosselin,et al.  Original paper: Automatic grading of Bi-colored apples by multispectral machine vision , 2011 .

[50]  Zhaohua Zhang,et al.  The Role of a New Harvest Platform in Alleviation of Apple Workers' Occupational Injuries During Harvest. , 2019, Journal of agricultural safety and health.

[51]  Manoj Karkee,et al.  A HIERARCHICAL APPROACH TO APPLE IDENTIFICATION FOR ROBOTIC HARVESTING , 2016 .

[52]  Yuanshen Zhao,et al.  A review of key techniques of vision-based control for harvesting robot , 2016, Comput. Electron. Agric..

[53]  Renfu Lu,et al.  Cost Benefits Analysis of In-field Presorting for the Apple Industry , 2011 .

[54]  Jude Liu,et al.  The Development of Mechanical Apple Harvesting Technology: A Review , 2016 .

[55]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[56]  Jun Zhou,et al.  Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking , 2017, Comput. Electron. Agric..

[57]  U. A. Rosa,et al.  Development of a Canopy Volume Reduction Technique for Easy Assessment and Harvesting of Valencia Citrus Fruits , 2006 .

[58]  R. G. Diener,et al.  Vibration Characteristics of Trellis-Trained Apple Trees with Reference to Fruit Detachment , 1965 .

[59]  Qin Zhang,et al.  Determination of key canopy parameters for mass mechanical apple harvesting using supervised machine learning and principal component analysis (PCA) , 2020 .

[60]  Denis Stajnko,et al.  Original papers: Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry , 2011 .

[61]  J. Robert Cooke,et al.  Vibratory Harvesting: An Experimental Analysis of Fruit-Stem Dynamics , 1972 .

[62]  Won Suk Lee,et al.  Green citrus detection using 'eigenfruit', color and circular Gabor texture features under natural outdoor conditions , 2011 .

[63]  E. D. Markwardt,et al.  Effects of Tree Structure on Damage to Apples During Mechanical Harvesting , 1968 .

[64]  R. Zhou,et al.  Using colour features of cv. ‘Gala’ apple fruits in an orchard in image processing to predict yield , 2012, Precision Agriculture.

[65]  Hossein Pourghassem,et al.  Computer vision-based apple grading for golden delicious apples based on surface features , 2017 .

[66]  Jose L Pons,et al.  A SURVEY OF COMPUTER VISION METHODS FOR LOCATING FRUIT ON TREES , 2000 .

[67]  Andris Freivalds,et al.  Effect of belt/bucket interface in apple harvesting , 2006 .

[68]  Volkan Isler,et al.  Semantic mapping for orchard environments by merging two‐sides reconstructions of tree rows , 2018, J. Field Robotics.

[69]  Ofer Levi,et al.  Detection of Green Apples in Hyperspectral Images of Apple-Tree Foliage Using Machine Vision , 2007 .

[70]  José Blasco,et al.  Machine Vision System for Automatic Quality Grading of Fruit , 2003 .

[71]  C. T. Morrow,et al.  Over-the-Row Harvester for Dwarf Fruit Trees , 1972 .

[72]  Donald L. Peterson,et al.  Advances in mechanical harvesting of fresh market quality apples , 1989 .

[73]  E. A. Parrish,et al.  Pictorial Pattern Recognition Applied to Fruit Harvesting , 1977 .

[74]  Nuria Aleixos,et al.  Erratum to: Advances in Machine Vision Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables , 2011 .

[75]  Renfu Lu,et al.  A review of bin filling technologies for apple harvest and postharvest handling , 2018 .

[76]  M. J. Le Flufy.  The design of a prototype apple harvester , 1982 .

[77]  Xiangjun Zou,et al.  Color-, depth-, and shape-based 3D fruit detection , 2019, Precision Agriculture.

[78]  Robert D. Howe,et al.  The Highly Adaptive SDM Hand: Design and Performance Evaluation , 2010, Int. J. Robotics Res..

[79]  Manoj Karkee,et al.  A Semi-Automated Harvesting Prototype for Shaking Fruit Tree Limbs , 2015 .

[80]  Xiangjun Zou,et al.  In-field citrus detection and localisation based on RGB-D image analysis , 2019, Biosystems Engineering.

[81]  Ahmad Al-Mallahi,et al.  A novel image processing algorithm to separate linearly clustered kiwifruits , 2019, Biosystems Engineering.

[82]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[83]  José Blasco,et al.  Shock absorbing surfaces for collecting fruit during the mechanical harvesting of citrus , 2011 .

[84]  Zhao Zhang,et al.  Innovative technology for apple harvest and in-field sorting , 2017 .

[85]  Changki Mo,et al.  Hand-Picking Dynamic Analysis for Undersensed Robotic Apple Harvesting , 2016 .

[86]  R. G. Diener,et al.  The West Virginia University tree fruit harvester , 1982 .

[87]  Volkan Isler,et al.  A comparative study of fruit detection and counting methods for yield mapping in apple orchards , 2018, J. Field Robotics.

[88]  R. W. Guest,et al.  Improving the Processing Quality of Mechanically Harvested Apples , 1965 .

[89]  Yael Edan,et al.  Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead , 2014, J. Field Robotics.