Towards Autonomous Apple Fruitlet Sizing with Next Best View Planning

As a result of the expected effects of global population growth and the anticipated need to increase food production, agricultural robotics has become a popular research area. Robots are able to automate laborious and time-consuming tasks which allows farmers to make faster decisions and ultimately improves yield. In this paper, we build towards designing a robotic system that autonomously sizes apple fruitlets. Our proposed system adopts a viewpoint planning approach targeted towards sizing smaller fruit. We utilize a coarse and fine planning tree along with a region of interest utility-gain mechanism to generate next-best view candidates to capture images of fruitlets. A truncated signed distance function is used to build a dense surface point cloud and fruits are sized using a combination of 3D and 2D techniques. We provide preliminary simulated results demonstrating that our system can effectively size fruitlets in occluded environments.

[1]  Maren Bennewitz,et al.  Deep Reinforcement Learning for Next-Best-View Planning in Agricultural Applications , 2022, 2022 International Conference on Robotics and Automation (ICRA).

[2]  Maren Bennewitz,et al.  Fruit Mapping with Shape Completion for Autonomous Crop Monitoring , 2022, 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE).

[3]  Saquib Shaikh,et al.  Fruit Harvesting Robot Using Computer Vision , 2022, 2022 International Conference for Advancement in Technology (ICONAT).

[4]  Terry Bates,et al.  Bumblebee: A Path Towards Fully Autonomous Robotic Vine Pruning , 2021, Field Robotics.

[5]  Zachary Teed,et al.  RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching , 2021, 2021 International Conference on 3D Vision (3DV).

[6]  Maren Bennewitz,et al.  Viewpoint Planning for Fruit Size and Position Estimation , 2020, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Yawei Wang,et al.  Fruit Morphological Measurement Based on Three-Dimensional Reconstruction , 2020, Agronomy.

[8]  Juan Manuel Ponce,et al.  Automatic Counting and Individual Size and Mass Estimation of Olive-Fruits Through Computer Vision Techniques , 2019, IEEE Access.

[9]  Manoj Karkee,et al.  Apple fruit size estimation using a 3D machine vision system , 2018, Information Processing in Agriculture.

[10]  Brijesh Verma,et al.  In Field Fruit Sizing Using A Smart Phone Application , 2018, Sensors.

[11]  Kulbir Singh,et al.  Volumetric estimation using 3D reconstruction method for grading of fruits , 2018, Multimedia Tools and Applications.

[12]  Brijesh Verma,et al.  On-Tree Mango Fruit Size Estimation Using RGB-D Images , 2017, Sensors.

[13]  Hong Cheng,et al.  Early Yield Prediction Using Image Analysis of Apple Fruit and Tree Canopy Features with Neural Networks , 2017, J. Imaging.

[14]  Guilherme N. DeSouza,et al.  Vinobot and Vinoculer: Two Robotic Platforms for High-Throughput Field Phenotyping , 2017, Sensors.

[15]  Roland Siegwart,et al.  Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  James Patrick Underwood,et al.  Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry , 2016, Sensors.

[17]  Tristan Perez,et al.  Sweet pepper pose detection and grasping for automated crop harvesting , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Nikolaus Correll,et al.  Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study , 2014, ArXiv.

[19]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[20]  Edwin Olson,et al.  AprilTag: A robust and flexible visual fiducial system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[22]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[23]  Maren Bennewitz,et al.  Viewpoint Planning based on Shape Completion for Fruit Mapping and Reconstruction , 2022, ArXiv.

[24]  M. Kaess,et al.  Robotic Vision for 3D Modeling and Sizing in Agriculture , 2021 .