Modeling Dormant Fruit Trees for Agricultural Automation

Dormant pruning of fruit trees is one of the most costly and labor-intensive activities in specialty crop production. We present a system that solves the first step in the process of automated pruning: accurately measuring and modeling the fruit trees. Our system employs a laser sensor to collect observations of fruit trees from multiple perspectives, and it uses these observations to measure parameters needed for pruning. A split-and-merge clustering algorithm divides the collected data into three sets of points: trunk candidates, junction point candidates, and branches. The trunk candidates and junction point candidates are then further refined by a robust fitting algorithm that models as cylinders each segment of the trunk and primary branches. In this work, we focus on measuring the diameters of the primary branches and the trunk, which are important factors in dormant pruning and can be obtained directly from the cylindrical models. We show that the results are qualitatively satisfactory using synthetic and real data. Our experiments with three synthetic and three real apple trees of two different varieties showed that the system is able to identify the primary branches with an average accuracy of 98% and estimate their diameters with an average error of 0.6 cm. Although the current implementation of the system is too slow for large-scale practical applications (it can measure approximately two trees per hour), our study shows that the proposed approach may serve as a fundamental building block of robotic pruners in the near future.

[1]  Gaurav S. Sukhatme,et al.  3D tree reconstruction from laser range data , 2009, 2009 IEEE International Conference on Robotics and Automation.

[2]  Yung-Sheng Chen,et al.  Image-based tree modeling from a few images with very narrow viewing range , 2008, The Visual Computer.

[3]  Radomír Mech,et al.  Self-organizing tree models for image synthesis , 2009, ACM Trans. Graph..

[4]  Martial Hebert,et al.  Natural terrain classification using three‐dimensional ladar data for ground robot mobility , 2006, J. Field Robotics.

[5]  Philippe Santenoise,et al.  Terrestrial laser scanning for measuring the solid wood volume, including branches, of adult standing trees in the forest environment , 2012 .

[6]  Sam Friedman,et al.  Automatic Procedural Modeling of Tree Structures in Point Clouds Using Wavelets , 2013, 2013 International Conference on 3D Vision.

[7]  Henry Medeiros,et al.  Measuring and modeling apple trees using time-of-flight data for automation of dormant pruning applications , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[8]  Burcu Akinci,et al.  A Comparative Analysis of Depth-Discontinuity and Mixed-Pixel Detection Algorithms , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[9]  Oliver Deussen,et al.  Approximate image-based tree-modeling using particle flows , 2007, ACM Trans. Graph..

[10]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  David C. Ferree,et al.  Apple Tree Performance with Mechanical Hedging or Root Pruning in Intensive Orchards , 1993 .

[12]  E. J. van Henten,et al.  Stem localization of sweet-pepper plants using the support wire as a visual cue , 2014 .

[13]  Richard A. Fournier,et al.  An architectural model of trees to estimate forest structural attributes using terrestrial LiDAR , 2011, Environ. Model. Softw..

[14]  Dong-Ming Yan,et al.  Efficient and robust reconstruction of botanical branching structure from laser scanned points , 2009, 2009 11th IEEE International Conference on Computer-Aided Design and Computer Graphics.

[15]  A. Escolà,et al.  Obtaining the three-dimensional structure of tree orchards from remote 2D terrestrial LIDAR scanning , 2009 .

[16]  Long Quan,et al.  Image-based tree modeling , 2007, ACM Trans. Graph..

[17]  Winfried Kurth,et al.  Relational Growth Grammars - A Parallel Graph Transformation Approach with Applications in Biology and Architecture , 2007, AGTIVE.

[18]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..

[19]  Jie Long,et al.  3D tree modeling using motion capture , 2012, 2012 IEEE 4th International Symposium on Plant Growth Modeling, Simulation, Visualization and Applications.

[20]  S. Sotoodeh OUTLIER DETECTION IN LASER SCANNER POINT CLOUDS , 2006 .

[21]  Chakkrit Preuksakarn,et al.  Reconstructing plant architecture from 3D laser scanner data. (Acquisition et validation de modèles architecturaux virtuels de plantes) , 2012 .

[22]  Pete Watt,et al.  Measuring forest structure with terrestrial laser scanning , 2005 .

[23]  Yael Edan,et al.  Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead , 2014, J. Field Robotics.

[24]  Marc Jaeger,et al.  Image-based lightweight tree modeling , 2009, VRCAI '09.

[25]  Norbert Pfeifer,et al.  Structuring laser-scanned trees using 3D mathematical morphology , 2004 .

[26]  P. Prusinkiewicz,et al.  Computational models of plant development and form. , 2012, The New phytologist.

[27]  C. Glasbey,et al.  SPICY: towards automated phenotyping of large pepper plants in the greenhouse. , 2012, Functional plant biology : FPB.

[28]  N. Pfeifer,et al.  AUTOMATIC RECONSTRUCTION OF SINGLE TREES FROM TERRESTRIAL LASER SCANNER DATA , 2004 .

[29]  S. Delagrange,et al.  Reconstruction and analysis of a deciduous sapling using digital photographs or terrestrial-LiDAR technology. , 2011, Annals of botany.

[30]  Matt Olson,et al.  Automatic reconstruction of tree skeletal structures from point clouds , 2010, ACM Trans. Graph..

[31]  Guilherme N. DeSouza,et al.  3-D Modeling of Real-World Objects Using Range and Intensity Images , 2005, Machine Learning and Robot Perception.

[32]  Hui Xu,et al.  Knowledge and heuristic-based modeling of laser-scanned trees , 2007, TOGS.

[33]  Martial Hebert,et al.  Analysis and Removal of Artifacts in 3-D LADAR Data , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[34]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[35]  Xiaopeng Zhang,et al.  Simple Reconstruction of Tree Branches from a Single Range Image , 2007, Journal of Computer Science and Technology.

[36]  Volker Steinhage,et al.  Towards an Automated 3D Reconstruction of Plant Architecture , 2011, AGTIVE.

[37]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[38]  Avinash C. Kak,et al.  Automation of dormant pruning in specialty crop production: An adaptive framework for automatic reconstruction and modeling of apple trees , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).