Measuring and modeling apple trees using time-of-flight data for automation of dormant pruning applications

Dormant pruning is one of the most expensive, labor-intensive, but, unavoidable procedure in the field of horticulture to ensure quality crop production. During winter, skilled farmers remove certain branches that are connected directly with the trunk of a tree carefully using a set of predefined rules. In order to reduce this dependence on a large manpower, our goal is to automate this pruning process by building 3D models of dormant apple trees, which eventually would be fed to an intelligent robotic system. In this paper, we present a semicircle fitting based robust 3D reconstruction scheme for modeling the trunk and primary branches of apple trees. The method involves estimating the diameter-error, creating semicircle fit model of the tree from a single depth image, and reconstructing the final 3D model of the tree by aligning a sequence of depth images. Analysis of the qualitative as well as the quantitative evaluations of our algorithm on five different dormant apple trees from our dataset under various indoor and outdoor environments demonstrate the effectiveness of the proposed framework for automatic 3D reconstruction. The results show that on an average, the proposed schemes provide a performance of 89.4% for correctly estimating the diameters of the primary branches with a tolerance of 5 mm and 100%c for correctly identifying the branches.

[1]  Richard A. Fournier,et al.  The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial lidar , 2009 .

[2]  Alexander Bucksch,et al.  SkelTre - Robust skeleton extraction from imperfect point clouds , 2010, Vis. Comput..

[3]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[4]  Norbert Pfeifer,et al.  Structuring laser-scanned trees using 3D mathematical morphology , 2004 .

[5]  Partha Pratim Das,et al.  Characterizations of Noise in Kinect Depth Images: A Review , 2014, IEEE Sensors Journal.

[6]  Pushmeet Kohli,et al.  When Can We Use KinectFusion for Ground Truth Acquisition , 2012 .

[7]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[8]  Fabio Morbidi,et al.  Easy-to-Use and Accurate Calibration of RGB-D Cameras from Spheres , 2013, PSIVT.

[9]  Ayan Chaudhury,et al.  A Method for Global Non-rigid Registration of Multiple Thin Structures , 2015, 2015 12th Conference on Computer and Robot Vision.

[10]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[12]  Olaf Hellwich,et al.  Automatic registration of unordered point clouds acquired by Kinect sensors using an overlap heuristic , 2015 .

[13]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[15]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[16]  Avinash C. Kak,et al.  Automation of dormant pruning in specialty crop production: An adaptive framework for automatic reconstruction and modeling of apple trees , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Shahram Izadi,et al.  Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[18]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[20]  Matt Olson,et al.  Automatic reconstruction of tree skeletal structures from point clouds , 2010, ACM Trans. Graph..

[21]  Amit Singer,et al.  Global Registration of Multiple Point Clouds Using Semidefinite Programming , 2013, SIAM J. Optim..

[22]  Marsette Vona,et al.  Moving Volume KinectFusion , 2012, BMVC.

[23]  Alexander Bucksch,et al.  CAMPINO : A skeletonization method for point cloud processing , 2008 .

[24]  Bikram Adhikari,et al.  3D reconstruction of appletrees for mechanical pruning , 2012 .

[25]  Bikram Adhikari,et al.  Identification of pruning branches in tall spindle apple trees for automated pruning , 2014 .

[26]  Junjie Cao,et al.  Point Cloud Skeletons via Laplacian Based Contraction , 2010, 2010 Shape Modeling International Conference.

[27]  Anne Verroust-Blondet,et al.  Extracting skeletal curves from 3D scattered data , 2000, The Visual Computer.

[28]  Qi Wang,et al.  Three-Dimensional Reconstruction of a Dormant Tree Using RGB-D Cameras , 2013 .

[29]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  S Q Jin,et al.  Novel Calibration and Lens Distortion Correction of 3D Reconstruction Systems , 2006 .