Multi-View Semantic Labeling of 3D Point Clouds for Automated Plant Phenotyping

Semantic labeling of 3D point clouds is important for the derivation of 3D models from real world scenarios in several economic fields such as building industry, facility management, town planning or heritage conservation. In contrast to these most common applications, we describe in this study the semantic labeling of 3D point clouds derived from plant organs by high-precision scanning. Our approach is optimized for the task of plant phenotyping with its very specific challenges and is employing a deep learning framework. Thereby, we report important experiences concerning detailed parameter initialization and optimization techniques. By evaluating our approach with challenging datasets we achieve state-of-the-art results without difficult and time consuming feature engineering as being necessary in traditional approaches to semantic labeling.

[1]  Heiner Kuhlmann,et al.  Surface feature based classification of plant organs from 3D laserscanned point clouds for plant phenotyping , 2013, BMC Bioinformatics.

[2]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[4]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Alexandre Boulch,et al.  SnapNet-R: Consistent 3D Multi-view Semantic Labeling for Robotics , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Stefan Hinz,et al.  CONTEXTUAL CLASSIFICATION OF POINT CLOUD DATA BY EXPLOITING INDIVIDUAL 3D NEIGBOURHOODS , 2015 .

[11]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[12]  M. Tester,et al.  Phenomics--technologies to relieve the phenotyping bottleneck. , 2011, Trends in plant science.

[13]  SchölerFlorian,et al.  Automated 3D reconstruction of grape cluster architecture from sensor data for efficient phenotyping , 2015 .

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Volker Steinhage,et al.  High-precision 3D detection and reconstruction of grapes from laser range data for efficient phenotyping based on supervised learning , 2017, Comput. Electron. Agric..

[16]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[17]  Marc Pollefeys,et al.  Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark , 2017, ArXiv.

[18]  Alexandre Boulch,et al.  Unstructured Point Cloud Semantic Labeling Using Deep Segmentation Networks , 2017, 3DOR@Eurographics.

[19]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Volker Steinhage,et al.  Automated 3D reconstruction of grape cluster architecture from sensor data for efficient phenotyping , 2015, Comput. Electron. Agric..

[21]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).