Three-dimensional photogrammetric mapping of cotton bolls in situ based on point cloud segmentation and clustering

Abstract Three-dimensional high throughput plant phenotyping techniques provide an opportunity to measure plant organ-level traits which can be highly useful to plant breeders. The number and locations of cotton bolls, which are the fruit of cotton plants and an important component of fiber yield, are arguably among the most important phenotypic traits but are complex to quantify manually. Hence, there is a need for effective and efficient cotton boll phenotyping solutions to support breeding research and monitor the crop yield leading to better production management systems. We developed a novel methodology for 3D cotton boll mapping within a plot in situ. Point clouds were reconstructed from multi-view images using the structure from motion algorithm. The method used a region-based classification algorithm that successfully accounted for noise due to sunlight. The developed density-based clustering method could estimate boll counts for this situation, in which bolls were in direct contact with other bolls. By applying the method to point clouds from 30 plots of cotton plants, boll counts, boll volume and position data were derived. The average accuracy of boll counting was up to 90% and the R2 values between fiber yield and boll number, as well as fiber yield and boll volume were 0.87 and 0.66, respectively. The 3D boll spatial distribution could also be analyzed using this method. This method, which was low-cost and provided improved site-specific data on cotton bolls, can also be applied to other plant/fruit mapping analysis after some modification.

[1]  Akihiro Sugimoto,et al.  Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understanding , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[2]  I. Colomina,et al.  Unmanned aerial systems for photogrammetry and remote sensing: A review , 2014 .

[3]  Yin Zhou,et al.  VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Slawomir T. Wierzchon,et al.  Modern Algorithms of Cluster Analysis , 2018 .

[5]  Markus Vincze,et al.  Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Giancarlo Troni,et al.  A pattern recognition strategy for visual grape bunch detection in vineyards , 2018, Comput. Electron. Agric..

[7]  Armin B. Cremers,et al.  DeepCotton: in-field cotton segmentation using deep fully convolutional network , 2017, J. Electronic Imaging.

[8]  Yu Jiang,et al.  3D computer vision and machine learning based technique for high throughput cotton boll mapping under field conditions , 2018 .

[9]  Alison L. Thompson,et al.  Professor: A motorized field-based phenotyping cart , 2018, HardwareX.

[10]  Glen L. Ritchie,et al.  Cotton growth and development , 2007 .

[11]  Gonzalo Pajares,et al.  Overview and Current Status of Remote Sensing Applications Based on Unmanned Aerial Vehicles (UAVs) , 2015 .

[12]  Rodel D. Lasco,et al.  A LiDAR-based flood modelling approach for mapping rice cultivation areas in Apalit, Pampanga , 2017 .

[13]  S. Sankaran,et al.  Low-altitude, high-resolution aerial imaging systems for row and field crop phenotyping: A review , 2015 .

[14]  James Patrick Underwood,et al.  Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry , 2016, Sensors.

[15]  Q. Guo,et al.  Crop 3D—a LiDAR based platform for 3D high-throughput crop phenotyping , 2018, Science China Life Sciences.

[16]  Shawn M. Kaeppler,et al.  A robust, high‐throughput method for computing maize ear, cob, and kernel attributes automatically from images , 2017, The Plant journal : for cell and molecular biology.

[17]  Juan Feng,et al.  Location of apples in trees using stereoscopic vision , 2015, Comput. Electron. Agric..

[18]  Qin Zhang,et al.  A Review of Imaging Techniques for Plant Phenotyping , 2014, Sensors.

[19]  Terry Townsend,et al.  Natural Fibres and the World Economy , 2016 .

[20]  Huang Yanbo,et al.  Cotton Yield Estimation Using Very High-Resolution Digital Images Acquired with a Low-Cost Small Unmanned Aerial Vehicle , 2016 .

[21]  Jeffrey W. White,et al.  Development and evaluation of a field-based high-throughput phenotyping platform. , 2013, Functional plant biology : FPB.

[22]  P. Langridge,et al.  Breeding Technologies to Increase Crop Production in a Changing World , 2010, Science.

[23]  Florentin Wörgötter,et al.  Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[25]  Dong-Ming Yan,et al.  Realistic Procedural Plant Modeling from Multiple View Images , 2020, IEEE Transactions on Visualization and Computer Graphics.

[26]  Achim Walter,et al.  The ETH field phenotyping platform FIP: a cable-suspended multi-sensor system. , 2016, Functional plant biology : FPB.

[27]  J. Léon,et al.  High-precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants , 2014 .

[28]  James Patrick Underwood,et al.  Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards , 2016, J. Field Robotics.

[29]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[30]  Luke Wallace,et al.  Non‐destructive estimation of above‐ground surface and near‐surface biomass using 3D terrestrial remote sensing techniques , 2017 .

[31]  Michael P. Pound,et al.  Approaches to three-dimensional reconstruction of plant shoot topology and geometry. , 2016, Functional plant biology : FPB.

[32]  William L. Rooney,et al.  Automated detection and measurement of individual sorghum panicles using density-based clustering of terrestrial lidar data , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[33]  Xiangjun Zou,et al.  Localisation of litchi in an unstructured environment using binocular stereo vision , 2016 .

[34]  Jun Zhou,et al.  Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking , 2017, Comput. Electron. Agric..

[35]  P. Schnable,et al.  Field-based architectural traits characterisation of maize plant using time-of-flight 3D imaging , 2019, Biosystems Engineering.

[36]  Byron Boots,et al.  4D crop monitoring: Spatio-temporal reconstruction for agriculture , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[38]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[39]  Xinkai Zhu,et al.  Estimation of biomass in wheat using random forest regression algorithm and remote sensing data , 2016 .

[40]  Jin Chen,et al.  Joint Multi-Leaf Segmentation, Alignment, and Tracking for Fluorescence Plant Videos , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  D. Cordell,et al.  The story of phosphorus: Global food security and food for thought , 2009 .

[42]  C. Stevens,et al.  A Statistical Description of Plant Shoot Architecture , 2017, Current Biology.

[43]  Nelson L. Max,et al.  Structured Light-Based 3D Reconstruction System for Plants , 2015, Sensors.

[44]  Yi Lin,et al.  LiDAR: An important tool for next-generation phenotyping technology of high potential for plant phenomics? , 2015, Comput. Electron. Agric..

[45]  Changying Li,et al.  Multispectral imaging and unmanned aerial systems for cotton plant phenotyping , 2019, PloS one.

[46]  Patrick S Schnable,et al.  A high-throughput , field-based phenotyping technology for tall biomass crops , 2018 .

[47]  A. Raftery,et al.  World population stabilization unlikely this century , 2014, Science.

[48]  Armin B. Cremers,et al.  In-field cotton detection via region-based semantic image segmentation , 2016, Comput. Electron. Agric..

[49]  Sheikh Ziauddin,et al.  Detection and Counting of On-Tree Citrus Fruit for Crop Yield Estimation , 2016 .

[50]  Jin Liu,et al.  Stem–Leaf Segmentation and Phenotypic Trait Extraction of Individual Maize Using Terrestrial LiDAR Data , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[51]  S. Chapman,et al.  Dynamic quantification of canopy structure to characterize early plant vigour in wheat genotypes , 2016, Journal of experimental botany.

[52]  David Reiser,et al.  3-D Imaging Systems for Agricultural Applications—A Review , 2016, Sensors.

[53]  Frédéric Baret,et al.  Wheat ear detection in plots by segmenting mobile laser scanning data , 2017 .

[54]  Khaled M. Elleithy,et al.  Sensor Fusion Based Model for Collision Free Mobile Robot Navigation , 2015, Sensors.

[55]  G. Ritchie,et al.  Contribution of Boll Mass and Boll Number to Irrigated Cotton Yield , 2015 .

[56]  Daniel Munoz,et al.  Inference Machines: Parsing Scenes via Iterated Predictions , 2013 .

[57]  Changying Li,et al.  In-field High Throughput Phenotyping and Cotton Plant Growth Analysis Using LiDAR , 2018, Front. Plant Sci..

[58]  Xu Wang,et al.  Development of a field-based high-throughput mobile phenotyping platform , 2016, Comput. Electron. Agric..

[59]  Changying Li,et al.  Quantitative Analysis of Cotton Canopy Size in Field Conditions Using a Consumer-Grade RGB-D Camera , 2018, Front. Plant Sci..

[60]  A. Ranjan,et al.  Enhancing crop yield by optimizing plant developmental features , 2016, Development.

[61]  Grant D. Pearse,et al.  Comparison of optical LAI measurements under diffuse and clear skies after correcting for scattered radiation , 2016 .