Automated segmentation of soybean plants from 3D point cloud using machine learning

Abstract Image-based plant phenotyping has become a promising method for high-throughput measurement of plant traits in breeding programs. Plant geometric features that are essential for understanding plant growth can be obtained from the point cloud built using three-dimensional (3D) reconstruction of plant imagery data. A key task in the data processing pipeline is the automated and accurate segmentation of individual plants. Machine learning is a promising approach due to its strong ability in the extraction of details from images and has been successfully applied in plant leaf segmentation from two-dimensional (2D) images. The aim of this paper was to evaluate the performance of three machine learning methods, i.e. boosting, Support Vector Machine (SVM) and K-means clustering, in the segmentation of non-overlapped and overlapped soybean plants at early growth stages using 3D point cloud. Images of 75 soybean plants at two growth stages in a greenhouse were collected using an image-based high-throughput phenotyping platform and were used to develop 3D point cloud using the Structure from Motion (SfM) method. Plant features including position (coordinate x, y, and z), and color (Red, Green, Blue, hue, saturation and Triangular Greenness Index) were used for background removal and the separation of non-overlapped plants. A Histogram of Oriented Gradient (HOG) descriptor was used for the separation of overlapped plants. The percentage of mismatched points between manual and automated segmentation was calculated and results showed that K-means clustering had the least mean error rates (0.36% and 0.20%) for the background removal and the non-overlapped plant separation. The least mean error rate for the separation of overlapped plants was 2.57% using SVM with labeled HOG descriptor. The developed image segmentation pipeline was evaluated in a case study where 69 plants at different growth stages were continuously monitored. Results showed that it took three minutes on average for completing all procedures in the pipeline and the extracted features (i.e. height and shooting area) were able to quantify the plant growth.

[1]  S. Tsaftaris,et al.  Phenotiki: an open software and hardware platform for affordable and easy image‐based phenotyping of rosette‐shaped plants , 2017, The Plant Journal.

[2]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[3]  Michael Schirrmann,et al.  Estimating wheat biomass by combining image clustering with crop height , 2016, Comput. Electron. Agric..

[4]  Jing Zhou,et al.  Development of an automated phenotyping platform for quantifying soybean dynamic responses to salinity stress in greenhouse environment , 2018, Comput. Electron. Agric..

[5]  Larry C. Purcell,et al.  Soybean Canopy Coverage and Light Interception Measurements Using Digital Imagery , 2000 .

[6]  Yufeng Ge,et al.  High Throughput In vivo Analysis of Plant Leaf Chemical Properties Using Hyperspectral Imaging , 2017, Front. Plant Sci..

[7]  Hanno Scharr,et al.  Leaf segmentation in plant phenotyping: a collation study , 2016, Machine Vision and Applications.

[8]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  P. Schnable,et al.  Field-based architectural traits characterisation of maize plant using time-of-flight 3D imaging , 2019, Biosystems Engineering.

[10]  Georgios Tzimiropoulos,et al.  A 3D Scene Analysis Framework and Descriptors for Risk Evaluation , 2015, 2015 International Conference on 3D Vision.

[11]  Y. Ge,et al.  Field-Based Scoring of Soybean Iron Deficiency Chlorosis Using RGB Imaging and Statistical Learning , 2018, Front. Plant Sci..

[12]  D. Rousseau,et al.  High throughput quantitative phenotyping of plant resistance using chlorophyll fluorescence image analysis , 2013, Plant Methods.

[13]  Stephen M. Welch,et al.  Plant high-throughput phenotyping using photogrammetry and imaging techniques to measure leaf length and rosette area , 2016, Comput. Electron. Agric..

[14]  Christian Klukas,et al.  3-D Histogram-Based Segmentation and Leaf Detection for Rosette Plants , 2014, ECCV Workshops.

[15]  Fabio Fiorani,et al.  The art of growing plants for experimental purposes: a practical guide for the plant biologist. , 2012, Functional plant biology : FPB.

[16]  Ulrich Schurr,et al.  Future scenarios for plant phenotyping. , 2013, Annual review of plant biology.

[17]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[18]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[19]  Matías Gámez,et al.  adabag: An R Package for Classification with Boosting and Bagging , 2013 .

[20]  Lei Tian,et al.  Development of methods to improve soybean yield estimation and predict plant maturity with an unmanned aerial vehicle based platform , 2016 .

[21]  Martin J. Wooster,et al.  High Throughput Field Phenotyping of Wheat Plant Height and Growth Rate in Field Plot Trials Using UAV Based Remote Sensing , 2016, Remote. Sens..

[22]  S. N. Geethalakshmi,et al.  Plant Leaf Segmentation Using Non Linear K means Clustering , 2012 .

[23]  H. Scharr,et al.  Simultaneous phenotyping of leaf growth and chlorophyll fluorescence via GROWSCREEN FLUORO allows detection of stress tolerance in Arabidopsis thaliana and other rosette plants. , 2009, Functional plant biology : FPB.

[24]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[25]  Johanna Link,et al.  Combined Spectral and Spatial Modeling of Corn Yield Based on Aerial Images and Crop Surface Models Acquired with an Unmanned Aircraft System , 2014, Remote. Sens..

[26]  Jin Chen,et al.  Multi-leaf alignment from fluorescence plant images , 2014, IEEE Winter Conference on Applications of Computer Vision.

[27]  C. Daughtry,et al.  Remote Sensing Leaf Chlorophyll Content Using a Visible Band Index , 2011 .

[28]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[29]  J. Vollmann,et al.  Original paper: Digital image analysis and chlorophyll metering for phenotyping the effects of nodulation in soybean , 2011 .

[30]  A R Smith,et al.  Color Gamut Transformation Pairs , 1978 .

[31]  A. Walter,et al.  REVIEW: PART OF A HIGHLIGHT ON BREEDING STRATEGIES FOR FORAGE AND GRASS IMPROVEMENT Advanced phenotyping offers opportunities for improved breeding of forage and turf species , 2012 .

[32]  Baskar Ganapathysubramanian,et al.  A real-time phenotyping framework using machine learning for plant stress severity rating in soybean , 2017, Plant Methods.

[33]  Jing Zhou,et al.  Evaluating Geometric Measurement Accuracy Based on 3D Reconstruction of Automated Imagery in a Greenhouse , 2018, Sensors.

[34]  H. Nguyen,et al.  Integrating omic approaches for abiotic stress tolerance in soybean , 2014, Front. Plant Sci..

[35]  I. Xenarios,et al.  Measuring the diurnal pattern of leaf hyponasty and growth in Arabidopsis - a novel phenotyping approach using laser scanning. , 2012, Functional plant biology : FPB.

[36]  W. Kruijer,et al.  Phenomics for photosynthesis, growth and reflectance in Arabidopsis thaliana reveals circadian and long-term fluctuations in heritability , 2016, Plant Methods.

[37]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.