Machine Vision System for 3D Plant Phenotyping

Machine vision for plant phenotyping is an emerging research area for producing high throughput in agriculture and crop science applications. Since 2D based approaches have their inherent limitations, 3D plant analysis is becoming state of the art for current phenotyping technologies. We present an automated system for analyzing plant growth in indoor conditions. A gantry robot system is used to perform scanning tasks in an automated manner throughout the lifetime of the plant. A 3D laser scanner mounted as the robot's payload captures the surface point cloud data of the plant from multiple views. The plant is monitored from the vegetative to reproductive stages in light/dark cycles inside a controllable growth chamber. An efficient 3D reconstruction algorithm is used, by which multiple scans are aligned together to obtain a 3D mesh of the plant, followed by surface area and volume computations. The whole system, including the programmable growth chamber, robot, scanner, data transfer, and analysis is fully automated in such a way that a naive user can, in theory, start the system with a mouse click and get back the growth analysis results at the end of the lifetime of the plant with no intermediate intervention. As evidence of its functionality, we show and analyze quantitative results of the rhythmic growth patterns of the dicot Arabidopsis thaliana (L.), and the monocot barley (Hordeum vulgare L.) plants under their diurnal light/dark cycles.

[1]  Christian Klukas,et al.  Integrated Analysis Platform: An Open-Source Information System for High-Throughput Plant Phenotyping1[C][W][OPEN] , 2014, Plant Physiology.

[2]  Kourosh Khoshelham,et al.  Localized Registration of Point Clouds of Botanic Trees , 2013, IEEE Geoscience and Remote Sensing Letters.

[3]  I. Xenarios,et al.  Measuring the diurnal pattern of leaf hyponasty and growth in Arabidopsis - a novel phenotyping approach using laser scanning. , 2012, Functional plant biology : FPB.

[4]  Sebastian Riedel,et al.  Automated Analysis of Barley Organs Using 3D Laser Scanning: An Approach for High Throughput Phenotyping , 2014, Sensors.

[5]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Bin Wang,et al.  Automatic Registration of Tree Point Clouds From Terrestrial LiDAR Scanning for Reconstructing the Ground Scene of Vegetated Surfaces , 2014, IEEE Geoscience and Remote Sensing Letters.

[7]  Ayan Chaudhury,et al.  Junction-Based Correspondence Estimation of Plant Point Cloud Data Using Subgraph Matching , 2016, IEEE Geoscience and Remote Sensing Letters.

[8]  E. Garnier,et al.  Effect of water stress on stem diameter changes of peach trees growing in the field , 1986 .

[9]  Heiner Kuhlmann,et al.  Surface feature based classification of plant organs from 3D laserscanned point clouds for plant phenotyping , 2013, BMC Bioinformatics.

[10]  Carme Torras,et al.  3D modelling of leaves from color and ToF data for robotized plant measuring , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  H. Chui,et al.  A feature registration framework using mixture models , 2000, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. MMBIA-2000 (Cat. No.PR00737).

[12]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[13]  Kristian Kersting,et al.  Automated interpretation of 3D laserscanned point clouds for plant organ segmentation , 2015, BMC Bioinformatics.

[14]  Nathan D. Miller,et al.  Image analysis is driving a renaissance in growth measurement. , 2013, Current opinion in plant biology.

[15]  Andrea Tagliasacchi,et al.  Sparse Iterative Closest Point , 2013, Comput. Graph. Forum.

[16]  Toni Kazic,et al.  Extracting complex lesion phenotypes in Zea mays , 2015, Machine Vision and Applications.

[17]  Gert Kootstra,et al.  Validation of plant part measurements using a 3D reconstruction method suitable for high-throughput seedling phenotyping , 2015, Machine Vision and Applications.

[18]  Ulrich Schurr,et al.  Future scenarios for plant phenotyping. , 2013, Annual review of plant biology.

[19]  Carlos Fernández Andrés,et al.  Development of a Configurable Growth Chamber with a Computer Vision System to Study Circadian Rhythm in Plants , 2012, Sensors.

[20]  Nicola J. Ferrier,et al.  A high throughput robot system for machine vision based plant phenotype studies , 2012, Machine Vision and Applications.

[21]  T. Simonneau,et al.  Diurnal Changes in Stem Diameter Depend Upon Variations in Water Content: Direct Evidence in Peach Trees , 1993 .

[22]  Hanno Scharr,et al.  Modeling leaf growth of rosette plants using infrared stereo image sequences , 2015, Comput. Electron. Agric..

[23]  Qin Zhang,et al.  A Review of Imaging Techniques for Plant Phenotyping , 2014, Sensors.

[24]  Minh N. Do,et al.  CODE: Coherence Based Decision Boundaries for Feature Correspondence , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[26]  Ayan Chaudhury,et al.  A Method for Global Non-rigid Registration of Multiple Thin Structures , 2015, 2015 12th Conference on Computer and Robot Vision.

[27]  Hanno Scharr,et al.  Leaf segmentation in plant phenotyping: a collation study , 2016, Machine Vision and Applications.

[28]  David G. Kirkpatrick,et al.  On the shape of a set of points in the plane , 1983, IEEE Trans. Inf. Theory.

[29]  V. H. Blackman,et al.  The Compound Interest Law and Plant Growth , 1919 .

[30]  Christophe Godin,et al.  Application to the structural analysis of plants , 2019 .

[31]  Tsuhan Chen,et al.  Efficient feature extraction for 2D/3D objects in mesh representation , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[32]  P. Benfey,et al.  Advanced imaging techniques for the study of plant growth and development. , 2014, Trends in plant science.

[33]  Falk Schreiber,et al.  HTPheno: An image analysis pipeline for high-throughput plant phenotyping , 2011, BMC Bioinformatics.

[34]  Daniel Cremers,et al.  Field phenotyping of grapevine growth using dense stereo reconstruction , 2015, BMC Bioinformatics.

[35]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[36]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  The Arabidopsis Genome Initiative.  Analysis of the genome sequence of the flowering plant Arabidopsis thaliana , 2000, Nature.

[38]  J. Léon,et al.  High-precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants , 2014 .

[39]  Wei Xiong,et al.  An Adaptive Gaussian Mixture Model for Non-rigid Image Registration , 2012, Journal of Mathematical Imaging and Vision.

[40]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Yll Haxhimusa,et al.  A framework for the extraction of quantitative traits from 2D images of mature Arabidopsis thaliana , 2016, Machine Vision and Applications.

[42]  Rajnikant V. Patel,et al.  Computer Vision Based Autonomous Robotic System for 3D Plant Growth Measurement , 2015, 2015 12th Conference on Computer and Robot Vision.

[43]  José Luis Pons Rovira,et al.  Machine Vision and Applications Manuscript-nr. a Vision System Based on a Laser Range--nder Applied to Robotic Fruit Harvesting , 2022 .

[44]  John L. Barron,et al.  Oscillations in Corn Seedling Growth as Measured by Optical Flow , 1995 .

[45]  Mark A Brophy,et al.  Surface Reconstruction from Noisy and Sparse Data , 2015 .

[46]  Tony P. Pridmore,et al.  Surface Reconstruction of Plant Shoots from Multiple Views , 2014, ECCV Workshops.

[47]  Steve A. Kay,et al.  The ELF4-ELF3-LUX Complex Links the Circadian Clock to Diurnal Control of Hypocotyl Growth , 2011, Nature.

[48]  E. Garnier,et al.  [Dynamics of regional ecological frangibility under natural hazard stress: a case study in Qingping Town of Sichuan Province, Southwest China]. , 1986 .

[49]  Daniel Cohen-Or,et al.  Analyzing growing plants from 4D point cloud data , 2013, ACM Trans. Graph..

[50]  Tony P. Pridmore,et al.  A patch-based approach to 3D plant shoot phenotyping , 2016, Machine Vision and Applications.

[51]  M. Tester,et al.  Phenomics--technologies to relieve the phenotyping bottleneck. , 2011, Trends in plant science.

[52]  J. Fripp,et al.  A novel mesh processing based technique for 3D plant analysis , 2012, BMC Plant Biology.

[53]  Xiaopeng Zhang,et al.  Plant growth modelling and applications: the increasing importance of plant architecture in growth models. , 2007, Annals of botany.

[54]  Yongqiang Ye,et al.  Use of leaf color images to identify nitrogen and potassium deficient tomatoes , 2011, Pattern Recognit. Lett..

[55]  A. Liptay,et al.  1 Optical Flow to Measure Minute Increments in Plant , 1994 .

[56]  Jayme G. A. Barbedo,et al.  3D Plant Modeling: Localization, Mapping and Segmentation for Plant Phenotyping Using a Single Hand-held Camera , 2014, ECCV Workshops.

[57]  Roberto Toldo,et al.  Global registration of multiple point clouds embedding the Generalized Procrustes Analysis into an ICP framework , 2010 .

[58]  J G Huguet,et al.  A biophysical analysis of stem and root diameter variations in woody plants. , 2001, Plant physiology.

[59]  Etienne Belin,et al.  Simulation of image acquisition in machine vision dedicated to seedling elongation to validate image processing root segmentation algorithms , 2014 .

[60]  Thiago T. Santos,et al.  Flexible three-dimensional modeling of plants using low- resolution cameras and visual odometry , 2015, Machine Vision and Applications.

[61]  Hanno Scharr,et al.  Growth Signatures of Rosette Plants from Time-Lapse Video , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[62]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[63]  Richard M. Leahy,et al.  Non-rigid Image Registration Using Gaussian Mixture Models , 2012, WBIR.