Instance segmentation for assessment of plant growth dynamics in artificial soilless conditions

The paper presents a technology for plant growth dynamics estimation in an artificial soilless system. The approach consists of a hardware setup for automated image acquisition, plant feeding system, conditional monitoring and a software for automatic leaves segmentation and tracking. The software part of the system relies on a convolution neural network for instance segmentation. To train the neural network a manually annotated dataset was made. We conducted experiments on salad. Observations were taken for 31 days with the fixed time frame of 30 minutes, resulting in a large image dataset for each plant. It was shown how obtained results on instance segmentation for a particular leaf can serve for detailed reconstruction of the dynamics of plant growth. Datasets and source code are publicly available: https://github.com/DmitriiShadrin/PlantGrowthDynamics.

[1]  Luc Van Gool,et al.  Semantic Instance Segmentation with a Discriminative Loss Function , 2017, ArXiv.

[2]  Richard S. Zemel,et al.  End-to-End Instance Segmentation with Recurrent Attention , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Toyoki Kozai,et al.  Plant Factory: An Indoor Vertical Farming System for Efficient Quality Food Production , 2015 .

[4]  Ross A Frick,et al.  Accurate inference of shoot biomass from high-throughput images of cereal plants , 2011, Plant Methods.

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Hanno Scharr,et al.  ARIGAN: Synthetic Arabidopsis Plants using Generative Adversarial Network , 2017, bioRxiv.

[7]  Philip H. S. Torr,et al.  Recurrent Instance Segmentation , 2015, ECCV.

[8]  P. C. P. De Silva,et al.  Ipanera: An Industry 4.0 based architecture for distributed soil-less food production systems , 2016, 2016 Manufacturing & Industrial Engineering Symposium (MIES).

[9]  Rupert Gerzer,et al.  Pervasive agriculture: Measuring and predicting plant growth using statistics and 2D/3D imaging , 2018, 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[10]  André R. S. Marçal,et al.  Evaluation of Features for Leaf Discrimination , 2013, ICIAR.

[11]  Uwe Schmidt,et al.  A new concept for aquaponic systems to improve sustainability, increase productivity, and reduce environmental impacts , 2015 .

[12]  S. Chapman,et al.  Dynamic monitoring of NDVI in wheat agronomy and breeding trials using an unmanned aerial vehicle , 2017 .

[13]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[14]  Hanno Scharr,et al.  Leaf segmentation in plant phenotyping: a collation study , 2016, Machine Vision and Applications.

[15]  Jin Chen,et al.  Multi-modality imagery database for plant phenotyping , 2016, Machine Vision and Applications.

[16]  Przemyslaw Prusinkiewicz,et al.  The use of plant models in deep learning: an application to leaf counting in rosette plants , 2018, Plant Methods.

[17]  Ahmad Nizar Harun,et al.  Plant growth optimization using variable intensity and Far Red LED treatment in indoor farming , 2015, 2015 International Conference on Smart Sensors and Application (ICSSA).

[18]  T. Sinclair,et al.  Physiological phenotyping of plants for crop improvement. , 2015, Trends in plant science.

[19]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[20]  C. Granier,et al.  Phenotyping and beyond: modelling the relationships between traits. , 2014, Current opinion in plant biology.

[21]  Jun Zhou,et al.  3D Plant Modelling via Hyperspectral Imaging , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[22]  S. Tsaftaris,et al.  Phenotiki: an open software and hardware platform for affordable and easy image‐based phenotyping of rosette‐shaped plants , 2017, The Plant Journal.

[23]  Weixing Cao,et al.  Predicting grain yield in rice using multi-temporal vegetation indices from UAV-based multispectral and digital imagery , 2017 .

[24]  J. Munkres.  ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[25]  Victor S. Lempitsky,et al.  Instance Segmentation by Deep Coloring , 2018, ArXiv.

[26]  T. Morimoto,et al.  Dynamic optimization of water temperature for maximizing leaf water content of tomato in hydroponics using an intelligent control technique , 2017 .

[27]  Frederik Coppens,et al.  High-resolution time-resolved imaging of in vitro Arabidopsis rosette growth. , 2014, The Plant journal : for cell and molecular biology.

[28]  Rajnikant V. Patel,et al.  Computer Vision Based Autonomous Robotic System for 3D Plant Growth Measurement , 2015, 2015 12th Conference on Computer and Robot Vision.

[29]  Niel Allen,et al.  Multispectral remote sensing for yield estimation using high-resolution imagery from an unmanned aerial vehicle , 2018, Commercial + Scientific Sensing and Imaging.

[30]  Michael P. Pound,et al.  Approaches to three-dimensional reconstruction of plant shoot topology and geometry. , 2016, Functional plant biology : FPB.

[31]  Atsushi Shimada,et al.  An Easy-to-Setup 3D Phenotyping Platform for KOMATSUNA Dataset , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[32]  Francois Tardieu,et al.  Modelling strategies for assessing and increasing the effectiveness of new phenotyping techniques in plant breeding. , 2019, Plant science : an international journal of experimental plant biology.

[33]  Qin Zhang,et al.  A Review of Imaging Techniques for Plant Phenotyping , 2014, Sensors.

[34]  Hanno Scharr,et al.  Finely-grained annotated datasets for image-based plant phenotyping , 2016, Pattern Recognit. Lett..

[35]  T. Sharkey,et al.  The relationship between leaf area growth and biomass accumulation in Arabidopsis thaliana , 2015, Front. Plant Sci..

[36]  Tobias Wojciechowski,et al.  Root phenomics of crops: opportunities and challenges. , 2009, Functional plant biology : FPB.

[37]  E. Steudle,et al.  Hydraulic conductivity of rice roots. , 2001, Journal of experimental botany.

[38]  Henry Yuliando,et al.  Soilless Culture System to Support Water Use Efficiency and Product Quality: A Review , 2015 .

[39]  F. Martínez,et al.  Investigating the effect of different soilless substrates on strawberry productivity and fruit composition , 2016 .

[40]  E. Heuvelink,et al.  Productivity of a building-integrated roof top greenhouse in a Mediterranean climate , 2017 .

[41]  Andrew Zisserman,et al.  Delving deeper into the whorl of flower segmentation , 2010, Image Vis. Comput..