3D Imaging of Greenhouse Plants with an Inexpensive Binocular Stereo Vision System

Nowadays, 3D imaging of plants not only contributes to monitoring and managing plant growth, but is also becoming an essential part of high-throughput plant phenotyping. In this paper, an inexpensive (less than 70 USD) and portable platform with binocular stereo vision is established, which can be controlled by a laptop. In the stereo matching step, an efficient cost calculating measure—AD-Census—is integrated with the adaptive support-weight (ASW) approach to improve the ASW’s performance on real plant images. In the quantitative assessment, our stereo algorithm reaches an average error rate of 6.63% on the Middlebury datasets, which is lower than the error rates of the original ASW approach and several other popular algorithms. The imaging experiments using the proposed stereo system are carried out in three different environments including an indoor lab, an open field with grass, and a multi-span glass greenhouse. Six types of greenhouse plants are used in experiments; half of them are ornamentals and the others are greenhouse crops. The imaging accuracy of the proposed method at different baseline settings is investigated, and the results show that the optimal length of the baseline (distance between the two cameras of the stereo system) is around 80 mm for reaching a good trade-off between the depth accuracy and the mismatch rate for a plant that is placed within 1 m of the cameras. Error analysis from both theoretical and experimental sides show that for an object that is approximately 800 mm away from the stereo platform, the measured depth error of a single point is no higher than 5 mm, which is tolerable considering the dimensions of greenhouse plants. By applying disparity refinement, the proposed methodology generates dense and accurate point clouds of crops in different environments including an indoor lab, an outdoor field, and a greenhouse. Our approach also shows invariance against changing illumination in a real greenhouse, as well as the capability of recovering 3D surfaces of highlighted leaf regions. The method not only works on a binocular stereo system, but is also potentially applicable to a SFM-MVS (structure-from-motion and multiple-view stereo) system or any multi-view imaging system that uses stereo matching.

[1]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[2]  Christian Klukas,et al.  3-D Histogram-Based Segmentation and Leaf Detection for Rosette Plants , 2014, ECCV Workshops.

[3]  S. Fleck,et al.  3D-laser scanning: A non-destructive method for studying above-ground biomass and growth of juvenile trees , 2011 .

[4]  Görres Grenzdörffer,et al.  Imagine All the Plants: Evaluation of a Light-Field Camera for On-Site Crop Growth Monitoring , 2016, Remote. Sens..

[5]  Yongqiang Ye,et al.  Use of leaf color images to identify nitrogen and potassium deficient tomatoes , 2011, Pattern Recognit. Lett..

[6]  Philippe Santenoise,et al.  Terrestrial laser scanning for measuring the solid wood volume, including branches, of adult standing trees in the forest environment , 2012 .

[7]  Shigehiko Hayashi,et al.  Growth Measurement of a Community of Strawberries Using Three-Dimensional Sensor , 2015 .

[8]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[9]  H. Scharr,et al.  A stereo imaging system for measuring structural parameters of plant canopies. , 2007, Plant, cell & environment.

[10]  Xing Mei,et al.  On building an accurate stereo matching system on graphics hardware , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[11]  Malia A. Gehan,et al.  Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. , 2015, Current opinion in plant biology.

[12]  Carme Torras,et al.  3D modelling of leaves from color and ToF data for robotized plant measuring , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Jin Chen,et al.  Multi-leaf alignment from fluorescence plant images , 2014, IEEE Winter Conference on Applications of Computer Vision.

[14]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Vamshhi Pavan Kumar Varma Vegeshna,et al.  Stereo Matching with Color-Weighted Correlation, Hierachical Belief Propagation and Occlusion Handling , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Yung-Sheng Chen,et al.  Leaf segmentation, classification, and three-dimensional recovery from a few images with close viewpoints , 2011 .

[18]  David Jones,et al.  Individual leaf extractions from young canopy images using Gustafson-Kessel clustering and a genetic algorithm , 2006 .

[19]  Erik D. Goodman,et al.  Digitization and Visualization of Greenhouse Tomato Plants in Indoor Environments , 2015, Sensors.

[20]  Hui Xu,et al.  Knowledge and heuristic-based modeling of laser-scanned trees , 2007, TOGS.

[21]  Philippe Lucidarme,et al.  On the use of depth camera for 3D phenotyping of entire plants , 2012 .

[22]  André Ricardo Backes,et al.  Plant Leaf Identification Based on Volumetric Fractal Dimension , 2009, Int. J. Pattern Recognit. Artif. Intell..

[23]  David Jones,et al.  Intensified fuzzy clusters for classifying plant, soil, and residue regions of interest from color images , 2004 .

[24]  Margrit Gelautz,et al.  Secrets of adaptive support weight techniques for local stereo matching , 2013, Comput. Vis. Image Underst..

[25]  M. Stitt,et al.  Phytotyping(4D) : a light-field imaging system for non-invasive and accurate monitoring of spatio-temporal plant growth. , 2015, The Plant journal : for cell and molecular biology.

[26]  Gonzalo Pajares,et al.  A new Expert System for greenness identification in agricultural images , 2013, Expert Syst. Appl..

[27]  Lutz Plümer,et al.  Low-Cost 3D Systems: Suitable Tools for Plant Phenotyping , 2014, Sensors.

[28]  Won Suk Lee,et al.  Identifying blueberry fruit of different growth stages using natural outdoor color images , 2014 .

[29]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[30]  Manuel A. Armada,et al.  Combination of RGB and Multispectral Imagery for Discrimination of Cabernet Sauvignon Grapevine Elements , 2013, Sensors.

[31]  Hanno Scharr,et al.  Leaf segmentation in plant phenotyping: a collation study , 2016, Machine Vision and Applications.

[32]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  W. Diepenbrock,et al.  A method to extract morphological traits of plant organs from 3D point clouds as a database for an architectural plant model , 2007 .

[34]  Roemi Fernández,et al.  Multisensory System for Fruit Harvesting Robots. Experimental Testing in Natural Scenarios and with Different Kinds of Crops , 2014, Sensors.

[35]  David Reiser,et al.  3D Maize Plant Reconstruction Based on Georeferenced Overlapping LiDAR Point Clouds , 2015, Remote. Sens..

[36]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[37]  Chengliang Liu,et al.  Algorithm based on marker-controlled watershed transform for overlapping plant fruit segmentation , 2009 .

[38]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Odemir Martinez Bruno,et al.  Fractal dimension applied to plant identification , 2008, Inf. Sci..

[40]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  S. Chatterjee,et al.  Quantization error analysis in stereo vision , 1992, [1992] Conference Record of the Twenty-Sixth Asilomar Conference on Signals, Systems & Computers.

[42]  Lars Kuhnert,et al.  Structure overview of vegetation detection. A novel approach for efficient vegetation detection using an active lighting system , 2012, Robotics Auton. Syst..

[43]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  S. Chapman,et al.  Dynamic quantification of canopy structure to characterize early plant vigour in wheat genotypes , 2016, Journal of experimental botany.

[45]  Daniel Cohen-Or,et al.  Analyzing growing plants from 4D point cloud data , 2013, ACM Trans. Graph..

[46]  Gonzalo Pajares,et al.  Automatic expert system for weeds/crops identification in images from maize fields , 2013, Expert Syst. Appl..

[47]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.