Machine Vision and Applications Manuscript No. Non-destructive Automatic Leaf Area Measurements by Combining Stereo and Time-of-flight Images

Leaf area measurements are commonly obtained by destructive and laborious practice. This study shows how stereo and time-of-flight (ToF) images can be combined for non-destructive automatic leaf area measurements. The authors focus on some challenging plant images captured in a greenhouse environment, and show that even the state-of-the-art stereo methods produce unsatisfactory results. By transforming depth information in a ToF image to a localised search range for dense stereo, a global optimisation strategy is adopted for producing smooth results that preserve discontinuity. They also use edges of colour and disparity images for automatic leaf detection and develop a smoothing method necessary for accurately estimating surface area. In addition to show that combining stereo and ToF images gives superior qualitative and quantitative results, 149 automatic measurements on leaf area using the authors system in a validation trial have a correlation of 0.97 with true values and the root-mean-square error is 10.97 cm(2), which is 9.3% of the average leaf area. Their approach could potentially be applied for combining other modalities of images with large difference in image resolutions and camera positions.

[1]  Tony DeRose,et al.  Piecewise smooth surface reconstruction , 1994, SIGGRAPH.

[2]  Graham W. Horgan,et al.  Image analysis for the biological sciences , 1997 .

[3]  Guy Marchal,et al.  Automated multi-modality image registration based on information theory , 1995 .

[4]  Peter Schröder,et al.  Interpolating Subdivision for meshes with arbitrary topology , 1996, SIGGRAPH.

[5]  W. Cleveland,et al.  Smoothing by Local Regression: Principles and Methods , 1996 .

[6]  Christer Sjöström,et al.  State-of-the-art report , 1997 .

[7]  Ep Heuvelink,et al.  Modelling biomass production and yield of horticultural crops: a review , 1998 .

[8]  W. Rainey,et al.  The Fifth International , 1999 .

[9]  E. Heuvelink Evaluation of a Dynamic Simulation Model for Tomato Crop Growth and Development , 1999 .

[10]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  Guohua Pan,et al.  Local Regression and Likelihood , 1999, Technometrics.

[13]  M. Wand Local Regression and Likelihood , 2001 .

[14]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[17]  Frédéric Baret,et al.  Review of methods for in situ leaf area index determination Part I. Theories, sensors and hemispherical photography , 2004 .

[18]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[19]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[20]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[21]  Nobuo Ezaki,et al.  Measurement of plants by stereo vision for agricultural applications , 2005, SIP.

[22]  Yiannis Aloimonos,et al.  Shape and the Stereo Correspondence Problem , 2005, International Journal of Computer Vision.

[23]  Roland Wilson,et al.  Surface Modelling of Plants from Stereo Images , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[24]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[25]  Reinhard Koch,et al.  A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images , 2007, DAGM-Symposium.

[26]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[27]  Andreas Kolb,et al.  Sub-pixel data fusion and edge-enhanced distance refinement for 2D/3D images , 2008, Int. J. Intell. Syst. Technol. Appl..

[28]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[29]  Ryan Crabb,et al.  Real-time foreground segmentation via range and color imaging , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[30]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Marc Alexa,et al.  Combining Time-Of-Flight depth and stereo images without accurate extrinsic calibration , 2008, Int. J. Intell. Syst. Technol. Appl..

[32]  Yu Song,et al.  Image analysis and statistical modelling for measurement and quality assessment of ornamental horticulture crops in glasshouses , 2009 .

[33]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[34]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, CVPR.

[35]  Yuntao Ma,et al.  Fruit Set and Yield Patterns in Six Capsicum Cultivars , 2009 .

[36]  Reinhard Koch,et al.  Time-of-Flight Sensors in Computer Graphics , 2009, Eurographics.

[37]  M. Tester,et al.  Quantifying the three main components of salinity tolerance in cereals. , 2009, Plant, cell & environment.

[38]  C. Glasbey,et al.  Spy - See - Advanced vision system for phenotyping in greenhouses , 2009 .

[39]  Rasmus Larsen,et al.  Improved 3D reconstruction in smart-room environments using ToF imaging , 2010, Comput. Vis. Image Underst..

[40]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[42]  Timo Schairer,et al.  Fusion of range and color images for denoising and resolution enhancement with a non-local filter , 2010, Comput. Vis. Image Underst..

[43]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Chris A. Glasbey,et al.  Combining Stereo and Time-of-Flight Images with Application to Automatic Plant Phenotyping , 2011, SCIA.

[45]  Anders Heyden,et al.  Proceedings of the 17th Scandinavian conference on Image analysis , 2011 .

[46]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Carme Torras,et al.  3D modelling of leaves from color and ToF data for robotized plant measuring , 2011, 2011 IEEE International Conference on Robotics and Automation.

[48]  Reinhard Koch,et al.  Improved Video Segmentation by Adaptive Combination of Depth Keying and Mixture-of-Gaussians , 2011, SCIA.

[49]  C. Glasbey,et al.  SPICY: towards automated phenotyping of large pepper plants in the greenhouse. , 2012, Functional plant biology : FPB.

[50]  W. John Kress,et al.  Leafsnap: A Computer Vision System for Automatic Plant Species Identification , 2012, ECCV.

[51]  H. Poorter,et al.  Phenotyping plants: genes, phenes and machines. , 2012, Functional plant biology : FPB.

[52]  J. J. Magán,et al.  Genetic and QTL analyses of yield and a set of physiological traits in pepper , 2013, Euphytica.

[53]  D. Inzé,et al.  Cell to whole-plant phenotyping: the best is yet to come. , 2013, Trends in plant science.

[54]  Ulrich Schurr,et al.  Future scenarios for plant phenotyping. , 2013, Annual review of plant biology.

[55]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[56]  Ruigang Yang,et al.  TofCut: Towards Robust Real-time Foreground Extraction using Time-of-flight Camera , 2016 .