Modelling and analysis of plant image data for crop growth monitoring in horticulture

Plants can be characterised by a range of attributes, and measuring these attributes accurately and reliably is a major challenge for the horticulture industry. The measurement of those plant characteristics that are most relevant to a grower has previously been tackled almost exclusively by a combination of manual measurement and visual inspection. The purpose of this work is to propose an automated image analysis approach in order to provide an objective measure of plant attributes to remove subjective factors from assessment and to reduce labour requirements in the glasshouse. This thesis describes a stereopsis approach for estimating plant height, since height information cannot be easily determined from a single image. The stereopsis algorithm proposed in this thesis is efficient in terms of the running time, and is more accurate when compared with other algorithms. The estimated geometry, together with colour information from the image, are then used to build a statistical plant surface model, which represents all the information from the visible spectrum. A self-organising map approach can be adopted to model plant surface attributes, but the model can be improved by using a probabilistic model such as a mixture model formulated in a Bayesian framework. Details of both methods are discussed in this thesis. A Kalman filter is developed to track the plant model over time, extending the model to the time dimension, which enables smoothing of the noisy measurements to produce a development trend for a crop. The outcome of this work could lead to a number of potentially important applications in horticulture.

[1]  Long Quan,et al.  Image-based plant modeling , 2006, SIGGRAPH 2006.

[2]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[3]  J. Speyer,et al.  A stochastic analysis of a modified gain extended Kalman filter with applications to estimation with bearings only measurements , 1983 .

[4]  Simon Braun,et al.  Signal processing, the model-based approach: James V. Candy, 230 pages, McGraw-Hill, New York, 1986. , 1987 .

[5]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[7]  J. L. Roux An Introduction to the Kalman Filter , 2003 .

[8]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .

[10]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[11]  Tony DeRose,et al.  Mesh optimization , 1993, SIGGRAPH.

[12]  Alan V. Oppenheim,et al.  Discrete-time Signal Processing. Vol.2 , 2001 .

[13]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[14]  A multi-resolution, probabilistic approach to 2D inverse conductivity problems , 1988 .

[15]  Nasir M. Rajpoot,et al.  Bayesian Surface Estimation from Multiple Cameras Using a Prior Based on the Visual Hull and its Application to Image Based Rendering , 2007, BMVC.

[16]  Nigel M. Allinson,et al.  Towards the optimal Bayes classifier using an extended self-organising map , 1995 .

[17]  Yizhou Yu Surface Reconstruction from Unorganized Points Using Self-Organizing Neural Networks , 1999 .

[18]  William H. Press,et al.  Numerical recipes in C , 2002 .

[19]  Curt Wells The Kalman Filter in Finance , 1995 .

[20]  Carl E. Rasmussen,et al.  The Infinite Gaussian Mixture Model , 1999, NIPS.

[21]  David Vernon,et al.  Machine vision - automated visual inspection and robot vision , 1991 .

[22]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[23]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Long Quan,et al.  Image-based tree modeling , 2007, SIGGRAPH 2007.

[25]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[26]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[27]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Karl Mosler,et al.  A Cautionary Note on Likelihood Ratio Tests in Mixture Models , 2000 .

[29]  A.D. Jepson,et al.  The fast computation of disparity from phase differences , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  K. C. Chou,et al.  Multiscale recursive estimation, data fusion, and regularization , 1994, IEEE Trans. Autom. Control..

[31]  Samuel Kaski,et al.  Winner-take-all networks for physiological models of competitive learning , 1994, Neural Networks.

[32]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[33]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Min C. Shin,et al.  Does colorspace transformation make any difference on skin detection? , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[37]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[38]  S. Birchfiled A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling , 1998 .

[39]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[40]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[41]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[42]  William Stallings,et al.  Wireless Communications & Networks , 2002 .

[43]  Leif Kobbelt,et al.  A survey of point-based techniques in computer graphics , 2004, Comput. Graph..

[44]  Katherine A. Heller,et al.  Bayesian hierarchical clustering , 2005, ICML.

[45]  T. D. Harwood,et al.  Graphical tracking systems revisited: a practical approach to computer scheduling in horticulture , 2004 .

[46]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[47]  Larry H. Matthies,et al.  Kalman filter-based algorithms for estimating depth from image sequences , 1989, International Journal of Computer Vision.

[48]  J. V. Stafford,et al.  Imaging as a technique for assessment and control in the field , 1995 .

[49]  E. Feron,et al.  Hierarchical control of small autonomous helicopters , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[50]  Hui Xu,et al.  Knowledge and heuristic-based modeling of laser-scanned trees , 2007, TOGS.

[51]  C. Robert Mixtures of Distributions: Inference and Estimation , 1996 .

[52]  Andrew Zisserman,et al.  Delving into the Whorl of Flower Segmentation , 2007, BMVC.

[53]  Gerald Sommer,et al.  Dynamic Cell Structure Learns Perfectly Topology Preserving Map , 1995, Neural Computation.

[54]  K Nakayama,et al.  Stereoscopic Depth: Its Relation to Image Segmentation, Grouping, and the Recognition of Occluded Objects , 1989, Perception.

[55]  Jorma Laaksonen,et al.  SOM_PAK: The Self-Organizing Map Program Package , 1996 .

[56]  Mohinder S. Grewal,et al.  Global Positioning Systems, Inertial Navigation, and Integration , 2000 .

[57]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[60]  U. van Meeteren Quality models in horticulture need product quality: a rare but challenging field of exploration. , 1998 .

[61]  R. M. Mersereau,et al.  Structural image codebooks and the self-organizing feature map algorithm , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[62]  Roland Wilson,et al.  Surface Modelling of Plants from Stereo Images , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[63]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter. , 1991 .

[64]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  R. Schaller,et al.  Moore's law: past, present and future , 1997 .

[66]  Tomaso Poggio,et al.  Cooperative computation of stereo disparity , 1988 .

[67]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[68]  Mohan M. Trivedi,et al.  3-D Computer Vision Using Structured Light: Design, Calibration, and Implementation Issues , 1996, Adv. Comput..

[69]  C. D. Kuglin,et al.  The phase correlation image alignment method , 1975 .

[70]  Christopher K. I. Williams A MCMC Approach to Hierarchical Mixture Modelling , 1999, NIPS.

[71]  J. Bontsema,et al.  An Autonomous Robot for Harvesting Cucumbers in Greenhouses , 2002, Auton. Robots.

[72]  John A. Marchant,et al.  Modelling uncertainty in agricultural image analysis , 1997 .

[73]  Amir Averbuch,et al.  Robust phase correlation , 2004, ICPR 2004.

[74]  Anil K. Jain,et al.  Handbook of Fingerprint Recognition , 2005, Springer Professional Computing.

[75]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[76]  Shabtai Bittman,et al.  Automated monitoring of greenhouse crops , 2001 .

[77]  Jussi Parkkinen,et al.  Color features for quality control in ceramic tile industry , 2001 .

[78]  Paul S. Bradley,et al.  Refining Initial Points for K-Means Clustering , 1998, ICML.

[79]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[80]  W. Eric L. Grimson,et al.  Computational Experiments with a Feature Based Stereo Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[82]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[83]  Chang-Tsun Li,et al.  A Class of Discrete Multiresolution Random Fields and Its Application to Image Segmentation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[84]  D. Hagenbeek,et al.  Thermal and chlorophyll-fluorescence imaging distinguish plant-pathogen interactions at an early stage. , 2004, Plant & cell physiology.

[85]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[86]  Hai Tao,et al.  Stereo Matching via Learning Multiple Experts Behaviors , 2006, BMVC.

[87]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[88]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[89]  S. Bramble Image analysis for the biological sciences , 1996 .

[90]  Nasir M. Rajpoot,et al.  Photo-consistency and multiresolution methods for light field disparity estimation , 2005 .

[91]  A. Jazwinski Stochastic Processes and Filtering Theory , 1970 .

[92]  T. Kostiainen,et al.  Self-organizing map as a probability density model , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[93]  Graham W. Horgan,et al.  The statistical analysis of plant part appearance — a review , 2001 .

[94]  Hans Knutsson,et al.  Multiresolution Estimation of 2-d Disparity Using a Frequency Domain Approach , 1992 .

[95]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[96]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[97]  Randall Smith,et al.  Estimating Uncertain Spatial Relationships in Robotics , 1987, Autonomous Robot Vehicles.

[98]  H. Knutsson,et al.  A multiresolution stereopsis algorithm based on the Gabor representation , 1989 .

[99]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[100]  Daniel B. Rowe,et al.  Multivariate Bayesian Statistics: Models for Source Separation and Signal Unmixing , 2002 .

[101]  Qican Zhang,et al.  Local stereo matching with adaptive support-weight, rank transform and disparity calibration , 2008, Pattern Recognit. Lett..

[102]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[103]  J. C. Noordam,et al.  Automated rose cutting in greenhouses with 3D vision and robotics: analysis of 3D vision techniques for stem detection , 2005 .

[104]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[105]  Changming Sun,et al.  Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques , 2002, International Journal of Computer Vision.

[106]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[107]  R. Moe,et al.  CONTROL OF PLANT HEIGHT IN POINSETTIA BY TEMPERATURE DROP AND GRAPHICAL TRACKING. , 1992 .

[108]  Roland Wilson,et al.  MGMM: multiresolution Gaussian mixture models for computer vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[109]  Simon Clippingdale Multiresolution image modelling and estimation , 1988 .