A Bayesian framework for joint structure and colour based pixel-wise classification of grapevine proximal images

Abstract Estimating the spatial variability of basic agronomic parameters at the scale of the plant is of prime importance for the development and monitoring of Precision Agriculture applications. It is all the more crucial in viticulture where intra-plot variabilities are exacerbated. This paper focuses on the description of the structure of the canopy at the plant scale by proximal imaging. A new framework is proposed for the pixel-wise classification of the grapevine canopy into organs at different phenological stages. The proposed processing chain proceeds in four steps: (i) foreground extraction, (ii) pixel-wise feature extraction, (iii) pixel-wise classification and (iv) spatial regularization. Step (i) is based on colour information only. For step (ii), colour is represented using an RGB triplet while texture is captured using the local structure tensor (LST). Two variants are proposed to associate colour and LST information into a single Euclidean vector. Step (iii) is a Bayesian decision process based on the joint modelling of colour and texture using multivariate Gaussian distributions. Finally, step (iv) combines stochastic relaxation and morphological filtering, allowing for the spatial regularisation of the classification output. This processing chain is applied to the pixel-wise classification of proximal images into grapevine organs. Images were taken from two 0.2 ha plots planted with the red variety “Merlot Noir” in Bordeaux area. Images were taken from an embedded acquisition system at three key phenological stages: flowerhood falling, pea-sized berriesand berries touching (BBCH 68, 75 and 79). Results are produced with leave-one-out cross validations where models are estimated from only 15 images per stage containing about 3.2 × 10 6 labellised pixels. The resulting classification performances are measured in terms of recall and precision that reached overall between 85% and 95% depending on the stage while overall accuracies range between 88% and 93%.

[1]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[2]  J. Serra.  Introduction to mathematical morphology , 1986 .

[3]  N. Ayache,et al.  Log‐Euclidean metrics for fast and simple calculus on diffusion tensors , 2006, Magnetic resonance in medicine.

[4]  Brett Whelan,et al.  Establishing Management Classes for Broadacre Agricultural Production , 2007 .

[5]  Javier Guevara,et al.  Mechatronic terrestrial LiDAR for canopy porosity and crown surface estimation , 2018, Comput. Electron. Agric..

[6]  Marc Donias,et al.  Structure tensor Log-Euclidean statistical models for texture analysis , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[7]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Marco Vieri,et al.  The RHEA-project robot for tree crops pesticide application , 2013 .

[9]  R.G.V. Bramley,et al.  Precision Viticulture: Managing vineyard variability for improved quality outcomes , 2022, Managing Wine Quality.

[10]  Sanjiv Singh,et al.  Modeling and Calibrating Visual Yield Estimates in Vineyards , 2012, FSR.

[11]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[12]  Christian Germain,et al.  Real-time Fruit Detection Using Deep Neural Networks , 2018 .

[13]  Marc Donias,et al.  Structure Tensor Riemannian Statistical Models for CBIR and Classification of Remote Sensing Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Rachid Deriche,et al.  Texture and color segmentation based on the combined use of the structure tensor and the image components , 2008, Signal Process..

[15]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[16]  Adam J. Mathews,et al.  Visualizing and Quantifying Vineyard Canopy LAI Using an Unmanned Aerial Vehicle (UAV) Collected High Density Structure from Motion Point Cloud , 2013, Remote. Sens..

[17]  R. Oberti,et al.  The profitability of precision spraying on specialty crops: a technical–economic analysis of protection equipment at increasing technological levels , 2017, Precision Agriculture.

[18]  Bruno Tisseyre,et al.  New technologies and methodologies for site-specific viticulture , 2007 .

[19]  Scarlett Liu,et al.  A computer vision system for early stage grape yield estimation based on shoot detection , 2017, Comput. Electron. Agric..

[20]  Jonathan H. Manton,et al.  Riemannian Gaussian Distributions on the Space of Symmetric Positive Definite Matrices , 2015, IEEE Transactions on Information Theory.

[21]  Barna Keresztes,et al.  Potential of on-board colour imaging for in-field detection and counting of grape bunches at early fruiting stages , 2017 .