A computer vision system for early stage grape yield estimation based on shoot detection

A vision system for automated yield estimation and variation mapping is proposed.The proposed method produces F1-score 0.90 in average over four experimental blocks.The developed shoot detection does not require manual labeling to build a classifier.The developed system only requires low-cost off-the-shelf image collection equipment.The best EL stage for imaging shoots is around EL stage 9 regarding yield estimation. Counting grapevine shoots early in the growing season is critical for adjusting management practices but is challenging to automate due to a range of environmental factors.This paper proposes a completely automatic system for grapevine yield estimation, comprised of robust shoot detection and yield estimation based on shoot counts produced from videos. Experiments were conducted on four vine blocks across two cultivars and trellis systems over two seasons. A novel shoot detection framework is presented, including image processing, feature extraction, unsupervised feature selection and unsupervised learning as a final classification step. Then a procedure for converting shoot counts from videos to yield estimates is introduced.The shoot detection framework accuracy was calculated to be 86.83% with an F1-score of 0.90 across the four experimental blocks. This was shown to be robust in a range of lighting conditions in a commercial vineyard. The absolute predicted yield estimation error of the system when applied to four blocks over two consecutive years ranged from 1.18% to 36.02% when the videos were filmed around E-L stage 9.The developed system has an advantage over traditional PCD mapping techniques in that yield variation maps can be obtained earlier in the season, thereby allowing farmers to adjust their management practices for improved outputs. The unsupervised feature selection algorithm combined with unsupervised learning removed the requirement for any prior training or labeling, greatly enhancing the applicability of the overall framework and allows full automation of shoot mapping on a large scale in vineyards.

[1]  Sanjiv Singh,et al.  Automated Visual Yield Estimation in Vineyards , 2014, J. Field Robotics.

[2]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[3]  Pilar Barreiro Elorza,et al.  A Comparison of Fuzzy Clustering Algorithms Applied to Feature Extraction on Vineyard , 2011 .

[4]  Mark Whitty,et al.  Spatial Map Generation from Low Cost Ground Vehicle Mounted Monocular Camera , 2016 .

[5]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[6]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[7]  Dennis Jarvis,et al.  Estimation of mango crop yield using image analysis - Segmentation method , 2013 .

[8]  Don P. Mitchell,et al.  Spectrally optimal sampling for distribution ray tracing , 1991, SIGGRAPH.

[9]  Scarlett Liu,et al.  Towards Automated Yield Estimation in Viticulture , 2013 .

[10]  A. Srikaew,et al.  Grape leaf disease detection from color imagery using hybrid intelligent system , 2008, 2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology.

[11]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[12]  C. Rosenberger,et al.  Grape Detection By Image Processing , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[13]  Stergios B. Fotopoulos,et al.  Introduction to Modern Nonparametric Statistics , 2004, Technometrics.

[14]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[15]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[16]  Malrey Lee,et al.  A computer Vision Algorithm for Tangerine Yield Estimation , 2013, BSBT 2013.

[17]  Eduard Clotet,et al.  Vineyard Yield Estimation Based on the Analysis of High Resolution Images Obtained with Artificial Illumination at Night , 2015, Sensors.

[18]  Qi Wang,et al.  Automated Crop Yield Estimation for Apple Orchards , 2012, ISER.

[19]  S. S. Iyengar,et al.  An Evaluation of Filter and Wrapper Methods for Feature Selection in Categorical Clustering , 2005, IDA.

[20]  I. Jolliffe.  Principal Component Analysis , 2002 .

[21]  Manuel A. Armada,et al.  Combination of RGB and Multispectral Imagery for Discrimination of Cabernet Sauvignon Grapevine Elements , 2013, Sensors.

[22]  Daniel Cremers,et al.  Field phenotyping of grapevine growth using dense stereo reconstruction , 2015, BMC Bioinformatics.

[23]  Scarlett Liu,et al.  Automatic grape bunch detection in vineyards with an SVM classifier , 2015, J. Appl. Log..

[24]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[25]  María-Paz Diago,et al.  Grapevine Yield and Leaf Area Estimation Using Supervised Classification Methodology on RGB Images Taken under Field Conditions , 2012, Sensors.