Computer Vision and Machine Learning for Viticulture Technology

This paper gives two contributions to the state-of-the-art for viticulture technology research. First, we present a comprehensive review of computer vision, image processing, and machine learning techniques in viticulture. We summarize the latest developments in vision systems and techniques with examples from various representative studies, including, harvest yield estimation, vineyard management and monitoring, grape disease detection, quality evaluation, and grape phenology. We focus on how computer vision and machine learning techniques can be integrated into current vineyard management and vinification processes to achieve industry relevant outcomes. The second component of the paper presents the new GrapeCS-ML database which consists of images of grape varieties at different stages of development together with the corresponding ground truth data (e.g., pH and Brix) obtained from chemical analysis. One of the objectives of this database is to motivate computer vision and machine learning researchers to develop practical solutions for deployment in smart vineyards. We illustrate the usefulness of the database for a color-based berry detection application for white and red cultivars and give baseline comparisons using various machine learning approaches and color spaces. This paper concludes by highlighting future challenges that need to be addressed prior to successful implementation of this technology in the viticulture industry.

[1]  Tien-Fu Lu,et al.  Image Processing and Analysis for Autonomous Grapevine Pruning , 2006, 2006 International Conference on Mechatronics and Automation.

[2]  Jaime Lloret,et al.  A Wireless Sensor Network for Vineyard Monitoring That Uses Image Processing , 2011, Sensors.

[3]  Michelle Moyer,et al.  Vineyard yield estimation , 2015 .

[4]  María-Paz Diago,et al.  Grapevine Yield and Leaf Area Estimation Using Supervised Classification Methodology on RGB Images Taken under Field Conditions , 2012, Sensors.

[5]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[6]  Marco Vieri,et al.  THE NEW ARCHITECTURE IN THE VINEYARD SYSTEM MANAGEMENT FOR VARIABLE RATE TECHNOLOGIES AND TRACEABILITY , 2013 .

[7]  Cemagref,et al.  Grape berry calibration by computer vision using elliptical model fitting , 2010 .

[8]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[9]  Scarlett Liu,et al.  Towards Automated Yield Estimation in Viticulture , 2013 .

[10]  Bryan Hed,et al.  Relationship Between Cluster Compactness and Bunch Rot in Vignoles Grapes. , 2009, Plant disease.

[11]  Suzy Y. Rogiers,et al.  Potassium in the Grape (Vitis vinifera L.) Berry: Transport and Function , 2017, Front. Plant Sci..

[12]  J. Blackman,et al.  Grapevine bunch rots: impacts on wine composition, quality, and potential procedures for the removal of wine faults. , 2013, Journal of agricultural and food chemistry.

[13]  P. Javier Herrera,et al.  A Novel Approach for Weed Type Classification Based on Shape Descriptors and a Fuzzy Decision-Making Method , 2014, Sensors.

[14]  I. Colomina,et al.  Unmanned aerial systems for photogrammetry and remote sensing: A review , 2014 .

[15]  Guanlin Li,et al.  Image Recognition of Grape Downy Mildew and Grape Powdery Mildew Based on Support Vector Machine , 2011, CCTA.

[16]  José Blasco,et al.  A new method for assessment of bunch compactness using automated image analysis , 2015 .

[17]  J. A. Martínez,et al.  Viticultura de precisión: predicción de cosecha a partir de variables de cultivo e índices de vegetación , 2005 .

[18]  Mark Whitty,et al.  A Lightweight Method for Grape Berry Counting based on Automated 3D Bunch Reconstruction from a Single Image , 2015 .

[19]  A. Srikaew,et al.  Grape leaf disease detection from color imagery using hybrid intelligent system , 2008, 2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology.

[20]  G. Dunn,et al.  Yield prediction from digital image analysis: A technique with potential for vineyard assessments prior to harvest , 2008 .

[21]  C. Rosenberger,et al.  Grape Detection By Image Processing , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[22]  Jordi Llorens,et al.  Development of two portable patternators to improve drift control and operator training in the operation of vineyard sprayers. , 2013 .

[23]  Tiago M. Fernández-Caramés,et al.  VineSens: An Eco-Smart Decision-Support Viticulture System , 2017, Sensors.

[24]  Neeraj Bhaskar Wadekar,et al.  Detection and Controlling of Grape Leaf Diseases using Image Processing and Embedded System , 2015 .

[25]  R. D. Tillett,et al.  Image Analysis for Pruning of Long Wood Grape Vines , 1997 .

[26]  N. G. Nair,et al.  Programming applications of dicarboximides to control bunch rot of grapes caused by Botrytis cinerea , 1987 .

[27]  Pilar Barreiro Elorza,et al.  Characterization of vineyard's canopy through fuzzy clustering and svm over color images , 2012 .

[28]  Mahesh Manik Kumbhar,et al.  Grape Leaf Diseases Detection & Analysisusing SGDM Matrix Method , 2014 .

[29]  Xiangjun Zou,et al.  Robust Grape Cluster Detection in a Vineyard by Combining the AdaBoost Framework and Multiple Color Components , 2016, Sensors.

[30]  Marco Mora,et al.  Shadow Detection in Complex Images Using Neural Networks: Application to Wine Grape Seed Segmentation , 2013, ICANNGA.

[31]  Pilar Barreiro,et al.  A Review of Wireless Sensor Technologies and Applications in Agriculture and Food Industry: State of the Art and Current Trends , 2009, Sensors.

[32]  Jenna Burrell,et al.  Vineyard computing: sensor networks in agricultural production , 2004, IEEE Pervasive Computing.

[33]  Sanjiv Singh,et al.  Yield estimation in vineyards by visual grape detection , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Eileen S. Scott,et al.  Effects of powdery mildew on the sensory properties and composition of Chardonnay juice and wine when grape sugar ripeness is standardised , 2005 .

[35]  M. P. Diago,et al.  Automatic estimation of the size and weight of grapevine berries by image analysis , 2012 .

[36]  Eduard Clotet,et al.  Vineyard Yield Estimation Based on the Analysis of High Resolution Images Obtained with Artificial Illumination at Night , 2015, Sensors.

[37]  Lucien Hoffmann,et al.  Impact of Grape Cluster Division on Cluster Morphology and Bunch Rot Epidemic , 2012, American Journal of Enology and Viticulture.

[38]  Salviano F. S. P. Soares,et al.  Automatic Detection of White Grapes in Natural Environment Using Image Processing , 2011, SOCO.

[39]  Qinghua Yang,et al.  Detection Method for the Buds on Winter Vines Based on Computer Vision , 2014, 2014 Seventh International Symposium on Computational Intelligence and Design.

[40]  Marco Mora,et al.  Grape maturity estimation based on seed images and neural networks , 2014, Eng. Appl. Artif. Intell..

[41]  Marco Mora,et al.  A method to estimate Grape Phenolic Maturity based on seed images , 2014 .

[42]  Sanjiv Singh,et al.  Automated Visual Yield Estimation in Vineyards , 2014, J. Field Robotics.

[43]  Alessandro Matese,et al.  Technology in precision viticulture: a state of the art review , 2015 .

[44]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[45]  Yogesh Dandawate,et al.  Detection and classification of diseases of Grape plant using opposite colour Local Binary Pattern feature and machine learning for automated Decision Support System , 2016, 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN).

[46]  Scarlett Liu,et al.  Automatic grape bunch detection in vineyards for precise yield estimation , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[47]  Facundo Bromberg,et al.  Image classification for detection of winter grapevine buds in natural conditions using scale-invariant features transform, bag of features and support vector machines , 2017, Comput. Electron. Agric..

[48]  Jamil Salem Barbar,et al.  Multilayer Perceptron and Stacked Autoencoder for Internet Traffic Prediction , 2014, NPC.

[49]  J. Santiago,et al.  Resistance of Eight Different Clones of the Grape Cultivar Albariño to Plasmopara viticola. , 2004, Plant disease.

[50]  José Antonio Martínez Casasnovas,et al.  Viticultura de precisión: Predicción de cosecha a partir de variables del cultivo e índices de vegetación. , 2005 .

[51]  Xiangjun Zou,et al.  Vision-based extraction of spatial information in grape clusters for harvesting robots , 2016 .

[52]  Hai Jun Yang,et al.  Boosted decision trees, a powerful event classifier , 2006 .

[53]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[54]  Sanjiv Singh,et al.  Modeling and Calibrating Visual Yield Estimates in Vineyards , 2012, FSR.

[55]  Javier Ibáñez,et al.  Evaluation of indexes for the quantitative and objective estimation of grapevine bunch compactness , 2013 .

[56]  Jesús Raúl Martínez-Sandoval,et al.  Implementation of Hough transform for fruit image segmentation , 2012 .

[57]  Vincent Casser,et al.  Using Feedforward Neural Networks for Color Based Grape Detection in Field Images , 2016 .

[58]  Jon Svensson,et al.  Assessment of Grapevine Vigour Using Image Processing , 2002 .

[59]  Michael A. McCarthy,et al.  Dynamics of grape berry growth and physiology of ripening. , 2000 .

[60]  Eric Duchêne,et al.  A semi-automatic non-destructive method to quantify grapevine downy mildew sporulation. , 2011, Journal of microbiological methods.

[61]  Javier Tardáguila,et al.  A new methodology for estimating the grapevine-berry number per cluster using image analysis , 2017 .

[62]  D. Lamb,et al.  Optical remote sensing applications in viticulture - a review , 2002 .

[63]  Gonzalo Pajares,et al.  New Trends in Robotics for Agriculture: Integration and Assessment of a Real Fleet of Robots , 2014, TheScientificWorldJournal.

[64]  Vijay S. Rajpurohit,et al.  “Diagnosis and classification of grape leaf diseases using neural networks” , 2013, 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT).

[65]  Manuel Melgosa,et al.  Ripeness estimation of grape berries and seeds by image analysis , 2012 .

[66]  R. Ferrer-Gallego,et al.  Preliminary study to determine the phenolic maturity stage of grape seeds by computer vision. , 2012, Analytica chimica acta.

[67]  Salviano F. S. P. Soares,et al.  Automatic detection of bunches of grapes in natural environment from color images , 2012, J. Appl. Log..