Comparison of grape flower counting using patch-based instance segmentation and density-based estimation with convolutional neural networks

Information on flower number per grapevine inflorescence is critical for grapevine genetic improvement, early yield estimation and vineyard management. Previous approaches to automize this process by traditional image processing techniques, are unsuccessful in the improvement of a universal system that can be applied to inflorescences with diverse morphology. In this paper, we validated the efficiency of advanced deep learning-based approaches for automatic flower number estimation. The results were analyzed on an inflorescence dataset of 204 images from four different cultivars during various growth stages. The algorithm developed on patch-based instance segmentation using Mask R-CNN, produced counting results highly correlated to manual counts (R2 = 0.96). Practically constant MAPE values among different cultivars (from 5.50% to 8.45%), implying a high robustness in this method. Achieving the fastest counting (0.33 second per image of size 512 × 512) with slightly lower counting accuracy (R2 = 0.91), the method based on object density-map using U-Net turned out to be suitable for real-time flower counting systems.

[1]  Reinhard Töpfer,et al.  Breeding the next-generation disease-resistant grapevine varieties , 2016 .

[2]  Volker Steinhage,et al.  Efficient identification, localization and quantification of grapevine inflorescences and flowers in unprepared field images using Fully Convolutional Networks , 2019 .

[3]  Lucien Hoffmann,et al.  Impact of Grape Cluster Division on Cluster Morphology and Bunch Rot Epidemic , 2012, American Journal of Enology and Viticulture.

[4]  Antoni B. Chan,et al.  Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Counting, Detection, and Tracking , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Eric Lebon,et al.  Adapting plant material to face water stress in vineyards: which physiological targets for an optimal control of plant water status? , 2017 .

[6]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[7]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[8]  Javier Ibáñez,et al.  What do we know about grapevine bunch compactness? A state‐of‐the‐art review , 2018 .

[9]  Derdour Khedidja,et al.  Estimation of the flower buttons per inflorescences of grapevine (Vitis vinifera L.) by image auto-assessment processing , 2016 .

[10]  Todd M. Schmit,et al.  Cluster Thinning Reduces the Economic Sustainability of Riesling Production , 2013, American Journal of Enology and Viticulture.

[11]  Thiago T. Santos,et al.  Grape detection, segmentation and tracking using deep neural networks and three-dimensional association , 2019, Comput. Electron. Agric..

[12]  Jesús Salido,et al.  Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions , 2020, Comput. Electron. Agric..

[13]  José Blasco,et al.  Assessment of flower number per inflorescence in grapevine by image analysis under field conditions. , 2014, Journal of the science of food and agriculture.

[14]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Shenghua Gao,et al.  Single-Image Crowd Counting via Multi-Column Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[17]  Lutz Plümer,et al.  Low-Cost 3D Systems: Suitable Tools for Plant Phenotyping , 2014, Sensors.

[18]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Stefano Poni,et al.  Effects of Early Defoliation on Shoot Photosynthesis, Yield Components, and Grape Composition , 2006, American Journal of Enology and Viticulture.

[21]  Paul R. Petrie,et al.  A robust automated flower estimation system for grape vines , 2018, Biosystems Engineering.

[22]  Rubén Linares,et al.  Comparison of different methods of grapevine yield prediction in the time window between fruitset and veraison , 2015 .

[23]  María-Paz Diago,et al.  vitisFlower®: Development and Testing of a Novel Android-Smartphone Application for Assessing the Number of Grapevine Flowers per Inflorescence Using Artificial Vision Techniques , 2015, Sensors.

[24]  B. G. Coombe,et al.  Growth Stages of the Grapevine: Adoption of a system for identifying grapevine growth stages , 1995 .

[25]  Mardi Longbottom,et al.  Classification of reproductive performance of ten winegrape varieties , 2010 .

[26]  A. Aquino,et al.  Image analysis-based modelling for flower number estimation in grapevine. , 2017, Journal of the science of food and agriculture.

[27]  Florian Rist,et al.  Automatic Flower Number Evaluation in Grapevine Inflorescences Using RGB Images , 2019, American Journal of Enology and Viticulture.

[28]  B. Holzapfel,et al.  Leaf area to fruit weight ratios for maximising grape berry weight, sugar concentration and anthocyanin content during ripening , 2016 .

[29]  Borja Millán,et al.  Grapevine flower estimation by applying artificial vision techniques on images with uncontrolled scene and multi-model analysis , 2015, Comput. Electron. Agric..