Detection of Single Grapevine Berries in Images Using Fully Convolutional Neural Networks

Yield estimation and forecasting are of special interest in the field of grapevine breeding and viticulture. The number of harvested berries per plant is strongly correlated with the resulting quality. Therefore, early yield forecasting can enable a focused thinning of berries to ensure a high quality end product. Traditionally yield estimation is done by extrapolating from a small sample size and by utilizing historic data. Moreover, it needs to be carried out by skilled experts with much experience in this field. Berry detection in images offers a cheap, fast and non-invasive alternative to the otherwise time-consuming and subjective on-site analysis by experts. We apply fully convolutional neural networks on images acquired with the Phenoliner, a field phenotyping platform. We count single berries in images to avoid the error-prone detection of grapevine clusters. Clusters are often overlapping and can vary a lot in the size which makes the reliable detection of them difficult. We address especially the detection of white grapes directly in the vineyard. The detection of single berries is formulated as a classification task with three classes, namely 'berry', 'edge' and 'background'. A connected component algorithm is applied to determine the number of berries in one image. We compare the automatically counted number of berries with the manually detected berries in 60 images showing Riesling plants in vertical shoot positioned trellis (VSP) and semi minimal pruned hedges (SMPH). We are able to detect berries correctly within the VSP system with an accuracy of 94.0 % and for the SMPH system with 85.6 %.

[1]  Javier Tardáguila,et al.  A new methodology for estimating the grapevine-berry number per cluster using image analysis , 2017 .

[2]  Udo Seiffert,et al.  Phenoliner: A New Field Phenotyping Platform for Grapevine Research , 2017, Sensors.

[3]  Yoshua Bengio,et al.  Count-ception: Counting by Fully Convolutional Redundant Counting , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[5]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Sanjiv Singh,et al.  Automated Visual Yield Estimation in Vineyards , 2014, J. Field Robotics.

[7]  María-Paz Diago,et al.  Automated early yield prediction in vineyards from on-the-go image acquisition , 2018, Comput. Electron. Agric..

[8]  Volker Steinhage,et al.  Efficient identification, localization and quantification of grapevine inflorescences in unprepared field images using Fully Convolutional Networks , 2018, ArXiv.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[11]  Andrew Zisserman,et al.  Counting in the Wild , 2016, ECCV.

[12]  Andrew Zisserman,et al.  Microscopy cell counting with fully convolutional regression networks , 2015 .

[13]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Ivan Vidovic,et al.  A nearest neighbor approach for fruit recognition in RGB-D images based on detection of convex surfaces , 2018, Expert Syst. Appl..

[15]  Andrew Zisserman,et al.  Learning To Count Objects in Images , 2010, NIPS.

[16]  Ribana Roscher,et al.  Automated Image Analysis Framework for the High-Throughput Determination of Grapevine Berry Sizes Using Conditional Random Fields , 2017, ArXiv.

[17]  Andrew Zisserman,et al.  Class-Agnostic Counting , 2018, ACCV.

[18]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.