Semantic Segmentation of Vineyard Images Using Convolutional Neural Networks

This paper aims to study the segmentation demands of vineyard images using Convolutional Neural Networks (CNNs). To this end, eleven CNN models able to provide semantic segmented images are examined as part of the sensing subsystem of an autonomous agricultural robot. The task is challenging due to the similar color between grapes, leaves and image’s background. Moreover, the lack of controlled lighting conditions results in varying color representation of grapes and leaves. The studied CNN model architectures combine three different feature learning sub-networks, with five meta-architectures for segmentation purposes. Investigation on three different datasets consisting of vineyard images of grape clusters and leaves, provided segmentation results, by mean pixel intersection over union (IU) performance index, of up to 87.89% for grape clusters and 83.45% for leaves, for the case of ResNet50_FRRN and MobileNetV2_PSPNet model, respectively. Comparative results reveal the efficacy of CNNs to separate grape clusters and leaves from image’s background. Thus, the proposed models can be used for in-field applications for real-time localization of grapes and leaves, towards automation of harvest, green harvest and defoliation agricultural activities by an autonomous robot.

[1]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[5]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Na Li,et al.  A NOVEL DEEP CONVOLUTIONAL NEURAL NETWORK FOR SPECTRAL–SPATIAL CLASSIFICATION OF HYPERSPECTRAL DATA , 2018 .

[8]  Volker Steinhage,et al.  Efficient identification, localization and quantification of grapevine inflorescences and flowers in unprepared field images using Fully Convolutional Networks , 2019 .

[9]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[10]  Tony E Grift,et al.  Variable field-of-view machine vision based row guidance of an agricultural robot , 2012 .

[11]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Eleni Vrochidou,et al.  Grapes Visual Segmentation for Harvesting Robots Using Local Texture Descriptors , 2019, ICVS.

[14]  Michał Grochowski,et al.  Data augmentation for improving deep learning in image classification problem , 2018, 2018 International Interdisciplinary PhD Workshop (IIPhDW).

[15]  H. T. Søgaard,et al.  Application Accuracy of a Machine Vision-controlled Robotic Micro-dosing System , 2007 .

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Martin Jägersand,et al.  RTSeg: Real-Time Semantic Segmentation Comparative Study , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[18]  Xin Du,et al.  Integrating the Continuous Wavelet Transform and a Convolutional Neural Network to Identify Vineyard Using Time Series Satellite Images , 2019, Remote. Sens..

[19]  Pranali Kosamkar,et al.  Grapes Ripeness Estimation using Convolutional Neural network and Support Vector Machine , 2019, 2019 Global Conference for Advancement in Technology (GCAT).

[20]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[21]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Tanya Monga.  Estimating Vineyard Grape Yield from Images , 2018, Canadian Conference on AI.

[23]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[24]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[25]  Eleni Vrochidou,et al.  Machine Vision Systems in Precision Agriculture for Crop Farming , 2019, J. Imaging.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[28]  Qian Du,et al.  Integrating spectral and spatial information into deep convolutional Neural Networks for hyperspectral classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).