Apple Detection in Natural Environment Using Deep Learning Algorithms

It is a challenging problem to detect the apple in natural environment using traditional object recognition algorithms due to occlusion, fluctuating illumination and complex backgrounds. Deep learning methods for object detection make impressive progress, which can automatically extract the number, pixel position, size and other features of apples from the images. In this paper, four deep learning recognition models, Faster RCNN based on AlexNet, Faster RCNN based on ResNet101, YOLOv3 based on DarkNet53 and improved YOLOv3 were employed to carry out recognition experiments on red and green apple under three illumination and two image sharpness conditions, with the transfer learning to accelerate the training process. The results showed that improved YOLOv3 model had the best recognition effect among the four detection models. F1 value of red apple recognition was 95.0%, 94.6% and 94.1% for normal, insufficient and excessive illumination, respectively, and F1 value of green apple recognition was 94.9%, 94.0% and 91.1%. There were F1 value of 92.8% and 92.1% for red and green apple recognition in blurred images, respectively. Moreover, improved YOLOv3 algorithm still had the better performance for occlusion, spot, overlap and incomplete apples, with a recognition recall rate higher than 88.5%. It can be concluded that improved YOLOv3 algorithm can provide a more efficient way for apple detection in natural environment.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  Bruce MacDonald,et al.  Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms , 2019, Biosystems Engineering.

[3]  Dongjian He,et al.  Recognition of green apples in an orchard environment by combining the GrabCut model and Ncut algorithm , 2019, Biosystems Engineering.

[4]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[5]  Dongjian He,et al.  Recognition of green apples based on fuzzy set theory and manifold ranking algorithm , 2018 .

[6]  Rui Li,et al.  Kiwifruit detection in field images using Faster R-CNN with VGG16 , 2019, IFAC-PapersOnLine.

[7]  Wenming Cao,et al.  Review of Pavement Defect Detection Methods , 2020, IEEE Access.

[8]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[9]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Wenqian Huang,et al.  Nondestructive measurement of soluble solids content in apple using near infrared hyperspectral imaging coupled with wavelength selection algorithm , 2019, Infrared Physics & Technology.

[11]  Avinash C. Tyagi,et al.  Towards a Second Green Revolution , 2016 .

[12]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[14]  Yuanyuan Shao,et al.  Research on Multi-class Fruits Recognition Based on Machine Vision and SVM , 2018 .

[15]  Changki Mo,et al.  Preliminary Design of a Robotic System for Catching and Storing Fresh Market Apples , 2016 .

[16]  Zhao Dean,et al.  System Design and Control of an Apple Harvesting Robot , 2020, ArXiv.

[17]  Henry Medeiros,et al.  Apple flower detection using deep convolutional networks , 2018, Comput. Ind..

[18]  Ying Zhang,et al.  Automatic recognition vision system guided for apple harvesting robot , 2012, Comput. Electr. Eng..

[19]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[20]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[21]  Zhao Dean,et al.  Recognition of apple fruit in natural environment , 2016 .

[22]  Tao Wang,et al.  L1-Norm Minimization for Multi-dimensional Signals Based on Geometric Algebra , 2019, Advances in Applied Clifford Algebras.

[23]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[24]  Rui Wang,et al.  GA-SURF: A new Speeded-Up robust feature extraction algorithm for multispectral images based on geometric algebra , 2019, Pattern Recognit. Lett..

[25]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  W. S. Lee,et al.  Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions , 2014 .