Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments

Abstract Achieving the rapid and accurate detection of apple flowers in natural environments is essential for yield estimation and the development of an automatic flower thinner. A real-time apple flower detection method using the channel pruned YOLO v4 deep learning algorithm was proposed. First, the YOLO v4 model under the CSPDarknet53 framework was built, and then, to simplify the apple flower detection model and ensure the efficiency of the model, the channel pruning algorithm was used to prune the model. Finally, a total of 2230 manually labeled apple flower images (including three varieties of Fuji, Red Love, and Gala) were used to fine-tune the model to achieve the fast and accurate detection of apple flowers. The test results showed that the number of parameters of the apple flower detection model after pruning was reduced by 96.74%, the model size was reduced by 231.51 MB, the inference time was decreased by 39.47%, and the mAP was 97.31%, which was only 0.24% lower than the model before pruning. To verify the effectiveness of the proposed method, five different deep learning algorithms including the Faster R-CNN, Tiny-YOLO v2, YOLO v3, SSD 300 and EfficientDet-D0 were compared. The comparative results showed that the mAP of the apple flower detection using the proposed method was 97.31%; the detection speed was 72.33f/s; the model size was 12.46 MB; the mAP was 12.21%, 15.56%, 14.19%, 5.67% and 7.79% higher than the other five algorithms, respectively; and the detection speed could meet the real-time requirements. Furthermore, the detection performance of apple flowers under different species of apple trees and illumination conditions was discussed. The results indicated that the proposed method had strong robustness to the changes of fruit tree varieties and illumination directions. The results showed that it was feasible to apply the proposed method for the real-time and accurate detection of apple flowers. The research could provide technical references for orchard yield estimation and the development of apple flower thinning robots.

[1]  Simon X. Yang,et al.  Intelligent Automation with Applications to Agriculture , 2016, Intell. Autom. Soft Comput..

[2]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[3]  Nesar Mohammadi Baneh,et al.  Mechatronic components in apple sorting machines with computer vision , 2018, Journal of Food Measurement and Characterization.

[4]  Xin He,et al.  On line detection of defective apples using computer vision system combined with deep learning methods , 2020 .

[5]  Henry Medeiros,et al.  Multispecies Fruit Flower Detection Using a Refined Semantic Segmentation Network , 2018, IEEE Robotics and Automation Letters.

[6]  Hong Chen,et al.  Detection and Recognition of Flower Image Based on SSD network in Video Stream , 2019, Journal of Physics: Conference Series.

[7]  Jun-Wei Hsieh,et al.  CSPNet: A New Backbone that can Enhance Learning Capability of CNN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Jiashi Feng,et al.  PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[9]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Qin Ouyang,et al.  Quantitative detection of apple watercore and soluble solids content by near infrared transmittance spectroscopy , 2020 .

[12]  Gernot A. Fink,et al.  Neuron Pruning for Compressing Deep Networks Using Maxout Architectures , 2017, GCPR.

[13]  Mingfang He,et al.  Detection of Apple Defects Based on the FCM-NPGA and a Multivariate Image Analysis , 2020, IEEE Access.

[14]  Malrey Lee,et al.  A Novel Technique for tangerine Yield Prediction using Flower Detection Algorithm , 2013, Int. J. Pattern Recognit. Artif. Intell..

[15]  André Ferreira,et al.  Face Detection in Thermal Images with YOLOv3 , 2019, ISVC.

[16]  Angelo Mencarelli,et al.  Detection of Tomato Flowers from Greenhouse Images Using Colorspace Transformations , 2019, EPIA.

[17]  Elpiniki I. Papageorgiou,et al.  Development and Evaluation of a Fuzzy Inference System and a Neuro-Fuzzy Inference System for Grading Apple Quality , 2018, Appl. Artif. Intell..

[18]  Quoc V. Le,et al.  EfficientDet: Scalable and Efficient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  C. Honda,et al.  A model for representing the relationships among crop load, timing of thinning, flower bud formation, and fruit weight in apples , 2018, Scientia Horticulturae.

[20]  Pavlo Molchanov,et al.  Importance Estimation for Neural Network Pruning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[22]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[23]  Xiangyu Zhang,et al.  Channel Pruning for Accelerating Very Deep Neural Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Gang Wan,et al.  Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark , 2020, ISPRS Journal of Photogrammetry and Remote Sensing.

[25]  Yael Edan,et al.  Computer vision for fruit harvesting robots - state of the art and challenges ahead , 2012, Int. J. Comput. Vis. Robotics.

[26]  Cem Direkoglu,et al.  Sea Turtle Detection Using Faster R-CNN for Conservation Purpose , 2019 .

[27]  Henry Medeiros,et al.  Apple flower detection using deep convolutional networks , 2018, Comput. Ind..

[28]  H. Link,et al.  Significance of flower and fruit thinning on fruit quality , 2000, Plant Growth Regulation.

[29]  Manan Shah,et al.  Artificial cognition for applications in smart agriculture: A comprehensive review , 2020 .

[30]  Victor Alchanatis,et al.  Detection and counting of flowers on apple trees for better chemical thinning decisions , 2019, Precision Agriculture.

[31]  Yuyu Tian,et al.  Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot , 2020, Comput. Electron. Agric..

[32]  Hongfei Fu,et al.  Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues , 2019, Artificial Intelligence in Agriculture.

[33]  Marcello Chiaberge,et al.  Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application , 2020, IEEE Access.

[34]  H Zhao,et al.  Flower Bud Detection Based on Saliency Map and SURF Feature-Points , 2015 .

[35]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Ping Han,et al.  Recent advances in Raman technology with applications in agriculture, food and biosystems: A review , 2019, Artificial Intelligence in Agriculture.

[37]  Debabrata Samanta,et al.  AUTOMATED DAMAGED FLOWER DETECTION USING IMAGE PROCESSING , 2013 .

[38]  A. Lakso,et al.  Advances in Predicting Chemical Thinner Response of Apple Using a Carbon Balance Model , 2012 .

[39]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[40]  Dongjian He,et al.  Recognition of green apples in an orchard environment by combining the GrabCut model and Ncut algorithm , 2019, Biosystems Engineering.

[41]  F. G. Jr. Dennis,et al.  The history of fruit thinning , 2000, Plant Growth Regulation.