Detecting Apples in the Wild: Potential for Harvest Quantity Estimation

Knowing the exact number of fruits and trees helps farmers to make better decisions in their orchard production management. The current practice of crop estimation practice often involves manual counting of fruits (before harvesting), which is an extremely time-consuming and costly process. Additionally, this is not practicable for large orchards. Thanks to the changes that have taken place in recent years in the field of image analysis methods and computational performance, it is possible to create solutions for automatic fruit counting based on registered digital images. The pilot study aims to confirm the state of knowledge in the use of three methods (You Only Look Once—YOLO, Viola–Jones—a method based on the synergy of morphological operations of digital imagesand Hough transformation) of image recognition for apple detecting and counting. The study compared the results of three image analysis methods that can be used for counting apple fruits. They were validated, and their results allowed the recommendation of a method based on the YOLO algorithm for the proposed solution. It was based on the use of mass accessible devices (smartphones equipped with a camera with the required accuracy of image acquisition and accurate Global Navigation Satellite System (GNSS) positioning) for orchard owners to count growing apples. In our pilot study, three methods of counting apples were tested to create an automatic system for estimating apple yields in orchards. The test orchard is located at the University of Warmia and Mazury in Olsztyn. The tests were carried out on four trees located in different parts of the orchard. For the tests used, the dataset contained 1102 apple images and 3800 background images without fruits.

[1]  Maryam Rahnemoonfar,et al.  Deep Count: Fruit Counting Based on Deep Simulated Learning , 2017, Sensors.

[2]  Vijay Kumar,et al.  Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Malrey Lee,et al.  An yield estimation in citrus orchards via fruit detection and counting using image processing , 2017, Comput. Electron. Agric..

[4]  Anuja Bhargava,et al.  Fruits and vegetables quality evaluation using computer vision: A review , 2021, J. King Saud Univ. Comput. Inf. Sci..

[5]  Davide Marenchino,et al.  Performance Analysis of the SIFT Operator for Automatic Feature Extraction and Matching in Photogrammetric Applications , 2009, Sensors.

[6]  M. Renigier‐Biłozor,et al.  Rating attributes toolkit for the residential property market , 2017 .

[7]  Yuanshen Zhao,et al.  Robust Tomato Recognition for Robotic Harvesting Using Feature Images Fusion , 2016, Sensors.

[8]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[9]  Volkan Isler,et al.  Apple Counting using Convolutional Neural Networks , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Xinkai Wu,et al.  Pedestrian Detection and Tracking from Low-Resolution Unmanned Aerial Vehicle Thermal Imagery , 2016, Sensors.

[11]  Volkan Isler,et al.  A comparative study of fruit detection and counting methods for yield mapping in apple orchards , 2018, J. Field Robotics.

[12]  Cristina E. Davis,et al.  Advanced methods of plant disease detection. A review , 2014, Agronomy for Sustainable Development.

[13]  Nikrooz Bagheri,et al.  Detection of Fire Blight disease in pear trees by hyperspectral data , 2018 .

[14]  Hui Wang,et al.  Differentiation Between Organic and Non-Organic Apples Using Diffraction Grating and Image Processing—A Cost-Effective Approach , 2018, Sensors.

[15]  José Carlos Barbosa,et al.  Automatic green fruit counting in orange trees using digital images , 2016, Comput. Electron. Agric..

[16]  J. Amiriparian,et al.  Apple Fruits Recognition Under Natural Luminance Using Machine Vision , 2010 .

[17]  Zhao Dean,et al.  Recognition of apple fruit in natural environment , 2016 .

[18]  Manjunath V. Joshi,et al.  Fruit Detection using Improved Multiple Features based Algorithm , 2011 .

[19]  Sridha Sridharan,et al.  Crowd Counting Using Multiple Local Features , 2009, 2009 Digital Image Computing: Techniques and Applications.

[20]  Katarzyna Bobkowska,et al.  3D MODELLING OF CYLINDRICAL-SHAPED OBJECTS FROM LIDAR DATA – AN ASSESSMENT BASED ON THEORETICAL MODELLING AND EXPERIMENTAL DATA , 2018 .

[21]  Artur Janowski,et al.  WEB-CAM AS A MEANS OF INFORMATION ABOUT EMOTIONAL ATTEMPT OF STUDENTS IN THE PROCESS OF DISTANT LEARNING , 2014 .

[22]  Long He,et al.  Field Evaluation of Targeted Shake-and-Catch Harvesting Technologies for Fresh Market Apple , 2020 .

[23]  Raphael Linker,et al.  Vision-based localisation of mature apples in tree images using convexity , 2014 .

[24]  José Blasco,et al.  Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm , 2007 .

[25]  Seishi Ninomiya,et al.  On Plant Detection of Intact Tomato Fruits Using Image Analysis and Machine Learning Methods , 2014, Sensors.

[26]  Xin Zhang,et al.  Faster R–CNN–based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting , 2020 .

[27]  Carlos Eugenio Oliveros,et al.  Automatic fruit count on coffee branches using computer vision , 2017, Comput. Electron. Agric..

[28]  Paolo Valigi,et al.  Weakly Supervised Fruit Counting for Yield Estimation Using Spatial Consistency , 2019, IEEE Robotics and Automation Letters.

[29]  Krystyna Nagrodzka-Godycka.  STATE-OF-THE-ART FRAMEWORK FOR HIGH-SPEED CAMERA AND PHOTOGRAMMETRIC USE IN GEOMETRY EVALUATION OF PRESTRESSED CONCRETE FAILURE PROCESS , 2016 .

[30]  José María Martínez Sanchez,et al.  Abandoned Object Detection in Video-Surveillance: Survey and Comparison , 2018, Sensors.

[31]  Andrea S. Laliberte,et al.  Automated wildlife counts from remotely sensed imagery , 2003 .

[32]  A. Vijayalakshmi,et al.  Comparison of Viola-Jones And Kanade-Lucas-Tomasi Face Detection Algorithms , 2017 .

[33]  Fernando Auat Cheein,et al.  Comparison of convolutional neural networks in fruit detection and counting: A comprehensive evaluation , 2020, Comput. Electron. Agric..

[34]  Xinkai Wu,et al.  A Hybrid Vehicle Detection Method Based on Viola-Jones and HOG + SVM from UAV Images , 2016, Sensors.

[35]  E. O. Gunes,et al.  Detection of the pomegranate fruits on tree using image processing , 2012, 2012 First International Conference on Agro- Geoinformatics (Agro-Geoinformatics).

[36]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Maciej Niedostatkiewicz,et al.  Machine Learning Techniques in Concrete Mix Design , 2019, Materials.

[38]  Paul Richard,et al.  Toward Joint Acquisition-Annotation of Images with Egocentric Devices for a Lower-Cost Machine Learning Application to Apple Detection , 2020, Sensors.

[39]  S. Źróbek,et al.  The concept of studying the impact of legal changes on the agricultural real estate market , 2019, Land Use Policy.

[40]  Yihao Zhang,et al.  Endozoochory by granivorous rodents in seed dispersal of green fruits , 2019, Canadian Journal of Zoology.

[41]  Yiliang Zeng,et al.  Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot , 2014 .

[42]  Dennis Jarvis,et al.  Estimating mango crop yield using image analysis using fruit at 'stone hardening' stage and night time imaging , 2014 .

[43]  Raphael Linker,et al.  Determination of the number of green apples in RGB images recorded in orchards , 2012 .

[44]  Lav R. Khot,et al.  Hyperspectral Imaging and Spectrometry-Derived Spectral Features for Bitter Pit Detection in Storage Apples , 2018, Sensors.

[45]  Jakub Szulwic.  PHOTOGRAMMETRIC DEVELOPMENT OF THE THRESHOLD WATER AT THE DAM ON THE VISTULA RIVER IN WLOCLAWEK FROM UNMANNED AERIAL VEHICLES (UAV) , 2011 .