Vision System for Automatic On-Tree Kiwifruit Counting and Yield Estimation

Yield estimation is an essential preharvest practice among most large-scale farming companies, since it enables the predetermination of essential logistics to be allocated (i.e., transportation means, supplies, labor force, among others). An overestimation may thus incur further costs, whereas an underestimation entails potential crop waste. More interestingly, an accurate yield estimation enables stakeholders to better place themselves in the market. Yet, computer-aided precision farming is set to play a pivotal role in this respect. Kiwifruit represents a major produce in several countries (e.g., Italy, China, New and Zealand). However, up to date, the relevant literature remains short of a complete as well as automatic system for kiwifruit yield estimation. In this paper, we present a fully automatic and noninvasive computer vision system for kiwifruit yield estimation across a given orchard. It consists mainly of an optical sensor mounted on a minitractor that surveys the orchard of interest at a low pace. Afterwards, the acquired images are fed to a pipeline that incorporates image preprocessing, stitching, and fruit counting stages and outputs an estimated fruit count and yield estimation. Experimental results conducted on two large kiwifruit orchards confirm a high plausibility (i.e., errors of 6% and 15%) of the proposed system. The proposed yield estimation solution has been in commercial use for about 2 years. With respect to the traditional manual yield estimation carried out by kiwifruit companies, it was demonstrated to save a significant amount of time and cut down on estimation errors, especially when speaking of large-scale farming.

[1]  Divya Garg,et al.  Apple fruit detection and counting using computer vision techniques , 2014, 2014 IEEE International Conference on Computational Intelligence and Computing Research.

[2]  Qi Wang,et al.  Automated Crop Yield Estimation for Apple Orchards , 2012, ISER.

[3]  E. J. Sadler,et al.  Long-term impact of a precision agriculture system on grain crop production , 2017, Precision Agriculture.

[4]  Karim Foughali,et al.  Monitoring system using web of things in precision agriculture , 2017, FNC/MobiSPC.

[5]  Kunlin Zou,et al.  A Method of Apple Image Segmentation Based on Color-Texture Fusion Feature and Machine Learning , 2020, Agronomy.

[6]  Salah Sukkarieh,et al.  Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review , 2018, Comput. Electron. Agric..

[7]  L. Deng,et al.  UAV-based multispectral remote sensing for precision agriculture: A comparison between different cameras , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[8]  Xinkai Wu,et al.  A Hybrid Vehicle Detection Method Based on Viola-Jones and HOG + SVM from UAV Images , 2016, Sensors.

[9]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Fernando Auat Cheein,et al.  Comparison of convolutional neural networks in fruit detection and counting: A comprehensive evaluation , 2020, Comput. Electron. Agric..

[11]  Xinkai Wu,et al.  An Enhanced Viola-Jones Vehicle Detection Method From Unmanned Aerial Vehicles Imagery , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[13]  Arun M. Patokar,et al.  Precision Agriculture System Design Using Wireless Sensor Network , 2018 .

[14]  Hong Cheng,et al.  Early Yield Prediction Using Image Analysis of Apple Fruit and Tree Canopy Features with Neural Networks , 2017, J. Imaging.

[15]  Malrey Lee,et al.  An yield estimation in citrus orchards via fruit detection and counting using image processing , 2017, Comput. Electron. Agric..

[16]  C. Glasbey,et al.  Automatic fruit recognition and counting from multiple images , 2014 .

[17]  Mathias Kölsch,et al.  Analysis of rotational robustness of hand detection with a Viola-Jones detector , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[18]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[19]  James Patrick Underwood,et al.  Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards , 2016, J. Field Robotics.

[20]  Won Suk Lee,et al.  Color Vision System for Estimating Citrus Yield in Real-time , 2004 .

[21]  Panagiotis G. Sarigiannidis,et al.  A Review on UAV-Based Applications for Precision Agriculture , 2019, Inf..

[22]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[23]  Lorenzo Comba,et al.  Unsupervised detection of vineyards by 3D point-cloud UAV photogrammetry for precision agriculture , 2018, Comput. Electron. Agric..

[24]  Ελπινίκη Ι. Παπαγεωργίου,et al.  Yield prediction in apples using Fuzzy Cognitive Map learning approach , 2015 .

[25]  Chris McCarthy,et al.  Deep learning - Method overview and review of use for fruit detection and yield estimation , 2019, Comput. Electron. Agric..

[26]  Sheikh Ziauddin,et al.  Detection and Counting of On-Tree Citrus Fruit for Crop Yield Estimation , 2016 .

[27]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[28]  Salah Sukkarieh,et al.  A Feature Learning Based Approach for Automated Fruit Yield Estimation , 2013, FSR.

[29]  Per-Erik Forssén,et al.  Maximally Stable Colour Regions for Recognition and Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  W. Maes,et al.  Perspectives for Remote Sensing with Unmanned Aerial Vehicles in Precision Agriculture. , 2019, Trends in plant science.

[31]  Bernard Gosselin,et al.  Artificial neural network-based segmentation and apple grading by machine vision , 2005, IEEE International Conference on Image Processing 2005.

[32]  Maryam Rahnemoonfar,et al.  Deep Count: Fruit Counting Based on Deep Simulated Learning , 2017, Sensors.

[33]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[34]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[35]  R. Linker Machine learning based analysis of night-time images for yield prediction in apple orchard , 2018 .

[36]  Won Suk Lee,et al.  Strawberry Yield Prediction Based on a Deep Neural Network Using High-Resolution Aerial Orthoimages , 2019, Remote. Sens..