A Cloud-Based Environment for Generating Yield Estimation Maps From Apple Orchards Using UAV Imagery and a Deep Learning Technique

Farmers require accurate yield estimates, since they are key to predicting the volume of stock needed at supermarkets and to organizing harvesting operations. In many cases, the yield is visually estimated by the crop producer, but this approach is not accurate or time efficient. This study presents a rapid sensing and yield estimation scheme using off-the-shelf aerial imagery and deep learning. A Region-Convolutional Neural Network was trained to detect and count the number of apple fruit on individual trees located on the orthomosaic built from images taken by the unmanned aerial vehicle (UAV). The results obtained with the proposed approach were compared with apple counts made in situ by an agrotechnician, and an R2 value of 0.86 was acquired (MAE: 10.35 and RMSE: 13.56). As only parts of the tree fruits were visible in the top-view images, linear regression was used to estimate the number of total apples on each tree. An R2 value of 0.80 (MAE: 128.56 and RMSE: 130.56) was obtained. With the number of fruits detected and tree coordinates two shapefile using Python script in Google Colab were generated. With the previous information two yield maps were displayed: one with information per tree and another with information per tree row. We are confident that these results will help to maximize the crop producers' outputs via optimized orchard management.

[1]  Manoj Karkee,et al.  A HIERARCHICAL APPROACH TO APPLE IDENTIFICATION FOR ROBOTIC HARVESTING , 2016 .

[2]  Kamel Barkaoui,et al.  Performability evaluation of server virtualized systems under bursty workload , 2018 .

[3]  S. Musacchi,et al.  Optimizing Crop Load for New Apple Cultivar: “WA38” , 2019, Agronomy.

[4]  Brian O'Connor,et al.  Analysis of Sentinel-2 and RapidEye for Retrieval of Leaf Area Index in a Saltmarsh Using a Radiative Transfer Model , 2019, Remote. Sens..

[5]  Nancy Alonistioti,et al.  Farm management systems and the Future Internet era , 2012 .

[6]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[7]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[8]  En Li,et al.  Detection of Apple Lesions in Orchards Based on Deep Learning Methods of CycleGAN and YOLOV3-Dense , 2019, J. Sensors.

[9]  S LewMichael,et al.  Deep learning for visual understanding , 2016 .

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Arko Lucieer,et al.  An Automated Technique for Generating Georectified Mosaics from Ultra-High Resolution Unmanned Aerial Vehicle (UAV) Imagery, Based on Structure from Motion (SfM) Point Clouds , 2012, Remote. Sens..

[12]  Param S. Rajpura,et al.  Transfer Learning for Object Detection using State-of-the-Art Deep Neural Networks , 2018, 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN).

[13]  Ana Cláudia Teodoro,et al.  Open-source GIS application for UAV photogrammetry based on MicMac , 2017 .

[14]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[15]  Jacek Rapinski,et al.  JupyTEP IDE as an Online Tool for Earth Observation Data Processing , 2019, Remote. Sens..

[16]  Siddhartha Kumar Khaitan,et al.  Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection , 2017 .

[17]  Thomas Udelhoven,et al.  CropGIS - A web application for the spatial and temporal visualization of past, present and future crop biomass development , 2019, Comput. Electron. Agric..

[18]  James Patrick Underwood,et al.  Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry , 2016, Sensors.

[19]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[20]  Um Rao Mogili,et al.  Review on Application of Drone Systems in Precision Agriculture , 2018 .

[21]  Aman Jantan,et al.  State-of-the-art in artificial neural network applications: A survey , 2018, Heliyon.

[22]  M. Oliver Precision agriculture and geostatistics: How to manage agriculture more exactly , 2013 .

[23]  Shaohua Wang,et al.  A Robust Photogrammetric Processing Method of Low-Altitude UAV Images , 2015, Remote. Sens..

[24]  Louise Willemen,et al.  Machine Learning Using Hyperspectral Data Inaccurately Predicts Plant Traits Under Spatial Dependency , 2018, Remote. Sens..

[25]  Nguyen Duy Hung,et al.  Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV) , 2019, PloS one.

[26]  Jorge Torres-Sánchez,et al.  Mapping the 3D structure of almond trees using UAV acquired photogrammetric point clouds and object-based image analysis , 2018, Biosystems Engineering.

[27]  Theofanis Gemtos,et al.  Delineation of management zones in an apple orchard in Greece using a multivariate approach , 2013 .

[28]  En Li,et al.  Apple detection during different growth stages in orchards using the improved YOLO-V3 model , 2019, Comput. Electron. Agric..

[29]  Patrick J. Fox,et al.  Shake Table Test of Large-Scale Bridge Columns Supported on Rocking Shallow Foundations , 2015 .

[30]  K. Walsh,et al.  Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’ , 2019, Precision Agriculture.

[31]  Siva Kumar Balasundram,et al.  Research and development in agricultural robotics: a perspective of digital farming. , 2018 .

[32]  Filiberto Pla,et al.  Vision systems for the location of citrus fruit in a tree canopy , 1992 .

[33]  Salah Sukkarieh,et al.  Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review , 2018, Comput. Electron. Agric..

[34]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[35]  J. Pérez‐Sánchez,et al.  European Sea Bass (Dicentrarchus labrax) Immune Status and Disease Resistance Are Impaired by Arginine Dietary Supplementation , 2015, PloS one.

[36]  Gui-Bin Bian,et al.  Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications , 2018, IEEE Access.

[37]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  W. Maes,et al.  Perspectives for Remote Sensing with Unmanned Aerial Vehicles in Precision Agriculture. , 2019, Trends in plant science.

[39]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[40]  anonymous,et al.  Comprehensive review , 2019 .

[41]  Hervé Sinoquet,et al.  Light interception in apple trees influenced by canopy architecture manipulation , 2004, Trees.

[42]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[43]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[44]  E. Lötze,et al.  Early prediction of harvest fruit size distribution of an apple and pear cultivar , 2004 .

[45]  Rafael Rieder,et al.  Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review , 2018, Comput. Electron. Agric..

[46]  Manoj Karkee,et al.  Apple fruit size estimation using a 3D machine vision system , 2018, Information Processing in Agriculture.

[47]  Mohamed A. E. AbdelRahman,et al.  Assessment of land suitability and capability by integrating remote sensing and GIS for agriculture in Chamarajanagar district, Karnataka, India , 2016 .

[48]  Won Suk Lee,et al.  Strawberry Yield Prediction Based on a Deep Neural Network Using High-Resolution Aerial Orthoimages , 2019, Remote. Sens..

[49]  Volkan Isler,et al.  Vision-Based Preharvest Yield Mapping for Apple Orchards , 2018, Comput. Electron. Agric..

[50]  Jose L Pons,et al.  A SURVEY OF COMPUTER VISION METHODS FOR LOCATING FRUIT ON TREES , 2000 .

[51]  Henry Medeiros,et al.  Apple flower detection using deep convolutional networks , 2018, Comput. Ind..

[52]  Andreas Kamilaris,et al.  A review on the practice of big data analysis in agriculture , 2017, Comput. Electron. Agric..

[53]  Jorge Torres-Sánchez,et al.  High-Throughput 3-D Monitoring of Agricultural-Tree Plantations with Unmanned Aerial Vehicle (UAV) Technology , 2015, PloS one.

[54]  Marcela C. González-Araya,et al.  Harvest Planning in Apple Orchards Using an Optimization Model , 2015 .

[55]  Shu Gan,et al.  Corrigendum to "Application of Fractional Differential Calculation in Pretreatment of Saline Soil Hyperspectral Reflectance Data" , 2019, J. Sensors.

[56]  Maryam Rahnemoonfar,et al.  Deep Count: Fruit Counting Based on Deep Simulated Learning , 2017, Sensors.

[57]  J. Senthilnath,et al.  Detection of tomatoes using spectral-spatial methods in remotely sensed RGB images captured by UAV , 2016 .

[58]  Angappa Gunasekaran,et al.  Big GIS analytics framework for agriculture supply chains: A literature review identifying the current trends and future perspectives , 2018, Comput. Electron. Agric..

[59]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[60]  Matthew F. McCabe,et al.  Using Multi-Spectral UAV Imagery to Extract Tree Crop Structural Properties and Assess Pruning Effects , 2018, Remote. Sens..

[61]  Saeid Nahavandi,et al.  A Classifier Graph Based Recurring Concept Detection and Prediction Approach , 2018, Comput. Intell. Neurosci..

[62]  Q. Zhang,et al.  Sensors and systems for fruit detection and localization: A review , 2015, Comput. Electron. Agric..

[63]  Aalap Doshi,et al.  A comprehensive review on automation in agriculture using artificial intelligence , 2019, Artificial Intelligence in Agriculture.

[64]  James Patrick Underwood,et al.  Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards , 2016, J. Field Robotics.

[65]  Dennis Jarvis,et al.  Estimating mango crop yield using image analysis using fruit at 'stone hardening' stage and night time imaging , 2014 .

[66]  Rui Li,et al.  Kiwifruit detection in field images using Faster R-CNN with VGG16 , 2019, IFAC-PapersOnLine.

[67]  David A. Widmar,et al.  FARMER PERCEPTIONS OF PRECISION AGRICULTURE TECHNOLOGY BENEFITS , 2018, Journal of Agricultural and Applied Economics.

[68]  Ekaba Bisong,et al.  Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners , 2019 .

[69]  Christopher O. Jaynes Computer vision and artificial intelligence , 1996, CROS.

[70]  S. Keesstra,et al.  Comparing Filtering Techniques for Removing Vegetation from UAV-Based Photogrammetric Point Clouds , 2019, Drones.