UAV-Based Computer Vision System for Orchard Apple Tree Detection and Health Assessment

Accurate and efficient orchard tree inventories are essential for acquiring up-to-date information, which is necessary for effective treatments and crop insurance purposes. Surveying orchard trees, including tasks such as counting, locating, and assessing health status, plays a vital role in predicting production volumes and facilitating orchard management. However, traditional manual inventories are known to be labor-intensive, expensive, and prone to errors. Motivated by recent advancements in UAV imagery and computer vision methods, we propose a UAV-based computer vision framework for individual tree detection and health assessment. Our proposed approach follows a two-stage process. Firstly, we propose a tree detection model by employing a hard negative mining strategy using RGB UAV images. Subsequently, we address the health classification problem by leveraging multi-band imagery-derived vegetation indices. The proposed framework achieves an F1-score of 86.24% for tree detection and an overall accuracy of 97.52% for tree health assessment. Our study demonstrates the robustness of the proposed framework in accurately assessing orchard tree health from UAV images. Moreover, the proposed approach holds potential for application in various other plantation settings, enabling plant detection and health assessment using UAV imagery.

[1]  J. Valente,et al.  Mapping the spatial variability of Botrytis bunch rot risk in vineyards using UAV multispectral imagery , 2023, European Journal of Agronomy.

[2]  S. Puliti,et al.  Automatic detection of snow breakage at single tree level using YOLOv5 applied to UAV imagery , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[3]  B. Leblon,et al.  COMPUTER VISION SYSTEM FOR DETECTING ORCHARD TREES FROM UAV IMAGES , 2022, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[4]  G. Petropoulos,et al.  Tree's detection & health's assessment from ultra-high resolution UAV imagery and deep learning , 2022, Geocarto International.

[5]  Lei Zhang,et al.  Object Detection in High-Resolution Remote Sensing Images Based on a Hard-Example-Mining Network , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Sarah J. Graves,et al.  A remote sensing derived data set of 100 million individual tree crowns for the National Ecological Observatory Network , 2021, eLife.

[7]  Junho Yeom,et al.  Comparison of Canopy Shape and Vegetation Indices of Citrus Trees Derived from UAV Multispectral Images for Characterization of Citrus Greening Disease , 2020, Remote. Sens..

[8]  Ning Zhang,et al.  Extraction of tree crowns damaged by Dendrolimus tabulaeformis Tsai et Liu via spectral-spatial classification using UAV-based hyperspectral images , 2020, Plant methods.

[9]  B. Leblon,et al.  DETECTING AND COUNTING ORCHARD TREES ON UNMANNED AERIAL VEHICLE (UAV)-BASED IMAGES USING ENTROPY AND NDVI FEATURES , 2020 .

[10]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[11]  Marian-Daniel Iordache,et al.  A Machine Learning Approach to Detecting Pine Wilt Disease Using Airborne Spectral Imagery , 2020, Remote. Sens..

[12]  Shanxin Guo,et al.  Identification of Short-Rotation Eucalyptus Plantation at Large Scale Using Multi-Satellite Imageries and Cloud Computing Platform , 2020, Remote. Sens..

[13]  Muhammad Adeel Hassan,et al.  Assessment of Water and Nitrogen Use Efficiencies Through UAV-Based Multispectral Phenotyping in Winter Wheat , 2020, Frontiers in Plant Science.

[14]  Sergio Marconi,et al.  DeepForest: A Python package for RGB deep learning tree crown delineation , 2020, bioRxiv.

[15]  Nobuyuki Kobayashi,et al.  Crop classification using spectral indices derived from Sentinel-2A imagery , 2019, J. Inf. Telecommun..

[16]  Francesco Marinello,et al.  Assessing the Feasibility of Using Sentinel-2 Imagery to Quantify the Impact of Heatwaves on Irrigated Vineyards , 2019, Remote. Sens..

[17]  Matthias Schubert,et al.  Automated Detection of Conifer Seedlings in Drone Imagery Using Convolutional Neural Networks , 2019, Remote. Sens..

[18]  Giuseppe Modica,et al.  A methodology based on GEOBIA and WorldView-3 imagery to derive vegetation indices at tree crown detail in olive orchards , 2019, Int. J. Appl. Earth Obs. Geoinformation.

[19]  Raul Queiroz Feitosa,et al.  Assessment of CNN-Based Methods for Individual Tree Detection on Images Captured by RGB Cameras Attached to UAVs , 2019, Sensors.

[20]  Hyoung Il Son,et al.  Unmanned Aerial Vehicles in Agriculture: A Review of Perspective of Platform, Control, and Applications , 2019, IEEE Access.

[21]  Nizar Bouguila,et al.  Computer Vision System for Automatic Counting of Planting Microsites Using UAV Imagery , 2019, IEEE Access.

[22]  Abhishek Dutta,et al.  The VGG Image Annotator (VIA) , 2019, ArXiv.

[23]  Jayme Garcia Arnal Barbedo,et al.  A Review on the Use of Unmanned Aerial Vehicles and Imaging Sensors for Monitoring and Assessing Plant Stresses , 2019, Drones.

[24]  Gérard Dedieu,et al.  On the Potentiality of UAV Multispectral Imagery to Detect Flavescence dorée and Grapevine Trunk Diseases , 2018, Remote. Sens..

[25]  Ye Zhang,et al.  Automated Classification Analysis of Geological Structures Based on Images Data and Deep Learning Model , 2018, Applied Sciences.

[26]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[27]  Alavi Kunhu,et al.  Automatic palm trees detection from multispectral UAV data using normalized difference vegetation index and circular Hough transform , 2018, Remote Sensing.

[28]  Xiaolin Zhu,et al.  Automatic detection of individual oil palm trees from UAV images using HOG features and an SVM classifier , 2018, International Journal of Remote Sensing.

[29]  Stefano Nembrini,et al.  The revival of the Gini importance? , 2018, Bioinform..

[30]  P. Wężyk,et al.  Estimating defoliation of Scots pine stands using machine learning methods and vegetation indices of Sentinel-2 , 2018 .

[31]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[32]  Raul Morais,et al.  Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry , 2017, Remote. Sens..

[33]  C. Silva,et al.  Individual tree detection from Unmanned Aerial Vehicle (UAV) derived canopy height model in an open canopy mixed conifer forest , 2017 .

[34]  Jonathan P. Dash,et al.  Assessing very high resolution UAV imagery for monitoring forest health during a simulated disease outbreak , 2017 .

[35]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[37]  Philippe Maillard,et al.  DETECTION AND COUNTING OF ORCHARD TREES FROM VHR IMAGES USING A GEOMETRICAL-OPTICAL MODEL AND MARKED TEMPLATE MATCHING , 2016 .

[38]  Uwe Stilla,et al.  PALM TREE DETECTION USING CIRCULAR AUTOCORRELATION OF POLAR SHAPE MATRIX , 2016 .

[39]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[40]  H. Piégay,et al.  Classification of riparian forest species and health condition using multi-temporal and hyperspatial imagery from unmanned aerial system , 2016, Environmental Monitoring and Assessment.

[41]  Stuart Barr,et al.  UAV-BORNE THERMAL IMAGING FOR FOREST HEALTH MONITORING: DETECTION OF DISEASE-INDUCED CANOPY TEMPERATURE INCREASE , 2015 .

[42]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[44]  J. Flexas,et al.  UAVs challenge to assess water stress for sustainable agriculture , 2015 .

[45]  Naif Alajlan,et al.  Efficient Framework for Palm Tree Detection in UAV Images , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  Mariano Bresciani,et al.  Aquatic vegetation indices assessment through radiative transfer modeling and linear mixture simulation , 2014, Int. J. Appl. Earth Obs. Geoinformation.

[47]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Aleksandrs Urbahs,et al.  Features of the use of unmanned aerial vehicles for agriculture applications , 2013 .

[49]  Johannes Breidenbach,et al.  Early Detection of Bark Beetle Green Attack Using TerraSAR-X and RapidEye Data , 2013, Remote. Sens..

[50]  O. Mutanga,et al.  Using WorldView-2 bands and indices to predict bronze bug (Thaumastocoris peregrinus) damage in plantation forests , 2013 .

[51]  Won Suk Lee,et al.  Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees , 2013 .

[52]  Amin Sedaghat,et al.  Uniform Robust Scale-Invariant Feature Matching for Optical Remote Sensing Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Pablo J. Zarco-Tejada,et al.  Mapping canopy conductance and CWSI in olive orchards using high resolution thermal remote sensing imagery , 2009 .

[54]  Jan Verbesselt,et al.  Forecasting tree mortality using change metrics derived from MODIS satellite data , 2009 .

[55]  C. Stone,et al.  Crown‐scale evaluation of spectral indices for defoliated and discoloured eucalypts , 2008 .

[56]  M. Bindi,et al.  A simple model of regional wheat yield based on NDVI data , 2007 .

[57]  J. G. White,et al.  Aerial Color Infrared Photography for Determining Early In‐Season Nitrogen Requirements in Corn , 2005 .

[58]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[59]  John R. Miller,et al.  Vegetation stress detection through chlorophyll a + b estimation and fluorescence effects on hyperspectral imagery. , 2002, Journal of environmental quality.

[60]  Lei Tian,et al.  Multispectral CIR Image Calibration for Cloud Shadow and Soil Background Influence Using Intensity Normalization , 2002 .

[61]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[63]  B. Datt Remote Sensing of Chlorophyll a, Chlorophyll b, Chlorophyll a+b, and Total Carotenoid Content in Eucalyptus Leaves , 1998 .

[64]  Claus Buschmann,et al.  In vivo spectroscopy and internal optics of leaves as basis for remote sensing of vegetation , 1993 .

[65]  C. Tucker Red and photographic infrared linear combinations for monitoring vegetation , 1979 .

[66]  A. El-Battay,et al.  Deep neural network based date palm tree detection in drone imagery , 2022, Comput. Electron. Agric..

[67]  Ahmad Kamal Nasir,et al.  Precision Forestry: Trees Counting in Urban Areas Using Visible Imagery based on an Unmanned Aerial Vehicle , 2016 .

[68]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[69]  J. A. Schell,et al.  Monitoring vegetation systems in the great plains with ERTS , 1973 .

[70]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.