Deep Learning Applied to Phenotyping of Biomass in Forages with UAV-Based RGB Imagery

Monitoring biomass of forages in experimental plots and livestock farms is a time-consuming, expensive, and biased task. Thus, non-destructive, accurate, precise, and quick phenotyping strategies for biomass yield are needed. To promote high-throughput phenotyping in forages, we propose and evaluate the use of deep learning-based methods and UAV (Unmanned Aerial Vehicle)-based RGB images to estimate the value of biomass yield by different genotypes of the forage grass species Panicum maximum Jacq. Experiments were conducted in the Brazilian Cerrado with 110 genotypes with three replications, totaling 330 plots. Two regression models based on Convolutional Neural Networks (CNNs) named AlexNet and ResNet18 were evaluated, and compared to VGGNet—adopted in previous work in the same thematic for other grass species. The predictions returned by the models reached a correlation of 0.88 and a mean absolute error of 12.98% using AlexNet considering pre-training and data augmentation. This proposal may contribute to forage biomass estimation in breeding populations and livestock areas, as well as to reduce the labor in the field.

[1]  Simon Bennertz,et al.  Estimating Biomass of Barley Using Crop Surface Models (CSMs) Derived from UAV-Based RGB Imaging , 2014, Remote. Sens..

[2]  Puyu Feng,et al.  Machine learning-based integration of remotely-sensed drought factors can improve the estimation of agricultural drought in South-Eastern Australia , 2019, Agricultural Systems.

[3]  Jonathan Li,et al.  A Machine Learning Framework to Predict Nutrient Content in Valencia-Orange Leaf Hyperspectral Measurements , 2020, Remote. Sens..

[4]  Tao Liu,et al.  Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system , 2018 .

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Aditya Khamparia,et al.  A systematic review on deep learning architectures and applications , 2019, Expert Syst. J. Knowl. Eng..

[7]  Jonathan Li,et al.  Predicting Canopy Nitrogen Content in Citrus-Trees Using Random Forest Algorithm Associated to Spectral Vegetation Indices from UAV-Imagery , 2019, Remote. Sens..

[8]  Naser El-Sheimy,et al.  CROP ROW DETECTION PROCEDURE USING LOW-COST UAV IMAGERY SYSTEM , 2019, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[9]  José Hernández-Orallo,et al.  ROC curves for regression , 2013, Pattern Recognit..

[10]  Heping Zhang,et al.  Comparison of machine learning algorithms for classification of LiDAR points for characterization of canola canopy structure , 2019, International Journal of Remote Sensing.

[11]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[12]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Dharmendra Saraswat,et al.  Evaluating remotely sensed plant count accuracy with differing unmanned aircraft system altitudes, physical canopy separations, and ground covers , 2017 .

[15]  Maoguo Gong,et al.  Automatic Tobacco Plant Detection in UAV Images via Deep Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Yiannis Ampatzidis,et al.  UAV-Based Remote Sensing Technique to Detect Citrus Canker Disease Utilizing Hyperspectral Imaging and Machine Learning , 2019, Remote. Sens..

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Eija Honkavaara,et al.  A Novel Deep Learning Method to Identify Single Tree Species in UAV-Based Hyperspectral Images , 2020, Remote. Sens..

[19]  John Langford,et al.  Beating the hold-out: bounds for K-fold and progressive cross-validation , 1999, COLT '99.

[20]  Simon Bennertz,et al.  Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring in barley , 2015, Int. J. Appl. Earth Obs. Geoinformation.

[21]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[22]  Flor Álvarez-Taboada,et al.  Spectroscopic Determination of Aboveground Biomass in Grasslands Using Spectral Transformations, Support Vector Machine and Partial Least Squares Regression , 2013, Sensors.

[23]  Lingxian Zhang,et al.  Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network , 2019, European Journal of Agronomy.

[24]  Nitesh K. Poona,et al.  Modelling Water Stress in a Shiraz Vineyard Using Hyperspectral Imaging and Machine Learning , 2018, Remote. Sens..

[25]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[26]  Adel Hafiane,et al.  Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images , 2018, Remote. Sens..

[27]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Hai Tao,et al.  Review of deep convolution neural network in image classification , 2017, 2017 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET).

[29]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[30]  Mathew Legg,et al.  Ultrasonic Arrays for Remote Sensing of Pasture Biomass , 2019, Remote. Sens..

[31]  Kevin F. Smith,et al.  Prospects for Measurement of Dry Matter Yield in Forage Breeding Programs Using Sensor Technologies , 2019, Agronomy.

[32]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[33]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Michael Wachendorf,et al.  Fusion of Ultrasonic and Spectral Sensor Data for Improving the Estimation of Biomass in Grasslands with Heterogeneous Sward Structure , 2017, Remote. Sens..

[35]  Sholom M. Weiss,et al.  An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods , 1989, IJCAI.

[36]  Eija Honkavaara,et al.  Estimating Biomass and Nitrogen Amount of Barley and Grass Using UAV and Aircraft Based Spectral and Photogrammetric 3D Features , 2018, Remote. Sens..

[37]  L. Jank,et al.  The value of improved pastures to Brazilian beef production , 2014, Crop and Pasture Science.

[38]  M. Weiss,et al.  Remote sensing for agricultural applications: A meta-review , 2020 .

[39]  M. Wachendorf,et al.  Remote sensing as a tool to assess botanical composition, structure, quantity and quality of temperate grasslands , 2018 .

[40]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[41]  Benjamin Wilkinson,et al.  Aboveground Biomass Estimation in Amazonian Tropical Forests: a Comparison of Aircraft- and GatorEye UAV-borne LiDAR Data in the Chico Mendes Extractive Reserve in Acre, Brazil , 2020, Remote. Sens..

[42]  ZhangGuangquan,et al.  Transfer learning using computational intelligence , 2015 .

[43]  Eija Honkavaara,et al.  A Novel Machine Learning Method for Estimating Biomass of Grass Swards Using a Photogrammetric Canopy Height Model, Images and Vegetation Indices Captured by a Drone , 2018 .

[44]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[45]  J. F. Ortega,et al.  Onion biomass monitoring using UAV-based RGB imaging , 2018, Precision Agriculture.

[46]  Li Zhang,et al.  Above-ground biomass estimation and yield prediction in potato by using UAV-based RGB and hyperspectral imaging , 2020 .

[47]  Jonathan Li,et al.  Estimating Pasture Biomass and Canopy Height in Brazilian Savanna Using UAV Photogrammetry , 2019, Remote. Sens..

[48]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[49]  R. L. Thorndike.  Who belongs in the family? , 1953 .

[50]  Jefferson R. Souza,et al.  Corn Plant Counting Using Deep Learning and UAV Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[51]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[52]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[53]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.