Rapeseed Stand Count Estimation at Leaf Development Stages With UAV Imagery and Convolutional Neural Networks

Rapeseed is an important oil crop in China. Timely estimation of rapeseed stand count at early growth stages provides useful information for precision fertilization, irrigation, and yield prediction. Based on the nature of rapeseed, the number of tillering leaves is strongly related to its growth stages. However, no field study has been reported on estimating rapeseed stand count by the number of leaves recognized with convolutional neural networks (CNNs) in unmanned aerial vehicle (UAV) imagery. The objectives of this study were to provide a case for rapeseed stand counting with reference to the existing knowledge of the number of leaves per plant and to determine the optimal timing for counting after rapeseed emergence at leaf development stages with one to seven leaves. A CNN model was developed to recognize leaves in UAV-based imagery, and rapeseed stand count was estimated with the number of recognized leaves. The performance of leaf detection was compared using sample sizes of 16, 24, 32, 40, and 48 pixels. Leaf overcounting occurred when a leaf was much bigger than others as this bigger leaf was recognized as several smaller leaves. Results showed CNN-based leaf count achieved the best performance at the four- to six-leaf stage with F-scores greater than 90% after calibration with overcounting rate. On average, 806 out of 812 plants were correctly estimated on 53 days after planting (DAP) at the four- to six-leaf stage, which was considered as the optimal observation timing. For the 32-pixel patch size, root mean square error (RMSE) was 9 plants with relative RMSE (rRMSE) of 2.22% on 53 DAP, while the mean RMSE was 12 with mean rRMSE of 2.89% for all patch sizes. A sample size of 32 pixels was suggested to be optimal accounting for balancing performance and efficiency. The results of this study confirmed that it was feasible to estimate rapeseed stand count in field automatically, rapidly, and accurately. This study provided a special perspective in phenotyping and cultivation management for estimating seedling count for crops that have recognizable leaves at their early growth stage, such as soybean and potato.

[1]  R. Rosner.  Computer software , 1978, Nature.

[2]  R. A. Schonengertt.  Techniques for Image Processing and Classification in Remote Sensing , 1983 .

[3]  H. Bleiholder,et al.  Explanations of the BBCH decimal codes for the growth stages of maize, rape, field beans, sunflower and peas -with illustrations , 1990 .

[4]  P. Lancashire,et al.  A uniform decimal code for growth stages of crops and weeds , 1991 .

[5]  G. Meyer,et al.  Verification of color vegetation indices for automated crop imaging applications , 2008 .

[6]  E. R. Davies,et al.  The application of machine vision to food and agriculture: a review , 2009 .

[7]  Qing Yang,et al.  Genetic analysis on oil content in rapeseed (Brassica napus L.) , 2010, Euphytica.

[8]  S. Robinson,et al.  Food Security: The Challenge of Feeding 9 Billion People , 2010, Science.

[9]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[10]  M. Tester,et al.  Phenomics--technologies to relieve the phenotyping bottleneck. , 2011, Trends in plant science.

[11]  Jeffrey W. White,et al.  Field-based phenomics for plant genetics research , 2012 .

[12]  Chunhua Zhang,et al.  The application of small unmanned aerial systems for precision agriculture: a review , 2012, Precision Agriculture.

[13]  Raphael Linker,et al.  Determination of the number of green apples in RGB images recorded in orchards , 2012 .

[14]  A. Walter,et al.  REVIEW: PART OF A HIGHLIGHT ON BREEDING STRATEGIES FOR FORAGE AND GRASS IMPROVEMENT Advanced phenotyping offers opportunities for improved breeding of forage and turf species , 2012 .

[15]  Georg Heigold,et al.  An empirical study of learning rates in deep neural networks for speech recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Randal K. Taylor,et al.  Automatic corn plant location and spacing measurement using laser line-scan technique , 2013, Precision Agriculture.

[17]  L. Rieseberg,et al.  Agriculture: Feeding the future , 2013, Nature.

[18]  V. Sadras,et al.  The phenotype and the components of phenotypic variance of crop traits , 2013 .

[19]  J. F. Ortega,et al.  Estimation of leaf area index in onion (Allium cepa L.) using an unmanned aerial vehicle , 2013 .

[20]  L. Xiong,et al.  Combining high-throughput phenotyping and genome-wide association studies to reveal natural genetic variation in rice , 2014, Nature Communications.

[21]  M. A. Moreno,et al.  Applications of georeferenced high-resolution images obtained with unmanned aerial vehicles. Part I: Description of image acquisition and processing , 2014, Precision Agriculture.

[22]  Abhiram Das,et al.  Image-Based High-Throughput Field Phenotyping of Crop Roots1[W][OPEN] , 2014, Plant Physiology.

[23]  Simon Bennertz,et al.  Estimating Biomass of Barley Using Crop Surface Models (CSMs) Derived from UAV-Based RGB Imaging , 2014, Remote. Sens..

[24]  Johanna Link,et al.  Combined Spectral and Spatial Modeling of Corn Yield Based on Aerial Images and Crop Surface Models Acquired with an Unmanned Aircraft System , 2014, Remote. Sens..

[25]  F. Baret,et al.  Green area index from an unmanned aerial system over wheat and rapeseed crops , 2014 .

[26]  J. Araus,et al.  Field high-throughput phenotyping: the new crop breeding frontier. , 2014, Trends in plant science.

[27]  T. Sinclair,et al.  Physiological phenotyping of plants for crop improvement. , 2015, Trends in plant science.

[28]  K. Moffett,et al.  Remote Sens , 2015 .

[29]  C. Klukas,et al.  Advanced phenotyping and phenotype data analysis for the study of plant growth and development , 2015, Front. Plant Sci..

[30]  Xiaogang Wang,et al.  Cross-scene crowd counting via deep convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  H. Stein,et al.  Energy concentration and amino acid digestibility in high-protein canola meal, conventional canola meal, and soybean meal fed to growing pigs. , 2015, Journal of animal science.

[32]  Shenghua Gao,et al.  Single-Image Crowd Counting via Multi-Column Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  W. S. Qureshi,et al.  Machine vision for counting fruit on mango tree canopies , 2017, Precision Agriculture.

[34]  Lei Tian,et al.  Development of methods to improve soybean yield estimation and predict plant maturity with an unmanned aerial vehicle based platform , 2016 .

[35]  Nithya Rajan,et al.  Unmanned Aerial Vehicles for High-Throughput Phenotyping and Agronomic Research , 2016, PloS one.

[36]  K. N. Reddy,et al.  Development and evaluation of low-altitude remote sensing systems for crop production management , 2016 .

[37]  R. Snowdon,et al.  Nitrogen use efficiency in rapeseed. A review , 2016, Agronomy for Sustainable Development.

[38]  Andrew Zisserman,et al.  Counting in the Wild , 2016, ECCV.

[39]  Andrew M. Cunliffe,et al.  Ultra-fine grain landscape-scale quantification of dryland vegetation structure with drone-acquired structure-from-motion photogrammetry , 2016 .

[40]  Pablo M. Granitto,et al.  Deep learning for plant identification using vein morphological patterns , 2016, Comput. Electron. Agric..

[41]  Shun-lin Zheng,et al.  Response of Potato Tuber Number and Spatial Distribution to Plant Density in Different Growing Seasons in Southwest China , 2016, Front. Plant Sci..

[42]  F. Baret,et al.  Estimates of plant density of wheat crops at emergence from very low altitude UAV imagery. , 2017 .

[43]  Dan Wu,et al.  Panicle-SEG: a robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization , 2017, Plant Methods.

[44]  Alexander Wendel,et al.  Illumination compensation in ground based hyperspectral imaging , 2017 .

[45]  Carlos Eugenio Oliveros,et al.  Automatic fruit count on coffee branches using computer vision , 2017, Comput. Electron. Agric..

[46]  S. Sankaran,et al.  High-Resolution Aerial Imaging Based Estimation of Crop Emergence in Potatoes , 2017, American Journal of Potato Research.

[47]  Edward J. Delp,et al.  Counting plants using deep learning , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[48]  K. Omasa,et al.  Estimating rice yield related traits and quantitative trait loci analysis under different nitrogen treatments using a simple tower-based field phenotyping system with modified single-lens reflex cameras , 2017 .

[49]  Peijun Du,et al.  A review of supervised object-based land-cover image classification , 2017 .

[50]  Hao Yang,et al.  Unmanned Aerial Vehicle Remote Sensing for Field-Based Crop Phenotyping: Current Status and Perspectives , 2017, Front. Plant Sci..

[51]  F. Baret,et al.  Estimation of Wheat Plant Density at Early Stages Using High Resolution Imagery , 2017, Front. Plant Sci..

[52]  A. Ghulam,et al.  Unmanned Aerial System (UAS)-Based Phenotyping of Soybean using Multi-sensor Data Fusion and Extreme Learning Machine , 2017 .

[53]  Tony P. Pridmore,et al.  Deep Learning for Multi-task Plant Phenotyping , 2017, bioRxiv.

[54]  Urs Schmidhalter,et al.  Digital Counts of Maize Plants by Unmanned Aerial Vehicles (UAVs) , 2017, Remote. Sens..

[55]  Ruizhi Chen,et al.  Monitoring cotton (Gossypium hirsutum L.) germination using ultrahigh-resolution UAS images , 2018, Precision Agriculture.

[56]  Paolo Remagnino,et al.  How deep learning extracts and learns leaf features for plant classification , 2017, Pattern Recognit..

[57]  Tao Liu,et al.  Evaluation of Seed Emergence Uniformity of Mechanically Sown Wheat with UAV RGB Imagery , 2017, Remote. Sens..

[58]  Weijia Li,et al.  Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images , 2016, Remote. Sens..

[59]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[60]  Huawu Deng,et al.  Individual tree crown detection in sub-meter satellite imagery using Marked Point Processes and a geometrical-optical model , 2018, Remote Sensing of Environment.

[61]  Terry Griffin,et al.  Early-Season Stand Count Determination in Corn via Integration of Imagery from Unmanned Aerial Systems (UAS) and Supervised Learning Techniques , 2018, Remote. Sens..

[62]  L. Deng,et al.  UAV-based multispectral remote sensing for precision agriculture: A comparison between different cameras , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[63]  Vishal M. Patel,et al.  A Survey of Recent Advances in CNN-based Single Image Crowd Counting and Density Estimation , 2017, Pattern Recognit. Lett..

[64]  E. Addink,et al.  Monitoring height and greenness of non-woody floodplain vegetation with UAV time series , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[65]  Chenghai Yang,et al.  Rapeseed Seedling Stand Counting and Seeding Performance Evaluation at Two Early Growth Stages Based on Unmanned Aerial Vehicle Imagery , 2018, Front. Plant Sci..

[66]  Changyin Sun,et al.  Crowd Counting via Weighted VLAD on a Dense Attribute Feature Map , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[67]  Maciel Zortea,et al.  A supervised approach for simultaneous segmentation and classification of remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[68]  Frédéric Baret,et al.  Ear density estimation from high resolution RGB imagery using deep learning technique , 2019, Agricultural and Forest Meteorology.

[69]  R R Mir,et al.  High-throughput phenotyping for crop improvement in the genomics era. , 2019, Plant science : an international journal of experimental plant biology.

[70]  J. Araus,et al.  Automatic wheat ear counting using machine learning based on RGB UAV imagery. , 2020, The Plant journal : for cell and molecular biology.

[71]  Wei Zhou,et al.  A Survey of Recent Advances in CNN-Based Fine-Grained Visual Categorization , 2020, 2020 IEEE 20th International Conference on Communication Technology (ICCT).

[72]  G. Still.  Crowd counting , 2021, Applied Crowd Science.