An Instance Segmentation-Based Method to Obtain the Leaf Age and Plant Centre of Weeds in Complex Field Environments

Leaf age and plant centre are important phenotypic information of weeds, and accurate identification of them plays an important role in understanding the morphological structure of weeds, guiding precise targeted spraying and reducing the use of herbicides. In this work, a weed segmentation method based on BlendMask is proposed to obtain the phenotypic information of weeds under complex field conditions. This study collected images from different angles (front, side, and top views) of three kinds of weeds (Solanum nigrum, barnyard grass (Echinochloa crus-galli), and Abutilon theophrasti Medicus) in a maize field. Two datasets (with and without data enhancement) and two backbone networks (ResNet50 and ResNet101) were replaced to improve model performance. Finally, seven evaluation indicators are used to evaluate the segmentation results of the model under different angles. The results indicated that data enhancement and ResNet101 as the backbone network could enhance the model performance. The F1 value of the plant centre is 0.9330, and the recognition accuracy of leaf age can reach 0.957. The mIOU value of the top view is 0.642. Therefore, deep learning methods can effectively identify weed leaf age and plant centre, which is of great significance for variable spraying.

[1]  Xiushan Wang,et al.  Research on maize canopy center recognition based on nonsignificant color difference segmentation , 2018, PloS one.

[2]  Yong Jae Lee,et al.  YOLACT: Real-Time Instance Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[3]  Juan C. Moreno.  Plant Physiology and Development , 2015 .

[4]  Hannah M. Dee,et al.  Leaf segmentation through the classification of edges , 2019, ArXiv.

[5]  Hao Chen,et al.  FCOS: Fully Convolutional One-Stage Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Frederic Truchetet,et al.  Wavelet transform to discriminate between crop and weed in perspective agronomic images , 2009 .

[7]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[8]  Meng Li,et al.  A review of computer vision technologies for plant phenotyping , 2020, Comput. Electron. Agric..

[9]  Adel Bakhshipour,et al.  Weed segmentation using texture features extracted from wavelet sub-images , 2017 .

[10]  L. Schreiber,et al.  Epicuticular wax on leaf cuticles does not establish the transpiration barrier, which is essentially formed by intracuticular wax. , 2018, Journal of plant physiology.

[11]  Abdolabbas Jafari,et al.  Evaluation of support vector machine and artificial neural networks in weed detection using shape features , 2018, Comput. Electron. Agric..

[12]  Sotirios A. Tsaftaris,et al.  Doing More With Less: A Multitask Deep Learning Approach in Plant Phenotyping , 2020, Frontiers in Plant Science.

[13]  Mei Jiang,et al.  Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments , 2020, Comput. Electron. Agric..

[14]  Gonzalo Pajares,et al.  On-line crop/weed discrimination through the Mahalanobis distance from images in maize fields , 2018 .

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[17]  Heping Zhu,et al.  Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination , 2011, Sensors.

[18]  Caixia Fan,et al.  Weed and Corn Seedling Detection in Field Based on Multi Feature Fusion and Support Vector Machine , 2020, Sensors.

[19]  Juan E. Tapia,et al.  Deepblueberry: Quantification of Blueberries in the Wild Using Instance Segmentation , 2019, IEEE Access.

[20]  Yi Zhang,et al.  Average Precision , 2009, Encyclopedia of Database Systems.

[21]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Abhishek Dutta,et al.  The VGG Image Annotator (VIA) , 2019, ArXiv.

[23]  Jongyoul Park,et al.  CenterMask: Real-Time Anchor-Free Instance Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Przemyslaw Prusinkiewicz,et al.  The use of plant models in deep learning: an application to leaf counting in rosette plants , 2018, Plant Methods.

[26]  Wang Qi,et al.  Maize seedling detection under different growth stages and complex field environments based on an improved Faster R–CNN , 2019, Biosystems Engineering.

[27]  Yuning Jiang,et al.  SOLO: Segmenting Objects by Locations , 2020, ECCV.

[28]  Kanda Runapongsa Saikaew,et al.  End-to-End Automatic Berry Counting for Table Grape Thinning , 2021, IEEE Access.

[29]  David C. Slaughter,et al.  Autonomous robotic weed control systems: A review , 2008 .

[30]  Bahareh Kalantar,et al.  Land Cover Classification from fused DSM and UAV Images Using Convolutional Neural Networks , 2019, Remote. Sens..

[31]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[32]  Thiago T. Santos,et al.  Grape detection, segmentation and tracking using deep neural networks and three-dimensional association , 2019, Comput. Electron. Agric..

[33]  Dino Ienco,et al.  Deep Recurrent Neural Networks for mapping winter vegetation quality coverage via multi-temporal SAR Sentinel-1 , 2017, ArXiv.

[34]  É. Lehoczky,et al.  Investigation of the Damage Caused by Weeds Competing with Maize for Nutrients , 2009 .

[35]  Yang Yu,et al.  Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN , 2019, Comput. Electron. Agric..

[36]  Hao Chen,et al.  BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Chunhua Shen,et al.  PolarMask: Single Shot Instance Segmentation With Polar Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Josep Ramon Morros,et al.  Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry , 2020, Comput. Electron. Agric..

[39]  Tristan Perez,et al.  Mixtures of Lightweight Deep Convolutional Neural Networks: Applied to Agricultural Robotics , 2017, IEEE Robotics and Automation Letters.

[40]  D. Peddle,et al.  Weed and crop discrimination using hyperspectral image data and reduced bandsets , 2014 .

[41]  Chao Sun,et al.  Deep localization model for intra-row crop detection in paddy field , 2020, Comput. Electron. Agric..

[42]  Dino Ienco,et al.  Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[43]  Adel Hafiane,et al.  Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images , 2018, Remote. Sens..

[44]  Zhe Wang,et al.  Instance segmentation of apple flowers using the improved mask R–CNN model , 2020 .