Bulk arthropod abundance, biomass and diversity estimation using deep learning for computer vision

Arthropod abundance, biomass and taxonomic diversity are key metrics often used to assess the efficacy of restoration efforts. Gathering these metrics is a slow and laborious process, quantified by an expert manually sorting and weighing arthropod specimens. We present a tool to accelerate bulk arthropod classification and biomass estimates utilizing machine learning methods for computer vision. Our approach requires pre‐sorted arthropod samples to create a training dataset. We construct a dataset considering 18 terrestrial arthropod functional groups collected in southern Ontario, Canada. The dataset contains 517 high‐resolution images with approximately 20 individuals per image taken from either a petri dish or a bulk tray. Our tool uses the watershed algorithm to obtain precisely cropped individuals without any object annotations. After manually sorting cropped images of biological ‘debris’ and petri dish edges, three classifiers, DenseNet121, ResNet101 and MobileNetv2, each with trade‐offs of computational efficiency versus accuracy, are trained and compared to predict arthropod functional groups for each cropped individual. To calculate biomass, we compare seven linear and nonlinear models considering the arthropod pixel masks obtained using the watershed algorithm, in combination with images of a single function group with recorded weights, to calculate the per pixel density per functional group. From our experimentation, we recommend using DenseNet121 as it had the highest top‐1 functional group classification accuracy, likely a result of being the model with the largest number of parameters, with 86.14% considering the 20 labelled classes (18 arthropods plus debris and petri dish edge) in comparison to ResNet101 (85.10%) and MobileNetv2 (84.94%). For biomass estimation, we recommend using the average per pixel density which had the highest ranked performance considering both total error, 0.043 g (0.855% error), and cumulative class‐specific error, 1.62 g (40.67% average error across all classes), in comparison to the total ground truth biomass of 5.10 g. Our estimated Simpson's Index of Diversity was 0.9404 in comparison to the ground truth 0.9408. Our method simultaneously classifies >1,000 arthropods to functional groupings while estimating total and class specific biomass, without any computer vision bounding box or mask annotations, all from a single photo. We release our code and dataset to further research efforts in computer vision for arthropod classification.

[1]  Stefan Schneider,et al.  Author response for "Bulk arthropod abundance, biomass and diversity estimation using deep learning for computer vision" , 2021 .

[2]  Yingqiu Huo,et al.  Recognition of Common Insect in Field Based on Deep Learning , 2020, Journal of Physics: Conference Series.

[3]  D. Hobern.  BIOSCAN: DNA Barcoding to accelerate taxonomy and biogeography for conservation and sustainability. , 2020, Genome.

[4]  D. Wagner.  Insect Declines in the Anthropocene. , 2020, Annual review of entomology.

[5]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[6]  Christian Ammer,et al.  Arthropod decline in grasslands and forests is associated with landscape-level drivers , 2019, Nature.

[7]  K. Wyckhuys,et al.  Worldwide decline of the entomofauna: A review of its drivers , 2019, Biological Conservation.

[8]  Graham W. Taylor,et al.  Past, present and future approaches using computer vision for animal re‐identification from camera trap data , 2018, Methods in Ecology and Evolution.

[9]  Paolo Napoletano,et al.  Benchmark Analysis of Representative Deep Neural Network Architectures , 2018, IEEE Access.

[10]  Mark W. Schmidt,et al.  Where are the Blobs: Counting by Localization with Point Supervision , 2018, ECCV.

[11]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[12]  Graham W. Taylor,et al.  Deep Learning Object Detection Methods for Ecological Camera Trap Data , 2018, 2018 15th Conference on Computer and Robot Vision (CRV).

[13]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  H. de Kroon,et al.  More than 75 percent decline over 27 years in total flying insect biomass in protected areas , 2017, PloS one.

[15]  S. Ratnasingham,et al.  Expedited Assessment of Terrestrial Arthropod Diversity by Coupling Malaise Traps with DNA Barcoding , 2017, bioRxiv.

[16]  Romain Raveaux,et al.  A survey on image-based insect classification , 2017, Pattern Recognit..

[17]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Forrest N. Iandola,et al.  DenseNet: Implementing Efficient ConvNet Descriptor Pyramids , 2014, ArXiv.

[23]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  E. Borer,et al.  Plant diversity controls arthropod biomass and temporal stability. , 2012, Ecology letters.

[26]  R. Didham,et al.  Landscape moderation of biodiversity patterns and processes ‐ eight hypotheses , 2012, Biological reviews of the Cambridge Philosophical Society.

[27]  Robert Planqué,et al.  Do differences in food web structure between organic and conventional farms affect the ecosystem service of pest control? , 2009, Ecology letters.

[28]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[29]  D. Janzen,et al.  Ten species in one: DNA barcoding reveals cryptic species in the neotropical skipper butterfly Astraptes fulgerator. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Shigeru Nakano,et al.  TERRESTRIAL–AQUATIC LINKAGES: RIPARIAN ARTHROPOD INPUTS ALTER TROPHIC CASCADES IN A STREAM FOOD WEB , 1999 .

[31]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[32]  Robert K. Colwell,et al.  Terrestrial Arthropod Assemblages: Their Use in Conservation Planning , 1993 .

[33]  G. B. Schaalje,et al.  Identification of ichneumonid wasps using image analysis of wings , 1992 .

[34]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[35]  Pierre Machart.  Morphological Segmentation , 2009 .

[36]  J. Free.  Insect pollination of crops , 1970 .