Stronger together: Combining automated classifiers with manual post-validation optimizes the workload vs reliability trade-off of species identification in bat acoustic surveys

Owing to major technological advances, bioacoustics has become a burgeoning field in ecological research worldwide. Autonomous passive acoustic recorders are becoming widely used to monitor aerial insectivorous bats, and automatic classifiers have emerged to aid researchers in the daunting task of analyzing the resulting massive acoustic datasets. However, the scarcity of comprehensive reference call libraries still hampers their wider application in highly diverse tropical assemblages. Capitalizing on a unique acoustic dataset of more than 650,000 bat call sequences collected over a 3-year period in the Brazilian Amazon, the aims of this study were (a) to assess how pre-identified recordings of free-flying and hand-released bats could be used to train an automatic classification algorithm (random forest), and (b) to optimize acoustic analysis protocols by combining automatic classification with visual post-validation, whereby we evaluated the proportion of sound files to be postvalidated for different thresholds of classification accuracy. Classifiers were trained at species or sonotype (group of species with similar calls) level. Random forest models confirmed the reliability of using calls of both free-flying and hand-released bats to train custom-built automatic classifiers. To achieve a general classification accuracy of ~85%, random forest had to be trained with at least 500 pulses per species/sonotype. For seven out of 20 sonotypes, the most abundant in our dataset, we obtained high classification accuracy (>90%). Adopting a desired accuracy probability threshold of 95% for the random forest classifier, we found that the percentage of sound files required for manual post-validation could be reduced by up to 75%, a significant saving in terms of workload. Combining automatic classification with manual ID through fully customizable classifiers implemented in open-source software as demonstrated here shows great potential to help overcome the acknowledged risks and biases associated with the sole reliance on automatic classification.

[1]  April E. Reside,et al.  Acoustic Call Library and Detection Distances for Bats of Swaziland , 2017, Acta Chiropterologica.

[2]  J. Simmons,et al.  Measurements of atmospheric attenuation at ultrasonic frequencies and the significance for echolocation by bats. , 1982, The Journal of the Acoustical Society of America.

[3]  A. López‐Baucells,et al.  Field guide to Amazonian Bats , 2016 .

[4]  Christian C. Voigt,et al.  The use of automated identification of bat echolocation calls in acoustic monitoring: A cautionary note for a sound analysis , 2016 .

[5]  Gareth Jones,et al.  Identification of twenty‐two bat species (Mammalia: Chiroptera) from Italy by analysis of time‐expanded recordings of echolocation calls , 2002 .

[6]  James Christie,et al.  Solo: an open source, customizable and inexpensive audio recorder for bioacoustic research , 2017 .

[7]  E. Bernard,et al.  Neotropical Bats as Indicators of Environmental Disturbance: What is the Emerging Message? , 2012 .

[8]  M. Willig,et al.  Carpe noctem: the importance of bats as bioindicators , 2009 .

[9]  J. A. Simonetti,et al.  Variation in search-phase calls of Lasiurus varius (Chiroptera: Vespertilionidae) in response to different foraging habitats , 2014 .

[10]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[11]  G Mukherjee,et al.  Weighted scale-free networks in Euclidean space using local selection rule. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Bruce W. Miller,et al.  Qualitative Identification of Free-Flying Bats Using the Anabat Detector , 1999 .

[13]  G. B. Williamson,et al.  An Amazonian rainforest and its fragments as a laboratory of global change , 2018, Biological reviews of the Cambridge Philosophical Society.

[14]  Amrita Madhukumar Menon,et al.  Are automated acoustic identification software reliable for bat surveys in the Neotropical region , 2018 .

[15]  Yves Bas,et al.  Tadarida: A Toolbox for Animal Detection on Acoustic Recordings , 2017 .

[16]  Emma C. Teeling,et al.  Female Mate Choice Can Drive the Evolution of High Frequency Echolocation in Bats: A Case Study with Rhinolophus mehelyi , 2014, PloS one.

[17]  C. F. Meyer,et al.  Geographical variation in the high-duty cycle echolocation of the cryptic common mustached bat Pteronotus cf. rubiginosus (Mormoopidae) , 2018 .

[18]  William L. Gannon,et al.  Geographic variation in the echolocation calls of the hoary bat ( Lasiurus cinereus) , 2000 .

[19]  Max Kuhn,et al.  The caret Package , 2007 .

[20]  W. Magnusson,et al.  Aerial insectivorous bat activity in relation to moonlight intensity , 2017 .

[21]  A. L. Gardner.  Mammals of South America, Volume 1: Marsupials, Xenarthrans, Shrews, and Bats , 2008 .

[22]  M. Brock Fenton,et al.  Data, Sample Sizes and Statistics Affect the Recognition of Species of Bats by Their Echolocation Calls , 2004 .

[23]  Kate E. Jones,et al.  Acoustic identification of Mexican bats based on taxonomic and ecological constraints on call design , 2016 .

[24]  Marc O. Lammers,et al.  Comparison of real-time and post-cruise acoustic species identification of dolphin whistles using ROCCA (Real-time Odontocete Call Classification Algorithm) , 2011 .

[25]  G. Jones,et al.  Sex and age variation in echolocation calls and flight morphology of Daubenton’s Bats Myotis daubentonii , 1994 .

[26]  James D. Nichols,et al.  Accounting for false‐positive acoustic detections of bats using occupancy models , 2014 .

[27]  A. Bruckner,et al.  Within-Site Variability of Field Recordings from Stationary, Passively Working Detectors , 2017, Acta Chiropterologica.

[28]  J. M. Palmeirim,et al.  Trait‐related responses to habitat fragmentation in Amazonian bats , 2015 .

[29]  M. Vihinen.  How to evaluate performance of prediction methods? Measures and their interpretation in variation effect analysis , 2012, BMC Genomics.

[30]  Julian D Olden,et al.  Machine Learning Methods Without Tears: A Primer for Ecologists , 2008, The Quarterly Review of Biology.

[31]  Kate E. Barlow,et al.  Citizen science reveals trends in bat populations: The National Bat Monitoring Programme in Great Britain , 2015 .

[32]  T. Penman,et al.  Optimizing ultrasonic sampling effort for monitoring forest bats , 2015 .

[33]  J. M. Palmeirim,et al.  Echolocation of the big red bat Lasiurus egregius (Chiroptera: Vespertilionidae) and first record from the Central Brazilian Amazon , 2014 .

[34]  G. McCracken,et al.  On Estimating the Economic Value of Insectivorous Bats: Prospects and Priorities for Biologists , 2013 .

[35]  Diogo F. Ferreira,et al.  Season‐modulated responses of Neotropical bats to forest fragmentation , 2017, Ecology and evolution.

[36]  E. Bernard,et al.  Bioacoustics as an Important Complementary Tool in Bat Inventories in the Caatinga Drylands of Brazil , 2017, Acta Chiropterologica.

[37]  Ramon Ferrer-i-Cancho,et al.  Acoustic sequences in non‐human animals: a tutorial review and prospectus , 2016, Biological reviews of the Cambridge Philosophical Society.

[38]  Echolocation Calls of the Bats of Trinidad, West Indies: Is Guild Membership Reflected in Echolocation Signal Design? , 2010 .

[39]  K. Winker.  Is it a species , 2010 .

[40]  J. Altringham,et al.  Acoustic Identification of Bats in the Southern Western Ghats, India , 2014 .

[41]  Alex Rogers,et al.  AudioMoth: Evaluation of a smart open acoustic device for monitoring biodiversity and the environment , 2018 .

[42]  E. Britzke,et al.  Surveying bat communities: a comparison between mist nets and the Anabat II bat detector system , 1999 .

[43]  David W. Armitage,et al.  A comparison of supervised learning techniques in the classification of bat echolocation calls , 2010, Ecol. Informatics.

[44]  D. Noble,et al.  Developing indicators for European birds , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  R. Julliard,et al.  Bat activity in intensively farmed landscapes with wind turbines and offset measures , 2015 .

[46]  M. Brock Fenton,et al.  Variation in Echolocation Call Frequencies Recorded from North American Vespertilionid Bats: A Cautionary Note , 1987 .

[47]  J. M. Palmeirim,et al.  Consequences of a large-scale fragmentation experiment for Neotropical bats: disentangling the relative importance of local and landscape-scale effects , 2016, Landscape Ecology.

[48]  Lynn W. Robbins,et al.  Acoustic Identification of Bats in the Eastern United States: A Comparison of Parametric and Nonparametric Methods , 2011 .

[49]  Gareth Jones,et al.  Classification of Echolocation Calls from 14 Species of Bat by Support Vector Machines and Ensembles of Neural Networks , 2009, Algorithms.

[50]  A. L. Gardner.  Mammals of South America , 2007, Journal of Mammalian Evolution.

[51]  David N. Bonter,et al.  Citizen Science as an Ecological Research Tool: Challenges and Benefits , 2010 .

[52]  G. Guillera‐Arroita,et al.  When Is a Species Declining? Optimizing Survey Effort to Detect Population Changes in Reptiles , 2012, PloS one.

[53]  C. F. Meyer,et al.  Molecular, morphological and acoustic identification of Eumops maurus and Eumops hansae (Chiroptera: Molossidae) with new reports from Central Amazonia , 2018 .

[54]  Michael T. Johnson,et al.  A Framework for Bioacoustic Vocalization Analysis Using Hidden Markov Models , 2009, Algorithms.

[55]  Forrest Briggs.  Acoustic Classification of Bird Species from Syllables : an Empirical Study , 2009 .

[56]  C. F. Meyer,et al.  The importance of lakes for bat conservation in Amazonian rainforests: an assessment using autonomous recorders , 2018 .

[57]  Theodoros Damoulas,et al.  Bayesian Classification of Flight Calls with a Novel Dynamic Time Warping Kernel , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[58]  B. Lim,et al.  Bat community structure at Iwokrama Forest, Guyana , 2001, Journal of Tropical Ecology.

[59]  Annemarie Surlykke,et al.  Intensity and directionality of bat echolocation signals , 2013, Front. Physiol..

[60]  Kate E. Jones,et al.  Challenges of Using Bioacoustics to Globally Monitor Bats , 2013 .

[61]  Christian Dietz,et al.  A continental-scale tool for acoustic identification of European bats , 2012 .

[62]  S. Parsons,et al.  Human vs. machine : identification of bat species from their echolocation calls by humans and by artificial neural networks , 2008 .

[63]  Paul A. Racey,et al.  What you see is not what you get : the role of ultrasonic detectors in increasing inventory completeness in Neotropical bat assemblages , 2008 .

[64]  S. Mori,et al.  A central Amazonian terra firme forest. I. High tree species richness on poor soils , 1999, Biodiversity & Conservation.

[65]  G. B. Williamson,et al.  The fate of Amazonian forest fragments: A 32-year investigation , 2011 .

[66]  A. Viera,et al.  Understanding interobserver agreement: the kappa statistic. , 2005, Family medicine.

[67]  E. Kalko,et al.  Adaptability and vulnerability of high flying Neotropical aerial insectivorous bats to urbanization , 2011 .

[68]  Arne Ludwig,et al.  The Value of Molecular vs. Morphometric and Acoustic Information for Species Identification Using Sympatric Molossid Bats , 2016, PloS one.

[69]  David S. Jacobs,et al.  Variation in the echolocation calls of the hoary bat (Lasiurus cinereus) : influence of body size, habitat structure, and geographic location , 1999 .

[70]  M. Girolami,et al.  Bat echolocation call identification for biodiversity monitoring: a probabilistic approach , 2018 .

[71]  C. Kerbiriou,et al.  Use of Large‐Scale Acoustic Monitoring to Assess Anthropogenic Pressures on Orthoptera Communities , 2013, Conservation biology : the journal of the Society for Conservation Biology.

[72]  John Hallam,et al.  Semi-automatic long-term acoustic surveying: A case study with bats , 2014, Ecol. Informatics.

[73]  Diogo F. Ferreira,et al.  Design matters: An evaluation of the impact of small man-made forest clearings on tropical bats using a before-after-control-impact design , 2017 .

[74]  E. Guerrieri,et al.  Pest control service provided by bats in Mediterranean rice paddies: linking agroecosystems structure to ecological functions , 2015 .