A comparison of supervised learning techniques in the classification of bat echolocation calls

Abstract Today's acoustic monitoring devices are capable of recording and storing tremendous amounts of data. Until recently, the classification of animal vocalizations from field recordings has been relegated to qualitative approaches. For large-scale acoustic monitoring studies, qualitative approaches are very time-consuming and suffer from the bias of subjectivity. Recent developments in supervised learning techniques can provide rapid, accurate, species-level classification of bioacoustics data. We compared the classification performances of four supervised learning techniques (random forests, support vector machines, artificial neural networks, and discriminant function analysis) for five different classification tasks using bat echolocation calls recorded by a popular frequency-division bat detector. We found that all classifiers performed similarly in terms of overall accuracy with the exception of discriminant function analysis, which had the lowest average performance metrics. Random forests had the advantage of high sensitivities, specificities, and predictive powers across the majority of classification tasks, and also provided metrics for determining the relative importance of call features in distinguishing between groups. Overall classification accuracy for each task was slightly lower than reported accuracies using calls recorded by time-expansion detectors. Myotis spp. were particularly difficult to separate; classifiers performed best when members of this genus were combined in genus-level classification and analyzed separately at the level of species. Additionally, we identified and ranked the relative contributions of all predictor features to classifier accuracy and found measurements of frequency, total call duration, and characteristic slope to be the most important contributors to classification success. We provide recommendations to maximize accuracy and efficiency when analyzing acoustic data, and suggest an application of automated bioacoustics monitoring to contribute to wildlife monitoring efforts.

[1]  S. Parsons,et al.  Human vs. machine : identification of bat species from their echolocation calls by humans and by artificial neural networks , 2008 .

[2]  THEODORE J. WELLER,et al.  Mist Net Effort Required to Inventory a Forest Bat Species Assemblage , 2007 .

[3]  William L. Gannon,et al.  Geographic variation in the echolocation calls of the hoary bat ( Lasiurus cinereus) , 2000 .

[4]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[5]  J. Hayes,et al.  Bat activity in thinned, unthinned, and old-growth forests in western Oregon , 1999 .

[6]  Sovan Lek,et al.  Neuronal Networks: Algorithms and Architectures for Ecologists and Evolutionary Ecologists , 2000 .

[7]  Joseph M. Szewczak,et al.  Detecting, recording and analysing the vocalisations of bats , 2009 .

[8]  Sandhya Samarasinghe,et al.  Neural Networks for Applied Sciences and Engineering: From Fundamentals to Complex Pattern Recognition , 2006 .

[9]  Anne E. Goodenough,et al.  Social networking for biodiversity: The BeeID project , 2010, 2010 International Conference on Information Society.

[10]  Kurt Hornik,et al.  The support vector machine under test , 2003, Neurocomputing.

[11]  B. Block,et al.  A new satellite technology for tracking the movements of Atlantic bluefin tuna. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Héctor Corrada Bravo,et al.  Automated classification of bird and amphibian calls using machine learning: A comparison of methods , 2009, Ecol. Informatics.

[13]  B. Betts Effects of interindividual variation in echolocation calls on identification of big brown and silver-haired bats , 1998 .

[14]  W. Mitchell Masters,et al.  INDIVIDUAL AND GROUP VARIATION IN ECHOLOCATION CALLS OF BIG BROWN BATS, EPTESICUS FUSCUS (CHIROPTERA: VESPERTILIONIDAE) , 2001 .

[15]  Gareth Jones,et al.  Identification of twenty‐two bat species (Mammalia: Chiroptera) from Italy by analysis of time‐expanded recordings of echolocation calls , 2002 .

[16]  C. Moss,et al.  Echolocation in bats and dolphins , 2003 .

[17]  S. Parsons,et al.  Acoustic identification of twelve species of echolocating bat by discriminant function analysis and artificial neural networks. , 2000, The Journal of experimental biology.

[18]  D. Anguita,et al.  A new method for multiclass support vector machines , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[19]  P. Racey Ecological and Behavioral Methods for the Study of Bats , 2011 .

[20]  Habitat Associations of Bat Species in the White Mountain National Forest , 2006 .

[21]  Ji Zhu,et al.  Variable Selection for the Linear Support Vector Machine , 2007, Trends in Neural Computation.

[22]  Sayan Mukherjee,et al.  Classifying Microarray Data Using Support Vector Machines , 2003 .

[23]  Surveying Forest-Bat Communities with Anabat Detectors , 2000 .

[24]  J. Drake,et al.  Modelling ecological niches with support vector machines , 2006 .

[25]  Aaron J. Corcoran Automated acoustic identification of nine bat species of the eastern United States , 2007 .

[26]  Roger Mundry,et al.  Discriminant function analysis with nonindependent data: consequences and an alternative , 2007, Animal Behaviour.

[27]  James D. Malley,et al.  Predictor correlation impacts machine learning algorithms: implications for genomic studies , 2009, Bioinform..

[28]  T. Kuhn The Structure of Scientific Revolutions 2nd edition , 1970 .

[29]  David M. Aanensen,et al.  EpiCollect: Linking Smartphones to Web Applications for Epidemiology, Ecology and Community Data Collection , 2009, PloS one.

[30]  Janet L. Erickson,et al.  Associations of bats with local structure and landscape features of forested stands in western Oregon and Washington , 2003 .

[31]  S. Hurlbert Pseudoreplication and the Design of Ecological Field Experiments , 1984 .

[32]  A. Herr,et al.  Identifaction of Bat Echolocation Calls Using a Decision Classification System. , 1997 .

[33]  Thomas H. Kunz,et al.  Ecological and behavioral methods for the study of bats, 2nd edition , 2009 .

[34]  Max Kuhn,et al.  Building Predictive Models in R Using the caret Package , 2008 .

[35]  M. Fenton,et al.  Choosing the correct' bat detector , 2000 .

[36]  M. Fenton,et al.  Recognition of Species of Insectivorous Bats by Their Echolocation Calls , 1981 .

[37]  Halbert White,et al.  Artificial Neural Networks: Approximation and Learning Theory , 1992 .

[38]  Bruce W. Miller,et al.  Confronting the Dogma: a Reply , 1999 .

[39]  Yan Cui,et al.  Prediction of the phenotypic effects of non-synonymous single nucleotide polymorphisms using structural and evolutionary information , 2005, Bioinform..

[40]  Sovan Lek,et al.  Artificial Neuronal Networks: Application To Ecology And Evolution , 2012 .

[41]  Keith A. Hobson,et al.  BAT ACTIVITY IN THE BOREAL FOREST: IMPORTANCE OF STAND TYPE AND VERTICAL STRATA , 1999 .

[42]  Gareth Jones,et al.  Classification of Echolocation Calls from 14 Species of Bat by Support Vector Machines and Ensembles of Neural Networks , 2009, Algorithms.

[43]  G. Fellers,et al.  Choosing the "correct" bat detector - A reply , 2001 .

[44]  Seppo Ilmari Fagerlund,et al.  Bird Species Recognition Using Support Vector Machines , 2007, EURASIP J. Adv. Signal Process..

[45]  Paola Laiolo,et al.  The emerging significance of bioacoustics in animal species conservation , 2010 .

[46]  D. Preatoni,et al.  IDENTIFYING BATS FROM TIME-EXPANDED RECORDINGS OF SEARCH CALLS: COMPARING CLASSIFICATION METHODS , 2005 .

[47]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[48]  M. Obrist,et al.  Variability in echolocation call design of 26 Swiss bat species: consequences, limits and options for automated field identification with a synergetic pattern recognition approach , 2004 .

[49]  R. Barclay,et al.  Bats are not birds- a cautionary note on using echolocation calls to identify bats: a comment , 1999 .

[50]  Alan H. Fielding,et al.  Machine Learning Methods for Ecological Applications , 2012, Springer US.

[51]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[52]  Bruce W. Miller,et al.  Qualitative Identification of Free-Flying Bats Using the Anabat Detector , 1999 .

[53]  J. Nichols,et al.  ESTIMATION OF TIGER DENSITIES IN INDIA USING PHOTOGRAPHIC CAPTURES AND RECAPTURES , 1998 .

[54]  Chenn-Jung Huang,et al.  Frog classification using machine learning techniques , 2009, Expert Syst. Appl..

[55]  Shigeo Abe Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[56]  D. R. Cutler,et al.  Utah State University From the SelectedWorks of , 2017 .

[57]  Mark D Skowronski,et al.  Acoustic detection and classification of Microchiroptera using machine learning: lessons learned from automatic speech recognition. , 2006, The Journal of the Acoustical Society of America.

[58]  David A Hill,et al.  Acoustic Identification of Eight Species of Bat (Mammalia: Chiroptera) Inhabiting Forests of Southern Hokkaido, Japan: Potential for Conservation Monitoring , 2004, Zoological science.

[59]  Alan H. Fielding,et al.  How should accuracy be measured , 1999 .

[60]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[61]  Kellie J. Archer,et al.  Empirical characterization of random forest variable importance measures , 2008, Comput. Stat. Data Anal..

[62]  L. P. J. Veelenturf,et al.  Analysis and applications of artificial neural networks , 1995 .

[63]  Mark D Skowronski,et al.  Quantifying bat call detection performance of humans and machines. , 2009, The Journal of the Acoustical Society of America.

[64]  Xianggui Qu,et al.  Multivariate Data Analysis , 2007, Technometrics.

[65]  S. Harris,et al.  Identification of British bat species by multivariate analysis of echolocation call parameters , 1997 .

[66]  M. Brock Fenton,et al.  Data, Sample Sizes and Statistics Affect the Recognition of Species of Bats by Their Echolocation Calls , 2004 .

[67]  George E. Marks,et al.  Bats of Florida , 2006 .

[68]  Lee A. Miller,et al.  A Portable Ultrasonic Detection System for Recording Bat Cries in the Field , 1977 .

[69]  Jennifer M. Menzel,et al.  EFFECT OF HABITAT AND FORAGING HEIGHT ON BAT ACTIVITY IN THE COASTAL PLAIN OF SOUTH CAROLINA , 2005 .

[70]  M. Brock Fenton,et al.  A technique for monitoring bat activity with results obtained from different environments in southern Ontario , 1970 .