Computational Bioacoustic Scene Analysis

The analysis of natural and animal sound makes a demonstrable contribution to important challenges in conservation, animal behaviour, and evolution. And now bioacoustics has entered its big data era. Thus automation is important, as is scalability in many cases to very large amounts of audio data and to real-time processing. This chapter will focus on the data science and the computational methods that can enable this. Computational bioacoustics has some commonalities with wider audio scene analysis, as well as with speech processing and other disciplines. However, the tasks required and the specific characteristics of bioacoustic data require new and adapted techniques. This chapter will survey the tasks and the methods of computational bioacoustics, and will place particular emphasis on existing work and future prospects which address scalable analysis. We will mostly focus on airborne sound; there has also been much work on freshwater and marine bioacoustics, and a small amount on solid-borne sounds.

[1]  Thad Starner,et al.  Feature Learning and Automatic Segmentation for Dolphin Communication Analysis , 2016, INTERSPEECH.

[2]  Hiroshi G. Okuno,et al.  Call Alternation Between Specific Pairs of Male Frogs Revealed by a Sound-Imaging Method in Their Natural Habitat , 2016, INTERSPEECH.

[3]  Lukas Machlica,et al.  Automatic recognition of bird individuals on an open set using as-is recordings , 2016 .

[4]  Wolfgang Goymann,et al.  Patterns of call communication between group-housed zebra finches change during the breeding cycle , 2015, eLife.

[5]  Michael T. Johnson,et al.  A Framework for Bioacoustic Vocalization Analysis Using Hidden Markov Models , 2009, Algorithms.

[6]  Todor Ganchev,et al.  Audio parameterization with robust frame selection for improved bird identification , 2015, Expert Syst. Appl..

[7]  Michael S. Webster and Gregory F. Budney Sound Archives and Media Specimens in the 21st Century , 2016 .

[8]  Arik Kershenbaum,et al.  Animal vocal sequences: not the Markov chains we thought they were , 2014, Proceedings of the Royal Society B: Biological Sciences.

[9]  Hannes Sagunsky,et al.  Zebra Finch Mates Use Their Forebrain Song System in Unlearned Call Communication , 2014, PloS one.

[10]  Stephen Nowicki Context-dependent categorical perception in a songbird , 2015 .

[11]  Nadia Pieretti,et al.  Acoustic Indices for Biodiversity Assessment and Landscape Investigation , 2014 .

[12]  J. P. Lewis Fast Normalized Cross-Correlation , 2010 .

[13]  Xiaoli Z. Fern,et al.  Dictionary Learning for Bioacoustics Monitoring with Applications to Species Classification , 2016, Journal of Signal Processing Systems.

[14]  Almo Farina,et al.  Ecoacoustics: the Ecological Investigation and Interpretation of Environmental Sound , 2015, Biosemiotics.

[15]  Rachel T. Buxton,et al.  Measuring nocturnal seabird activity and status using acoustic recording devices: applications for island restoration , 2012 .

[16]  Mark D. Plumbley,et al.  Large‐scale analysis of frequency modulation in birdsong data bases , 2013, ArXiv.

[17]  Paola Laiolo,et al.  The emerging significance of bioacoustics in animal species conservation , 2010 .

[18]  Sebastian Menze,et al.  The influence of sea ice, wind speed and marine mammals on Southern Ocean ambient sound , 2017, Royal Society Open Science.

[19]  Hervé Glotin,et al.  LifeCLEF Bird Identification Task 2016: The arrival of Deep learning , 2016, CLEF.

[20]  Giuseppa Buscaino,et al.  Temporal patterns in the soundscape of the shallow waters of a Mediterranean marine protected area , 2016, Scientific Reports.

[21]  Ciira wa Maina,et al.  A Bioacoustic Record of a Conservancy in the Mount Kenya Ecosystem , 2016, Biodiversity data journal.

[22]  Carel ten Cate,et al.  The Progressive Loss of Syntactical Structure in Bird Song along an Island Colonization Chain , 2013, Current Biology.

[23]  Arik Kershenbaum,et al.  Disentangling canid howls across multiple species and subspecies: Structure in a complex communication channel , 2016, Behavioural Processes.

[24]  Kentaro Abe,et al.  Songbirds possess the spontaneous ability to discriminate syntactic rules , 2011, Nature Neuroscience.

[25]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[26]  R. Zann The Zebra Finch: A Synthesis of Field and Laboratory Studies , 1996 .

[27]  Maria Sandsten,et al.  Robust feature representation for classification of bird song syllables , 2016, EURASIP J. Adv. Signal Process..

[28]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[29]  Willem H. Zuidema Context-freeness Revisited , 2013, CogSci.

[30]  Sandrine Pavoine,et al.  Assessing biodiversity with sound: Do acoustic diversity indices reflect phylogenetic and functional diversities of bird communities? , 2013 .

[31]  Almo Farina,et al.  A new methodology to infer the singing activity of an avian community: The Acoustic Complexity Index (ACI) , 2011 .

[32]  R. Lachlan,et al.  Context-dependent categorical perception in a songbird , 2015, Proceedings of the National Academy of Sciences.

[33]  R. H. Wiley Associations of Song Properties with Habitats for Territorial Oscine Birds of Eastern North America , 1991, The American Naturalist.

[34]  Jeffrey Podos,et al.  A fine-scale, broadly applicable index of vocal performance: frequency excursion , 2016, Animal Behaviour.

[35]  Paul Roe,et al.  Timed Probabilistic Automaton: A Bridge between Raven and Song Scope for Automatic Species Recognition , 2013, IAAI.

[36]  Hielke Freerk Boersma Characterization of the natural ambient sound environment: Measurements in open agricultural grassland , 1997 .

[37]  E. Vannoni,et al.  Low Frequency Groans Indicate Larger and More Dominant Fallow Deer (Dama dama) Males , 2008, PloS one.

[38]  Dan Stowell,et al.  On-Bird Sound Recordings: Automatic Acoustic Recognition of Activities and Contexts , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[39]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[40]  Paul Roe,et al.  A toolbox for animal call recognition , 2012 .

[41]  Christian Dietz,et al.  A continental-scale tool for acoustic identification of European bats , 2012 .

[42]  Otilia Kocsis,et al.  Automated Acoustic Classification of Bird Species from Real -Field Recordings , 2012, 2012 IEEE 24th International Conference on Tools with Artificial Intelligence.

[43]  E. Briefer,et al.  Indicators of Age, Body Size and Sex in Goat Kid Calls Revealed Using the Source–Filter Theory , 2011 .

[44]  Mario Lasseck Bird Song Classification in Field Recordings: Winning Solution for NIPS4B 2013 Competition * , 2013 .

[45]  Dan Stowell,et al.  Detailed temporal structure of communication networks in groups of songbirds , 2016, bioRxiv.

[46]  R. Ranft Natural sound archives: past, present and future. , 2004, Anais da Academia Brasileira de Ciencias.

[47]  Michael A. Casey,et al.  Song Intersection by Approximate Nearest Neighbor Search , 2006, ISMIR.

[48]  F. Jiguet,et al.  Monitoring temporal change of bird communities with dissimilarity acoustic indices , 2014 .

[49]  Robert Carrington Stein Modulation in bird sounds , 1968 .

[50]  Xiaoli Z. Fern,et al.  Audio Classification of Bird Species: A Statistical Manifold Approach , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[51]  M. Padgham Reverberation and frequency attenuation in forests--implications for acoustic communication in animals. , 2004, The Journal of the Acoustical Society of America.

[52]  P. Marler,et al.  Nature's Music: The Science of Birdsong , 2004 .

[53]  Sandrine Pavoine,et al.  Rapid Acoustic Survey for Biodiversity Appraisal , 2008, PloS one.

[54]  Hervé Glotin,et al.  Bird detection in audio: A survey and a challenge , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[55]  D. Ryan Norris,et al.  Three decades of cultural evolution in Savannah sparrow songs , 2013, Animal Behaviour.

[56]  Hervé Glotin,et al.  Scattering Decomposition for Massive Signal Classification: From Theory to Fast Algorithm and Implementation with Validation on International Bioacoustic Benchmark , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[57]  Daniel J. Mennill,et al.  Black-capped chickadees, Poecile atricapillus, avoid song overlapping: evidence for the acoustic interference hypothesis , 2016, Animal Behaviour.

[58]  Len Thomas,et al.  A method for detecting whistles, moans, and other frequency contour sounds. , 2011, The Journal of the Acoustical Society of America.

[59]  Andreas M. Ali,et al.  Acoustic monitoring in terrestrial environments using microphone arrays: applications, technological considerations and prospectus , 2011 .

[60]  Dan Stowell,et al.  Segregating event streams and noise with a Markov renewal process model , 2012, J. Mach. Learn. Res..

[61]  Xavier Anguera Miró,et al.  Acoustic Beamforming for Speaker Diarization of Meetings , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[62]  Zhixin Chen,et al.  Semi-automatic classification of bird vocalizations using spectral peak tracks. , 2006, The Journal of the Acoustical Society of America.

[63]  Fabian J. Theis,et al.  The signal separation evaluation campaign (2007-2010): Achievements and remaining challenges , 2012, Signal Process..

[64]  Michael W. Towsey,et al.  Visualization of Long-duration Acoustic Recordings of the Environment , 2014, ICCS.

[65]  T. Mitchell Aide,et al.  Real-time bioacoustics monitoring and automated species identification , 2013, PeerJ.

[66]  Nadia Pieretti,et al.  The soundscape methodology for long-term bird monitoring: A Mediterranean Europe case-study , 2011, Ecol. Informatics.

[67]  Hiroshi G. Okuno,et al.  Localizing Bird Songs Using an Open Source Robot Audition System with a Microphone Array , 2016, INTERSPEECH.

[68]  Panu Somervuo,et al.  Classification of the harmonic structure in bird vocalization , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[69]  Peter Jancovic,et al.  Acoustic Recognition of Multiple Bird Species Based on Penalized Maximum Likelihood , 2015, IEEE Signal Processing Letters.

[70]  Dan Stowell,et al.  Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning , 2014, PeerJ.

[71]  Frédéric E. Theunissen,et al.  The vocal repertoire of the domesticated zebra finch: a data-driven approach to decipher the information-bearing acoustic features of communication signals , 2016, Animal Cognition.

[72]  Kurt M Fristrup,et al.  Accuracy of an acoustic location system for monitoring the position of duetting songbirds in tropical forest. , 2006, The Journal of the Acoustical Society of America.

[73]  E. Vannoni,et al.  Fallow bucks get hoarse: vocal fatigue as a possible signal to conspecifics , 2009, Animal Behaviour.

[74]  S. Peters,et al.  Are there species-universal categories in bird song phonology and syntax? A comparative study of chaffinches (Fringilla coelebs), zebra finches (Taenopygia guttata), and swamp sparrows (Melospiza georgiana). , 2010, Journal of comparative psychology.

[75]  A. Thode,et al.  PAMGUARD : SEMIAUTOMATED , OPEN SOURCE SOFTWARE FOR REAL-TIME ACOUSTIC DETECTION AND LOCALISATION OF CETACEANS , 2008 .

[76]  Paul R. White,et al.  An adaptive filter-based method for robust, automatic detection and frequency estimation of whistles. , 2011, The Journal of the Acoustical Society of America.

[77]  Andreas Spanias,et al.  Segmentation, Indexing, and Retrieval for Environmental and Natural Sounds , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[78]  Abraham L Borker,et al.  Vocal Activity as a Low Cost and Scalable Index of Seabird Colony Size , 2014, Conservation biology : the journal of the Society for Conservation Biology.

[79]  Simon W. Townsend,et al.  Experimental Evidence for Phonemic Contrasts in a Nonhuman Vocal System , 2015, PLoS biology.

[80]  Kazuhiro Nakadai,et al.  Semi-automatic bird song analysis by spatial-cue-based integration of sound source detection, localization, separation, and identification , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[81]  Panu Somervuo,et al.  Parametric Representations of Bird Sounds for Automatic Species Recognition , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[82]  Kevin P. Murphy,et al.  Linear-time inference in Hierarchical HMMs , 2001, NIPS.

[83]  T. Scott Brandes,et al.  Automated sound recording and analysis techniques for bird surveys and conservation , 2008, Bird Conservation International.

[84]  H. C. Card,et al.  Birdsong recognition using backpropagation and multivariate statistics , 1997, IEEE Trans. Signal Process..

[85]  H. Soula,et al.  Impact of visual contact on vocal interaction dynamics of pair-bonded birds , 2015, Animal Behaviour.

[86]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[87]  C. E. Ho,et al.  A procedure for an automated measurement of song similarity , 2000, Animal Behaviour.

[88]  P. Tyack,et al.  Estimating animal population density using passive acoustics , 2012, Biological reviews of the Cambridge Philosophical Society.

[89]  Paul E. Allen,et al.  Random Forest for improved analysis efficiency in passive acoustic monitoring , 2014, Ecol. Informatics.

[90]  Ramon Ferrer-i-Cancho,et al.  Acoustic sequences in non‐human animals: a tutorial review and prospectus , 2016, Biological reviews of the Cambridge Philosophical Society.

[91]  Eduardo Freire Nakamura,et al.  An incremental technique for real-time bioacoustic signal segmentation , 2015, Expert Syst. Appl..

[92]  Daniel Hernández-Lobato,et al.  Generalized spike-and-slab priors for Bayesian group feature selection using expectation propagation , 2013, J. Mach. Learn. Res..

[93]  Murray G. Efford,et al.  Bird population density estimated from acoustic signals , 2009 .

[94]  Seppo Ilmari Fagerlund,et al.  Bird Species Recognition Using Support Vector Machines , 2007, EURASIP J. Adv. Signal Process..

[95]  Ofer Tchernichovski,et al.  Quantification of developmental birdsong learning from the subsyllabic scale to cultural evolution , 2011, Proceedings of the National Academy of Sciences.

[96]  Andrew Y. Ng,et al.  Learning Feature Representations with K-Means , 2012, Neural Networks: Tricks of the Trade.