A neural ensemble correlation code for sound category identification

Humans and other animals effortlessly identify sounds and categorize them into behaviorally relevant categories. Yet, the acoustic features and neural transformations that enable the formation of perceptual categories are largely unknown. Here we demonstrate that correlation statistics between frequency-organized cochlear sound channels are reflected in the neural ensemble activity of the auditory midbrain and that such activity, in turn, can contribute to discrimination of perceptual categories. Using multi-channel neural recordings in the auditory midbrain of unanesthetized rabbits, we first demonstrate that neuron ensemble correlations are highly structured in both time and frequency and can be decoded to distinguish sounds. Next, we develop a probabilistic framework for measuring the nonstationary spectro-temporal correlation statistics between frequency organized channels in an auditory model. In a 13-category sound identification task, classification accuracy is consistently high (>80%), improving with sound duration and plateauing at ~ 1-3 seconds, mirroring human performance trends. Nonstationary short-term correlation statistics are more informative about the sound category than the time-average correlation statistics (84% vs. 73% accuracy). When tested independently, the spectral and temporal correlations between the model outputs achieved a similar level of performance and appear to contribute equally. These results outline a plausible neural code in which correlation statistics between neuron ensembles of different frequencies can be read-out to identify and distinguish acoustic categories.

[1]  C. Schreiner,et al.  Nonlinear Spectrotemporal Sound Analysis by Neurons in the Auditory Midbrain , 2002, The Journal of Neuroscience.

[2]  S. Shamma,et al.  Temporal Coherence in the Perceptual Organization and Cortical Representation of Auditory Scenes , 2009, Neuron.

[3]  G. Barlow Gobies of the Genus Gillichthys, with Comments on the Sensory Canals as a Taxonomic Tool , 1961 .

[4]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[5]  Mamiko Niwa,et al.  Task Engagement Selectively Modulates Neural Correlations in Primary Auditory Cortex , 2015, The Journal of Neuroscience.

[6]  Monty A Escabí,et al.  Neural Modulation Tuning Characteristics Scale to Efficiently Encode Natural Sound Statistics , 2010, The Journal of Neuroscience.

[7]  R. DeCharms,et al.  Information coding in the cortex by independent or coordinated populations. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[8]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[9]  Benedikt Grothe,et al.  Efficient Temporal Processing of Naturalistic Sounds , 2008, PloS one.

[10]  M N Semple,et al.  Representation of sound frequency and laterality by units in central nucleus of cat inferior colliculus. , 1979, Journal of neurophysiology.

[11]  T. Irino,et al.  A time-domain, level-dependent auditory filter: The gammachirp , 1997 .

[12]  D. Oliver,et al.  The central nucleus of the inferior colliculus in the cat , 1984, The Journal of comparative neurology.

[13]  Chen Chen,et al.  Precise Feature Based Time Scales and Frequency Decorrelation Lead to a Sparse Auditory Code , 2012, The Journal of Neuroscience.

[14]  A. Pouget,et al.  Variance as a Signature of Neural Computations during Decision Making , 2011, Neuron.

[15]  M. Escabí,et al.  Spectrotemporal sound preferences of neighboring inferior colliculus neurons: implications for local circuitry and processing , 2012, Front. Neural Circuits.

[16]  J. Schnupp,et al.  Periodotopy in the gerbil inferior colliculus: local clustering rather than a gradient map , 2015, Front. Neural Circuits.

[17]  M. Kilgard,et al.  Cortical activity patterns predict speech discrimination ability , 2008, Nature Neuroscience.

[18]  Philip H Smith,et al.  Coincidence Detection in the Auditory System 50 Years after Jeffress , 1998, Neuron.

[19]  T. Houtgast,et al.  Predicting speech intelligibility in rooms from the modulation transfer function, I. General room acoustics , 1980 .

[20]  Josh H. McDermott,et al.  Adaptive and Selective Time Averaging of Auditory Scenes , 2018, Current Biology.

[21]  M M Merzenich,et al.  Representation of the cochlea within the inferior colliculus of the cat. , 1974, Brain research.

[22]  Eero P. Simoncelli,et al.  Article Sound Texture Perception via Statistics of the Auditory Periphery: Evidence from Sound Synthesis , 2022 .

[23]  Maria N. Geffen,et al.  Category-Specific Processing of Scale-Invariant Sounds in Infancy , 2014, PloS one.

[24]  H. Steven Colburn,et al.  Role of spectral detail in sound-source localization , 1998, Nature.

[25]  Neil C. Rabinowitz,et al.  Constructing Noise-Invariant Representations of Sound in the Auditory Pathway , 2013, PLoS biology.

[26]  B. B. Bauer,et al.  Fundamentals of acoustics , 1963 .

[27]  Lee M. Miller,et al.  Stimulus-Based State Control in the Thalamocortical System , 2000, The Journal of Neuroscience.

[28]  L A JEFFRESS,et al.  A place theory of sound localization. , 1948, Journal of comparative and physiological psychology.

[29]  Israel Nelken,et al.  Responses of auditory-cortex neurons to structural features of natural sounds , 1999, Nature.

[30]  N. C. Singh,et al.  Modulation spectra of natural sounds and ethological theories of auditory processing. , 2003, The Journal of the Acoustical Society of America.

[31]  M. Escabí,et al.  Spectral and temporal modulation tradeoff in the inferior colliculus. , 2010, Journal of neurophysiology.

[32]  Andrew J Oxenham,et al.  Correct tonotopic representation is necessary for complex pitch perception. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[33]  M. Konishi,et al.  Emergence of multiplicative auditory responses in the midbrain of the barn owl. , 2007, Journal of neurophysiology.

[34]  Rajiv Narayan,et al.  Distinct time scales in cortical discrimination of natural sounds in songbirds. , 2006, Journal of neurophysiology.

[35]  deCharms Rc,et al.  Information coding in the cortex by independent or coordinated populations. , 1998 .

[36]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[37]  E. C. Cherry,et al.  Mechanism of Binaural Fusion in the Hearing of Speech , 1957 .

[38]  Zachary M. Smith,et al.  Chimaeric sounds reveal dichotomies in auditory perception , 2002, Nature.

[39]  Isaac M. Carruthers,et al.  Gain Control in the Auditory Cortex Evoked by Changing Temporal Correlation of Sounds , 2016, Cerebral cortex.

[40]  Lonneke B. M. Eeuwes,et al.  Efficient Encoding of Vocalizations in the Auditory Midbrain , 2010, The Journal of Neuroscience.

[41]  Lee M. Miller,et al.  Naturalistic Auditory Contrast Improves Spectrotemporal Coding in the Cat Inferior Colliculus , 2003, The Journal of Neuroscience.

[42]  C E Schreiner,et al.  Neural processing of amplitude-modulated sounds. , 2004, Physiological reviews.

[43]  S. S. Stevens,et al.  Critical Band Width in Loudness Summation , 1957 .

[44]  R. M. Roark,et al.  B-spline design of maximally flat and prolate spheroidal-type FIR filters , 1999, IEEE Trans. Signal Process..

[45]  M. Cohen,et al.  Measuring and interpreting neuronal correlations , 2011, Nature Neuroscience.

[46]  C E Schreiner,et al.  Modular organization of intrinsic connections associated with spectral tuning in cat auditory cortex , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[47]  G. Langner,et al.  Neural processing and representation of periodicity pitch. , 1997, Acta oto-laryngologica. Supplementum.

[48]  Nathan C Higgins,et al.  Thalamic label patterns suggest primary and ventral auditory fields are distinct core regions , 2010, The Journal of comparative neurology.

[49]  Monty A. Escabí,et al.  Origins of scale invariance in vocalization sequences and speech , 2018, PLoS Comput. Biol..

[50]  T. Sejnowski,et al.  Correlated neuronal activity and the flow of neural information , 2001, Nature Reviews Neuroscience.

[51]  Sarah M. N. Woolley,et al.  Modulation Power and Phase Spectrum of Natural Sounds Enhance Neural Encoding Performed by Single Auditory Neurons , 2004, The Journal of Neuroscience.

[52]  Steven Greenberg,et al.  Speaking in shorthand - A syllable-centric perspective for understanding pronunciation variation , 1999, Speech Commun..

[53]  J. Maunsell,et al.  Attention improves performance primarily by reducing interneuronal correlations , 2009, Nature Neuroscience.

[54]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[55]  S Shamma,et al.  The case of the missing pitch templates: how harmonic templates emerge in the early auditory system. , 2000, The Journal of the Acoustical Society of America.

[56]  R. Shannon,et al.  Recognition of spectrally degraded and frequency-shifted vowels in acoustic and electric hearing. , 1999, The Journal of the Acoustical Society of America.

[57]  Joerg F. Hipp,et al.  Time-Frequency Analysis , 2014, Encyclopedia of Computational Neuroscience.

[58]  Judit Gervain,et al.  Auditory Perception of Self-Similarity in Water Sounds , 2011, Front. Integr. Neurosci..

[59]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[60]  Gal Chechik,et al.  Reduction of Information Redundancy in the Ascending Auditory Pathway , 2006, Neuron.

[61]  Julie E. Elie,et al.  Neural processing of natural sounds , 2014, Nature Reviews Neuroscience.

[62]  M. Escabí,et al.  Distinct Roles for Onset and Sustained Activity in the Neuronal Code for Temporal Periodicity and Acoustic Envelope Shape , 2008, The Journal of Neuroscience.

[63]  Eero P. Simoncelli,et al.  Summary statistics in auditory perception , 2013, Nature Neuroscience.