Reconstructing Tone Sequences from Functional Magnetic Resonance Imaging Blood-Oxygen Level Dependent Responses within Human Primary Auditory Cortex

Here we show that, using functional magnetic resonance imaging (fMRI) blood-oxygen level dependent (BOLD) responses in human primary auditory cortex, it is possible to reconstruct the sequence of tones that a person has been listening to over time. First, we characterized the tonotopic organization of each subject’s auditory cortex by measuring auditory responses to randomized pure tone stimuli and modeling the frequency tuning of each fMRI voxel as a Gaussian in log frequency space. Then, we tested our model by examining its ability to work in reverse. Auditory responses were re-collected in the same subjects, except this time they listened to sequences of frequencies taken from simple songs (e.g., “Somewhere Over the Rainbow”). By finding the frequency that minimized the difference between the model’s prediction of BOLD responses and actual BOLD responses, we were able to reconstruct tone sequences, with mean frequency estimation errors of half an octave or less, and little evidence of systematic biases.

[1]  Essa Yacoub,et al.  Representation of pitch chroma by multi-peak spectral tuning in human auditory cortex , 2015, NeuroImage.

[2]  Amir Amedi,et al.  Extensive Cochleotopic Mapping of Human Auditory Cortical Fields Obtained with Phase-Encoding fMRI , 2011, PloS one.

[3]  Katrin Krumbholz,et al.  Neuroimaging paradigms for tonotopic mapping (II): The influence of acquisition protocol , 2014, NeuroImage.

[4]  Elia Formisano,et al.  An anatomical and functional topography of human auditory cortical areas , 2014, Front. Neurosci..

[5]  Sadato Norihiro,et al.  Visual image reconstruction from human brain activity , 2009 .

[6]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[7]  Ione Fine,et al.  Population receptive field estimates of human auditory cortex , 2015, NeuroImage.

[8]  M. Schönwiesner,et al.  Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI , 2009, Proceedings of the National Academy of Sciences.

[9]  Wietske van der Zwaag,et al.  Tonotopic Gradients in Human Primary Auditory Cortex: Concurring Evidence From High-Resolution 7 T and 3 T fMRI , 2014, Brain Topography.

[10]  Essa Yacoub,et al.  Encoding of Natural Sounds at Multiple Spectral and Temporal Resolutions in the Human Auditory Cortex , 2014, PLoS Comput. Biol..

[11]  Richard S. J. Frackowiak,et al.  Human Primary Auditory Cortex Follows the Shape of Heschl's Gyrus , 2011, The Journal of Neuroscience.

[12]  R. Goebel,et al.  Mirror-Symmetric Tonotopic Maps in Human Primary Auditory Cortex , 2003, Neuron.

[13]  A. Dale,et al.  Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity. , 2004, Journal of neurophysiology.

[14]  Essa Yacoub,et al.  Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns , 2017, Proceedings of the National Academy of Sciences.

[15]  Essa Yacoub,et al.  Sensitivity and specificity considerations for fMRI encoding, decoding, and mapping of auditory cortex at ultra-high field , 2018, NeuroImage.

[16]  Colin Humphries,et al.  Tonotopic organization of human auditory cortex , 2010, NeuroImage.

[17]  Stefan Uppenkamp,et al.  Human auditory neuroimaging of intensity and loudness , 2014, Hearing Research.

[18]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[19]  Dave R. M. Langers,et al.  Tonotopic mapping of human auditory cortex , 2014, Hearing Research.

[20]  S. Dumoulin,et al.  Modeling center-surround configurations in population receptive fields using fMRI. , 2012, Journal of vision.

[21]  B. Wandell,et al.  Compressive spatial summation in human visual cortex. , 2013, Journal of neurophysiology.

[22]  Ione Fine,et al.  Minimizing biases in estimating the reorganization of human visual areas with BOLD retinotopic mapping. , 2013, Journal of vision.

[23]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[24]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[25]  Elia Formisano,et al.  Processing of Natural Sounds in Human Auditory Cortex: Tonotopy, Spectral Tuning, and Relation to Voice Sensitivity , 2012, The Journal of Neuroscience.

[26]  Katrin Krumbholz,et al.  Neuroimaging paradigms for tonotopic mapping (I): The influence of sound stimulus type , 2014, NeuroImage.

[27]  Ariel Rokem,et al.  Evaluating the Accuracy of Diffusion MRI Models in White Matter , 2015, PloS one.

[28]  Gregory Hickok,et al.  Orthogonal acoustic dimensions define auditory field maps in human cortex , 2012, Proceedings of the National Academy of Sciences.

[29]  David Temperley,et al.  A Probabilistic Model of Melody Perception , 2008, ISMIR.

[30]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[31]  Lee M. Miller,et al.  Tuning In to Sound: Frequency-Selective Attentional Filter in Human Primary Auditory Cortex , 2013, The Journal of Neuroscience.

[32]  Srivatsun Sadagopan,et al.  Nonlinear Spectrotemporal Interactions Underlying Selectivity for Complex Sounds in Auditory Cortex , 2009, The Journal of Neuroscience.

[33]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[34]  P. Heil,et al.  Frequency and periodicity are represented in orthogonal maps in the human auditory cortex: evidence from magnetoencephalography , 1997, Journal of Comparative Physiology A.

[35]  R. Goebel,et al.  Processing of Natural Sounds: Characterization of Multipeak Spectral Tuning in Human Auditory Cortex , 2013, The Journal of Neuroscience.

[36]  B. Godde,et al.  A Map of Periodicity Orthogonal to Frequency Representation in the Cat Auditory Cortex , 2009, Frontiers in integrative neuroscience.

[37]  Teemu Rinne,et al.  Functional Maps of Human Auditory Cortex: Effects of Acoustic Features and Attention , 2009, PloS one.

[38]  Adrian T. Lee,et al.  fMRI of human visual cortex , 1994, Nature.

[39]  Thomas Naselaris,et al.  Resolving Ambiguities of MVPA Using Explicit Models of Representation , 2015, Trends in Cognitive Sciences.

[40]  Timothy D. Griffiths,et al.  Orthogonal representation of sound dimensions in the primate midbrain , 2011, Nature Neuroscience.

[41]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[42]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[43]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[44]  D. Temperley Probabilistic Models of Melodic Interval , 2014 .

[45]  Dimitri Van De Ville,et al.  Decoding of Emotional Information in Voice-Sensitive Cortices , 2009, Current Biology.