Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.

[1]  Sanjeev Arora,et al.  A Simple but Tough-to-Beat Baseline for Sentence Embeddings , 2017, ICLR.

[2]  J. S. Guntupalli,et al.  A Model of Representational Spaces in Human Cortex , 2016, Cerebral cortex.

[3]  N. Kanwisher Functional specificity in the human brain: A window into the functional architecture of the mind , 2010, Proceedings of the National Academy of Sciences.

[4]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[5]  Yuan Chang Leong,et al.  Shared memories reveal shared structure in neural activity across individuals , 2016, Nature Neuroscience.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[8]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[9]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[10]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Daniel L. K. Yamins,et al.  A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy , 2018, Neuron.

[12]  M. Botvinick,et al.  Statistical learning of temporal community structure in the hippocampus , 2016, Hippocampus.

[13]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.

[14]  Michael Hanke,et al.  A studyforrest extension, retinotopic mapping and localization of higher visual areas , 2016, Scientific Data.

[15]  Kenneth A. Norman,et al.  Recollection, Familiarity, and Cortical Reinstatement: A Multivoxel Pattern Analysis , 2009, Neuron.

[16]  Guillermo A. Cecchi,et al.  Holographic brain: Distributed versus local activation patterns in fMRI , 2017, IBM J. Res. Dev..

[17]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[18]  Jordan M. Malof,et al.  Distributed solar photovoltaic array location and extent dataset for remote sensing object identification , 2016, Scientific Data.

[19]  Timothy T. Rogers,et al.  Connecting natural and artificial neural networks in functional brain imaging using structured sparsity , 2018, bioRxiv.

[20]  Ming Bo Cai,et al.  Facilitating open-science with realistic fMRI simulation: validation and application , 2020, PeerJ.

[21]  Theodore L. Willke,et al.  BrainIAK tutorials: User-friendly learning materials for advanced fMRI analysis , 2019, PLoS Comput. Biol..

[22]  K. Lashley,et al.  MASS ACTION IN CEREBRAL FUNCTION. , 1931, Science.

[23]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[24]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[25]  T. Rogers,et al.  The neural and computational bases of semantic cognition , 2016, Nature Reviews Neuroscience.

[26]  Sanjeev Arora,et al.  Mapping between fMRI responses to movies and their natural language annotations , 2016, NeuroImage.

[27]  Oliver Speck,et al.  A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie , 2014, Scientific Data.

[28]  Po-Hsuan Chen,et al.  A Reduced-Dimension fMRI Shared Response Model , 2015, NIPS.