Similarity representation of pattern-information fMRI

Representational similarity analysis (RSA) is a rapidly developing multivariate platform to investigate the structure of neural activities. Similarity/dissimilarity is the core concept of RSA, realized by the construction of a representational dissimilarity matrix, that addresses the closeness/distance for each pair of research elements (e.g., one minus the correlation between the brain responses to 2 different stimuli) and in turn, constitutes a multivariate pattern as its analytic foundation. This approach is also welcome for its sensitivity in detecting subtle differences of distributed experimental effects in the brain. Importantly, RSA is not only an experimental tool but a promising data-analytical framework that can integrate cross-modal imaging signals, explore brain-behavior link, and verify computational models according to measured neural activities. RSA substantiates its integrative power by relating similarity structure in one domain (e.g., stimulus features) to that in another domain (e.g., neural activities). This review summarizes dissimilarity/similarity definition of RSA, introduces how to derive the dissimilarity structure in neural response pattern, and carry out connectivity analysis based on RSA platform. Several recent advances are highlighted, such as the extraction of across-subjects regularity, cross-validation of brain reactivity in human beings and monkeys, the incorporation of computational models and behavioral profiles into RSA. Voxel receptor field modeling, another promising multivariate tool of pattern elucidation, is presented and compared. The application of RSA is expected to surge and extend in many fields of neuroscience, computation, psychology and medicine. We also discuss the limitations of RSA and some critical questions that need to be addressed in future research.

[1]  Paul A. Viola,et al.  Multi-modal volume registration by maximization of mutual information , 1996, Medical Image Anal..

[2]  Stephen José Hanson,et al.  Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area? , 2004, NeuroImage.

[3]  Denise C. Park,et al.  Nature versus Nurture in Ventral Visual Cortex: A Functional Magnetic Resonance Imaging Study of Twins , 2007, The Journal of Neuroscience.

[4]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[5]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[6]  Diana J. N. Armbruster,et al.  Similarity between Brain Activity at Encoding and Retrieval Predicts Successful Realization of Delayed Intentions , 2012, Journal of Cognitive Neuroscience.

[7]  Mei Tian,et al.  A computational coding model for saliency detection in primary visual cortex , 2012 .

[8]  C. Granger Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[9]  Xuchu Weng,et al.  Recent developments in multivariate pattern analysis for functional MRI , 2012, Neuroscience Bulletin.

[10]  S Edelman,et al.  Representation is representation of similarities , 1996, Behavioral and Brain Sciences.

[11]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[12]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[13]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[14]  N. Kriegeskorte,et al.  Categorical, Yet Graded – Single-Image Activation Profiles of Human Category-Selective Cortical Regions , 2012, The Journal of Neuroscience.

[15]  Xia Liang,et al.  Human connectome: Structural and functional brain networks , 2010, CSB 2010.

[16]  M. Milham,et al.  The ADHD-200 Consortium: A Model to Advance the Translational Potential of Neuroimaging in Clinical Neuroscience , 2012, Front. Syst. Neurosci..

[17]  Sheng Li,et al.  Multivariate pattern analysis in functional brain imaging. , 2011, Sheng li xue bao : [Acta physiologica Sinica].

[18]  Shuzhi Sam Ge,et al.  Detection of event-related hemodynamic response to neuroactivation by dynamic modeling of brain activity , 2012, NeuroImage.

[19]  Karl J. Friston Functional and effective connectivity in neuroimaging: A synthesis , 1994 .

[20]  S. Stigler Francis Galton's Account of the Invention of Correlation , 1989 .

[21]  Karl J. Friston Modalities, Modes, and Models in Functional Neuroimaging , 2009, Science.

[22]  R. Shepard,et al.  Second-order isomorphism of internal representations: Shapes of states ☆ , 1970 .

[23]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Gary Hatfield,et al.  Representation and constraints: the inverse problem and the structure of visual space. , 2003, Acta psychologica.

[25]  T. Carlson,et al.  Patterns of Activity in the Categorical Representations of Objects , 2003 .

[26]  C. Ranganath,et al.  Prefrontal and Medial Temporal Lobe Activity at Encoding Predicts Temporal Context Memory , 2010, The Journal of Neuroscience.

[27]  Nikolaus Kriegeskorte,et al.  Pattern-information analysis: From stimulus decoding to computational-model testing , 2011, NeuroImage.

[28]  A. Zador,et al.  Neural representation and the cortical code. , 2000, Annual review of neuroscience.

[29]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[30]  R. Poldrack Inferring Mental States from Neuroimaging Data: From Reverse Inference to Large-Scale Decoding , 2011, Neuron.

[31]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[32]  N. Logothetis What we can do and what we cannot do with fMRI , 2008, Nature.

[33]  Daniel Ansari,et al.  Second revision : Supplementary Material Linking brain-wide multivoxel activation patterns to behaviour : examples from language and math , 2010 .

[34]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[35]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[36]  N. Kriegeskorte,et al.  Revealing representational content with pattern-information fMRI--an introductory guide. , 2009, Social cognitive and affective neuroscience.

[37]  Daniel L. Schwartz,et al.  Beyond Natural Numbers: Negative Number Representation in Parietal Cortex , 2012, Front. Hum. Neurosci..

[38]  J. Mumford,et al.  Greater Neural Pattern Similarity Across Repetitions Is Associated with Better Memory , 2010, Science.

[39]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[40]  C. Price,et al.  Right anterior superior temporal activation predicts auditory sentence comprehension following aphasic stroke. , 2005, Brain : a journal of neurology.

[41]  A. Ishai,et al.  Recollection- and Familiarity-Based Decisions Reflect Memory Strength , 2008, Frontiers in systems neuroscience.

[42]  M. Lemay,et al.  Modularity of motor output evoked by intraspinal microstimulation in cats. , 2004, Journal of neurophysiology.

[43]  Nikolaus Kriegeskorte,et al.  Relating Population-Code Representations between Man, Monkey, and Computational Models , 2009, Front. Neurosci..

[44]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[45]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[46]  Michael S. Pratte,et al.  Decoding patterns of human brain activity. , 2012, Annual review of psychology.

[47]  J. S. Guntupalli,et al.  The Representation of Biological Classes in the Human Brain , 2012, The Journal of Neuroscience.

[48]  Richard Granger,et al.  Categorical Speech Processing in Broca's Area: An fMRI Study Using Multivariate Pattern-Based Analysis , 2012, The Journal of Neuroscience.

[49]  James V. Haxby,et al.  Multivariate pattern analysis of fMRI: The early beginnings , 2012, NeuroImage.

[50]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[51]  M. Peelen,et al.  Supramodal Representations of Perceived Emotions in the Human Brain , 2010, The Journal of Neuroscience.

[52]  Bradford Z. Mahon,et al.  What drives the organization of object knowledge in the brain? , 2011, Trends in Cognitive Sciences.

[53]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[54]  Nikolaus Kriegeskorte,et al.  Pattern-information fMRI: New questions which it opens up and challenges which face it , 2010 .

[55]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[56]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[57]  John-Dylan Haynes,et al.  Odor quality coding and categorization in human posterior piriform cortex , 2009, Nature Neuroscience.

[58]  Bruno L. Giordano,et al.  Abstract encoding of auditory objects in cortical activity patterns. , 2013, Cerebral cortex.

[59]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[60]  Alice J. O'Toole,et al.  Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[61]  Keiji Tanaka,et al.  Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. , 2007, Journal of neurophysiology.

[62]  Rajeev D. S. Raizada,et al.  What Makes Different People's Representations Alike: Neural Similarity Space Solves the Problem of Across-subject fMRI Decoding , 2012, Journal of Cognitive Neuroscience.