Measuring shared responses across subjects using intersubject correlation

Our capacity to jointly represent information about the world underpins our social experience. By leveraging one individual’s brain activity to model another’s, we can measure shared information across brains—even in dynamic, naturalistic scenarios where an explicit response model may be unobtainable. Introducing experimental manipulations allows us to measure, for example, shared responses between speakers and listeners, or between perception and recall. In this tutorial, we develop the logic of intersubject correlation (ISC) analysis and discuss the family of neuroscientific questions that stem from this approach. We also extend this logic to spatially distributed response patterns and functional network estimation. We provide a thorough and accessible treatment of methodological considerations specific to ISC analysis, and outline best practices.

[1]  Liberty S. Hamilton,et al.  The revolution will not be controlled: natural stimuli in speech neuroscience , 2018, Language, cognition and neuroscience.

[2]  Tamara Vanderwal,et al.  Shared understanding of narratives is correlated with shared neural responses , 2019, NeuroImage.

[3]  Enrico Glerean,et al.  Sharing the social world via intersubject neural synchronisation. , 2018, Current opinion in psychology.

[4]  Uri Hasson,et al.  Propagation of information along the cortical hierarchy as a function of attention while reading and listening to stories , 2018, bioRxiv.

[5]  Lucas C. Parra,et al.  Elucidating relations between fMRI, ECoG, and EEG through a common natural stimulus , 2018, NeuroImage.

[6]  Brian S Caffo,et al.  Modular preprocessing pipelines can reintroduce artifacts into fMRI data , 2018, bioRxiv.

[7]  Maurizio Corbetta,et al.  A New Modular Brain Organization of the BOLD Signal during Natural Vision , 2018, Cerebral cortex.

[8]  Uri Hasson,et al.  Infant and Adult Brains Are Coupled to the Dynamics of Natural Communication , 2018, bioRxiv.

[9]  Satrajit S. Ghosh,et al.  FMRIPrep: a robust preprocessing pipeline for functional MRI , 2018, bioRxiv.

[10]  Feilong Ma,et al.  Reliable individual differences in fine-grained cortical functional architecture , 2018, NeuroImage.

[11]  Nicholas B. Turk-Browne,et al.  Learning naturalistic temporal structure in the posterior medial network , 2018, bioRxiv.

[12]  Christian Keysers,et al.  Where and how our brain represents the temporal structure of observed action , 2018, NeuroImage.

[13]  Marc N. Coutanche,et al.  Beyond Functional Connectivity: Investigating Networks of Multivariate Representations , 2018, Trends in Cognitive Sciences.

[14]  Lei Ai,et al.  The Variability of Neural Responses to Naturalistic Videos Change with Age and Sex , 2018, eNeuro.

[15]  Lucas C. Parra,et al.  Elucidating relations between fMRI, ECoG and EEG through a common natural stimulus , 2017, bioRxiv.

[16]  Y. Hu,et al.  Brain-to-brain synchronization across two persons predicts mutual prosociality , 2017, Social cognitive and affective neuroscience.

[17]  Uri Hasson,et al.  Amplification of local changes along the timescale processing hierarchy , 2017, Proceedings of the National Academy of Sciences.

[18]  D. Poeppel,et al.  Brain-to-Brain Synchrony Tracks Real-World Dynamic Group Interactions in the Classroom , 2017, Current Biology.

[19]  Jussi Tohka,et al.  Functional brain segmentation using inter‐subject correlation in fMRI , 2017, Human brain mapping.

[20]  Thalia Wheatley,et al.  Pupil Dilation Patterns Spontaneously Synchronize Across Individuals During Shared Attention , 2017, Journal of experimental psychology. General.

[21]  Stefania Bracci,et al.  Avoiding illusory effects in representational similarity analysis: What (not) to do with the diagonal , 2017, NeuroImage.

[22]  Elise A. Piazza,et al.  Measuring speaker–listener neural coupling with functional near infrared spectroscopy , 2016, Scientific Reports.

[23]  Paul A. Taylor,et al.  Untangling the relatedness among correlations, Part II: Inter-subject correlation group analysis through linear mixed-effects modeling , 2017, NeuroImage.

[24]  Paul A. Taylor,et al.  Is the statistic value all we should care about in neuroimaging? , 2016, NeuroImage.

[25]  Lucas C Parra,et al.  Engaging narratives evoke similar neural activity and lead to similar time perception , 2017, bioRxiv.

[26]  Uri Hasson,et al.  Same Story, Different Story , 2017, Psychological science.

[27]  Yuan Chang Leong,et al.  Shared memories reveal shared structure in neural activity across individuals , 2016, Nature Neuroscience.

[28]  Yuan Chang Leong,et al.  How We Transmit Memories to Other Brains: Constructing Shared Neural Representations Via Communication , 2016, bioRxiv.

[29]  Gang Chen,et al.  Untangling the relatedness among correlations, part I: Nonparametric approaches to inter-subject correlation analysis at the group level , 2016, NeuroImage.

[30]  Janice Chen,et al.  Dynamic reconfiguration of the default mode network during narrative comprehension , 2016, Nature Communications.

[31]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[32]  Takayuki Nozawa,et al.  Interpersonal frontopolar neural synchronization in group communication: An exploration toward fNIRS hyperscanning of natural interactions , 2016, NeuroImage.

[33]  J. Haxby,et al.  Data-driven approaches in the investigation of social perception , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  J. S. Guntupalli,et al.  A Model of Representational Spaces in Human Cortex , 2016, Cerebral cortex.

[35]  Jason J. Ki,et al.  Attention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli , 2016, The Journal of Neuroscience.

[36]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[37]  Andrea Bergmann,et al.  Statistical Parametric Mapping The Analysis Of Functional Brain Images , 2016 .

[38]  Po-Hsuan Chen,et al.  A Reduced-Dimension fMRI Shared Response Model , 2015, NIPS.

[39]  Tamara Vanderwal,et al.  Inscapes: A movie paradigm to improve compliance in functional magnetic resonance imaging , 2015, NeuroImage.

[40]  Darren Price,et al.  Idiosyncratic responding during movie-watching predicted by age differences in attentional control , 2015, Neurobiology of Aging.

[41]  Gabriel Curio,et al.  Power‐law dynamics in neuronal and behavioral data introduce spurious correlations , 2015, Human brain mapping.

[42]  Emiliano Macaluso,et al.  Time‐resolved detection of stimulus/task‐related networks, via clustering of transient intersubject synchronization , 2015, Human brain mapping.

[43]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[44]  G. Ding,et al.  Leader emergence through interpersonal neural synchronization , 2015, Proceedings of the National Academy of Sciences.

[45]  Mikko Sams,et al.  Emotional speech synchronizes brains across listeners and engages large-scale dynamic brain networks , 2014, NeuroImage.

[46]  Jussi Tohka,et al.  Post-print: Effects of spatial smoothing on inter-subject correlation based analysis of FMRI , 2014 .

[47]  D. Poeppel,et al.  Coupled neural systems underlie the production and comprehension of naturalistic narrative speech , 2014, Proceedings of the National Academy of Sciences.

[48]  Nikolaus Kriegeskorte,et al.  Unique semantic space in the brain of each beholder predicts perceived similarity , 2014, Proceedings of the National Academy of Sciences.

[49]  L. Bernardi,et al.  Spontaneous Group Synchronization of Movements and Respiratory Rhythms , 2014, PloS one.

[50]  John S. Johnson,et al.  Audience preferences are predicted by temporal reliability of neural processing , 2014, Nature Communications.

[51]  J. S. Guntupalli,et al.  Decoding neural representational spaces using multivariate pattern analysis. , 2014, Annual review of neuroscience.

[52]  L. Astolfi,et al.  Social neuroscience and hyperscanning techniques: Past, present and future , 2014, Neuroscience & Biobehavioral Reviews.

[53]  C. Honey,et al.  Temporal scaling of neural responses to compressed and dilated natural speech. , 2014, Journal of neurophysiology.

[54]  Anjali Krishnan,et al.  Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations , 2014, NeuroImage.

[55]  U. Hasson,et al.  On the Same Wavelength: Predictable Language Enhances Speaker–Listener Brain-to-Brain Synchrony in Posterior Superior Temporal Gyrus , 2014, The Journal of Neuroscience.

[56]  Li Su,et al.  A Toolbox for Representational Similarity Analysis , 2014, PLoS Comput. Biol..

[57]  Jussi Tohka,et al.  A versatile software package for inter-subject correlation based analyses of fMRI , 2014, Front. Neuroinform..

[58]  Mikko Sams,et al.  Mental Action Simulation Synchronizes Action–Observation Circuits across Individuals , 2014, The Journal of Neuroscience.

[59]  Timothy O. Laumann,et al.  Methods to detect, characterize, and remove motion artifact in resting state fMRI , 2014, NeuroImage.

[60]  C. Honey,et al.  A place for time: the spatiotemporal structure of neural dynamics during natural audition. , 2013, Journal of neurophysiology.

[61]  L. Nummenmaa,et al.  The brains of high functioning autistic individuals do not synchronize with those of others☆ , 2013, NeuroImage: Clinical.

[62]  Christopher J. Honey,et al.  Selective and Invariant Neural Responses to Spoken and Written Narratives , 2013, The Journal of Neuroscience.

[63]  K. Vogeley,et al.  Toward a second-person neuroscience 1 , 2013, Behavioral and Brain Sciences.

[64]  Jessica F. Cantlon,et al.  Neural Activity during Natural Viewing of Sesame Street Statistically Predicts Test Scores in Early Childhood , 2013, PLoS biology.

[65]  Jack L. Gallant,et al.  A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain , 2012, Neuron.

[66]  Michael W. Cole,et al.  The role of default network deactivation in cognition and disease , 2012, Trends in Cognitive Sciences.

[67]  Chaozhe Zhu,et al.  Neural Synchronization during Face-to-Face Communication , 2012, The Journal of Neuroscience.

[68]  Peter E. Keller,et al.  Learning piano melodies in visuo-motor or audio-motor training conditions and the neural correlates of their cross-modal transfer , 2012, NeuroImage.

[69]  Uri Hasson,et al.  Not Lost in Translation: Neural Responses Shared Across Languages , 2012, The Journal of Neuroscience.

[70]  D. Heeger,et al.  Slow Cortical Dynamics and the Accumulation of Information over Long Timescales , 2012, Neuron.

[71]  Martin Wolf,et al.  Between-brain connectivity during imitation measured by fNIRS , 2012, NeuroImage.

[72]  Christopher J. Honey,et al.  Loss of reliable temporal structure in event-related averaging of naturalistic stimuli , 2012, NeuroImage.

[73]  F. Scholkmann,et al.  Between-brain coherence during joint n-back task performance: A two-person functional near-infrared spectroscopy study , 2012, Behavioural Brain Research.

[74]  Christopher J. Honey,et al.  Future trends in Neuroimaging: Neural processes as expressed within real-life contexts , 2012, NeuroImage.

[75]  Jukka-Pekka Kauppi,et al.  Inter-Subject Correlation in fMRI: Method Validation against Stimulus-Model Based Analysis , 2012, PloS one.

[76]  Thomas E. Nichols Multiple testing corrections, nonparametric methods, and random field theory , 2012, NeuroImage.

[77]  Mikko Sams,et al.  Functional Magnetic Resonance Imaging Phase Synchronization as a Measure of Dynamic Functional Connectivity , 2012, Brain Connect..

[78]  R. Hari,et al.  Emotions promote social interaction by synchronizing brain activity across individuals , 2012, Proceedings of the National Academy of Sciences.

[79]  Jouko Lampinen,et al.  Stimulus-Related Independent Component and Voxel-Wise Analysis of Human Brain Activity during Free Viewing of a Feature Film , 2012, PloS one.

[80]  Rajeev D. S. Raizada,et al.  What Makes Different People's Representations Alike: Neural Similarity Space Solves the Problem of Across-subject fMRI Decoding , 2012, Journal of Cognitive Neuroscience.

[81]  L. Parra,et al.  Human Neuroscience Original Research Article Correlated Components of Ongoing Eeg Point to Emotionally Laden Attention – a Possible Marker of Engagement? , 2022 .

[82]  M. Corbetta,et al.  Inter-species activity correlations reveal functional correspondences between monkey and human brain areas , 2012, Nature Methods.

[83]  S. Garrod,et al.  Brain-to-brain coupling: a mechanism for creating and sharing a social world , 2012, Trends in Cognitive Sciences.

[84]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[85]  Xu Cui,et al.  NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation , 2012, NeuroImage.

[86]  Uri Hasson,et al.  Temporal eye movement strategies during naturalistic viewing. , 2012, Journal of vision.

[87]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.

[88]  Thomas E. Nichols,et al.  Handbook of Functional MRI Data Analysis: Index , 2011 .

[89]  Guillaume Dumas,et al.  Towards a two-body neuroscience , 2011, Communicative & integrative biology.

[90]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[91]  D. Louis Collins,et al.  Unbiased average age-appropriate atlases for pediatric studies , 2011, NeuroImage.

[92]  Thomas E. Nichols,et al.  Handbook of Functional MRI Data Analysis: Index , 2011 .

[93]  Masamichi J. Hayashi,et al.  “Stay Tuned”: Inter-Individual Neural Synchronization During Mutual Gaze and Joint Attention , 2010, Front. Integr. Neurosci..

[94]  Mikko Sams,et al.  Clustering inter-subject correlation matrices in functional magnetic resonance imaging , 2010, Proceedings of the 10th IEEE International Conference on Information Technology and Applications in Biomedicine.

[95]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[96]  Line Garnero,et al.  Inter-Brain Synchronization during Social Interaction , 2010, PloS one.

[97]  U. Hasson,et al.  Speaker–listener neural coupling underlies successful communication , 2010, Proceedings of the National Academy of Sciences.

[98]  Marleen B. Schippers,et al.  Mapping the information flow from one brain to another during gestural communication , 2010, Proceedings of the National Academy of Sciences.

[99]  Asif A. Ghazanfar,et al.  Human-Monkey Gaze Correlations Reveal Convergent and Divergent Patterns of Movie Viewing , 2010, Current Biology.

[100]  D. Heeger,et al.  Reliability of cortical activity during natural stimulation , 2010, Trends in Cognitive Sciences.

[101]  Mikko Sams,et al.  Inter-Subject Correlation of Brain Hemodynamic Responses During Watching a Movie: Localization in Space and Frequency , 2009, Front. Neuroinform..

[102]  Uri Hasson,et al.  Shared and idiosyncratic cortical activation patterns in autism revealed under continuous real‐life viewing conditions , 2009, Autism research : official journal of the International Society for Autism Research.

[103]  K. Ochsner,et al.  The Need for a Cognitive Neuroscience of Naturalistic Social Cognition , 2009, Annals of the New York Academy of Sciences.

[104]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[105]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[106]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[107]  M. Sams,et al.  Inter-Subject Synchronization of Prefrontal Cortex Hemodynamic Activity During Natural Viewing , 2008, The open neuroimaging journal.

[108]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[109]  L. Davachi,et al.  Enhanced Intersubject Correlations during Movie Viewing Correlate with Successful Episodic Encoding , 2008, Neuron.

[110]  M. Onozuka,et al.  Novel trends in brain science , 2008 .

[111]  Christopher W. Tyler,et al.  Spectral analysis of fMRI signal and noise , 2008 .

[112]  Istvan Molnar-Szakacs,et al.  Beyond superior temporal cortex: intersubject correlations in narrative speech comprehension. , 2008, Cerebral cortex.

[113]  Vince D Calhoun,et al.  Interparticipant correlations: A model free FMRI analysis technique , 2007, Human brain mapping.

[114]  Thomas T. Liu,et al.  A component based noise correction method (CompCor) for BOLD and perfusion based fMRI , 2007, NeuroImage.

[115]  Rafael Malach,et al.  Extrinsic and intrinsic systems in the posterior cortex of the human brain revealed during natural sensory stimulation. , 2007, Cerebral cortex.

[116]  E. Saltzman,et al.  Action Representation of Sound: Audiomotor Recognition Network While Listening to Newly Acquired Actions , 2007, The Journal of Neuroscience.

[117]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[118]  Karl J. Friston,et al.  CHAPTER 2 – Statistical parametric mapping , 2007 .

[119]  E. Sell [Functional magnetic resonance]. , 2007, Medicina.

[120]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[121]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[122]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[123]  I. Fried,et al.  Coupling Between Neuronal Firing, Field Potentials, and fMRI in Human Auditory Cortex , 2005, Science.

[124]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[125]  Mark D'Esposito,et al.  Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses , 2004, NeuroImage.

[126]  R. Malach,et al.  Intersubject Synchronization of Cortical Activity During Natural Vision , 2004, Science.

[127]  S. Zeki,et al.  Functional brain mapping during free viewing of natural scenes , 2004, Human brain mapping.

[128]  Thomas E. Nichols,et al.  Controlling the familywise error rate in functional neuroimaging: a comparative review , 2003, Statistical methods in medical research.

[129]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.

[130]  Mukesh Dhamala,et al.  Hyperscanning : Simultaneous fMRI during Linked Social Interactions , 2001 .

[131]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[132]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[133]  P. Bandettini,et al.  Spatial Heterogeneity of the Nonlinear Dynamics in the FMRI BOLD Response , 2001, NeuroImage.

[134]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[135]  D. Bonett,et al.  Sample size requirements for estimating pearson, kendall and spearman correlations , 2000 .

[136]  D. Dacey Parallel pathways for spectral coding in primate retina. , 2000, Annual review of neuroscience.

[137]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[138]  Karl J. Friston,et al.  Characterizing Stimulus–Response Functions Using Nonlinear Regressors in Parametric fMRI Experiments , 1998, NeuroImage.

[139]  R. Turner,et al.  Event-Related fMRI: Characterizing Differential Responses , 1998, NeuroImage.

[140]  Mark S. Cohen,et al.  Parametric Analysis of fMRI Data Using Linear Systems Methods , 1997, NeuroImage.

[141]  A. Dale,et al.  Selective averaging of rapidly presented individual trials using fMRI , 1997, Human brain mapping.

[142]  Karl J. Friston,et al.  The Trouble with Cognitive Subtraction , 1996, NeuroImage.

[143]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[144]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[145]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[146]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[147]  Karl J. Friston Functional and effective connectivity in neuroimaging: A synthesis , 1994 .

[148]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[149]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[150]  Susan R. Wilson,et al.  Two guidelines for bootstrap hypothesis testing , 1991 .

[151]  N. C. Silver,et al.  Averaging Correlation Coefficients: Should Fishers z Transformation Be Used? , 1987 .

[152]  R. Fisher 014: On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. , 1921 .

[153]  R. Fisher FREQUENCY DISTRIBUTION OF THE VALUES OF THE CORRELATION COEFFIENTS IN SAMPLES FROM AN INDEFINITELY LARGE POPU;ATION , 1915 .