Title : Multi-Connection Pattern Analysis : Decoding the Representational 1 Content of Neural Communication 2 3 4 5 6 7

33 34 The lack of multivariate methods for decoding the representational content of 35 interregional neural communication has left it difficult to know what information is 36 represented in distributed brain circuit interactions. Here we present Multi-Connection 37 Pattern Analysis (MCPA), which works by learning mappings between the activity 38 patterns of the populations as a factor of the information being processed. These maps are 39 used to predict the activity from one neural population based on the activity from the 40 other population. Successful MCPA-based decoding indicates the involvement of 41 distributed computational processing and provides a framework for probing the 42 representational structure of the interaction. Simulations demonstrate the efficacy of 43 MCPA in realistic circumstances. Applying MCPA to fMRI data shows that interactions 44 between visual cortex regions are sensitive to information that distinguishes individual 45 natural images, suggesting that image individuation occurs through interactive 46 computation across the visual processing network. MCPA-based representational 47 similarity analyses (RSA) results support models of error coding in interactions among 48 regions of the network. Further RSA analyses relate the non-linear information 49 transformation operations between layers of a computational model (HMAX) of visual 50 processing to the information transformation between regions of the visual processing 51 network. Additionally, applying MCPA to human intracranial electrophysiological data 52 demonstrates that the interaction between occipital face area and fusiform face area 53 contains information about individual faces. Thus, MCPA can be used to assess the 54 information represented in the coupled activity of interacting neural circuits and probe the 55 underlying principles of information transformation between regions. 56 . CC-BY-NC-ND 4.0 International license peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/046441 doi: bioRxiv preprint first posted online Mar. 31, 2016;

[1]  Alumit Ishai,et al.  Let’s face it: It’s a cortical network , 2008, NeuroImage.

[2]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[3]  S. Grossberg How does a brain build a cognitive code , 1980 .

[4]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[5]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[6]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[7]  Timothy D. Verstynen,et al.  Explicating the Face Perception Network with White Matter Connectivity , 2013, PloS one.

[8]  M. Bar,et al.  Top-down predictions in the cognitive brain , 2007, Brain and Cognition.

[9]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[10]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[13]  John A. Pyles,et al.  Dynamic Encoding of Face Information in the Human Fusiform Gyrus , 2014, Nature Communications.

[14]  K. Grill-Spector,et al.  fMRI-adaptation and category selectivity in human ventral temporal cortex: regional differences across time scales. , 2010, Journal of neurophysiology.

[15]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[16]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[17]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[18]  C. Gilbert,et al.  Top-down influences on visual processing , 2013, Nature Reviews Neuroscience.

[19]  L. Leyman,et al.  The Karolinska Directed Emotional Faces: A validation study , 2008 .

[20]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[21]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[22]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[23]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[24]  M. Chun,et al.  A neuromarker of sustained attention from whole-brain functional connectivity , 2015, Nature Neuroscience.

[25]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[26]  Rebecca Saxe,et al.  A Common Neural Code for Perceived and Inferred Emotion , 2014, The Journal of Neuroscience.

[27]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.