Antagonistic cross-links and reciprocal couplings emerge in optimal multisensory integration

Information from multiple sources is exploited by the neural system to facilitate reliable and flexible information processing, in a near-optimal way as predicted by Bayes’ rule. However, the underlying network architecture achieving the optimal integration is largely unknown, especially when the prior distribution describing the multiple sensory stimuli is only partially correlated. Here, we study a decentralized architecture, in which each module is a recurrent neural network processing input from one source, and cross-talks among them can facilitate integration. To achieve inter-modular communication, each module receives direct input from other sources through the cross-links, and indirect input from the other modules through reciprocal links. Through theoretical analysis and network optimization, we investigate how multisensory likelihoods and priors are encoded in different components of the network structure. We found that the multisensory prior information is encoded in the crosstalks in a distributed manner. The most striking discovery is that the cross-links and the reciprocal couplings form an antagonistic pair and play complementary roles. Our results reveal the crucial dependence of the optimal network structure on the statistics of multisensory stimuli, especially the prior information.

[1]  Si Wu,et al.  How the Prior Information Shapes Neural Networks for Optimal Multisensory Integration , 2017, ISNN.

[2]  G. DeAngelis,et al.  Distributed Visual–Vestibular Processing in the Cerebral Cortex of Man and Macaque , 2017 .

[3]  M. Rasch,et al.  Decentralized Multisensory Information Integration in Neural Systems , 2016, The Journal of Neuroscience.

[4]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[5]  P. Bressloff.  Spatiotemporal dynamics of continuum neural fields , 2012 .

[6]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[7]  A. Pouget,et al.  Marginalization in Neural Circuits with Divisive Normalization , 2011, The Journal of Neuroscience.

[8]  S. R. Jammalamadaka,et al.  Directional Statistics, I , 2011 .

[9]  Konrad Kording,et al.  Annals of the New York Academy of Sciences Bayesian Models: the Structure of the World, Uncertainty, Behavior, and the Brain , 2022 .

[10]  Gregory C. DeAngelis,et al.  A Comparison of Vestibular Spatiotemporal Tuning in Macaque Parietoinsular Vestibular Cortex, Ventral Intraparietal Area, and Medial Superior Temporal Area , 2011, The Journal of Neuroscience.

[11]  Richard F Murray,et al.  Cue combination on the circle and the sphere. , 2010, Journal of vision.

[12]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[13]  G. DeAngelis,et al.  Multisensory integration: psychophysics, neurophysiology, and computation , 2009, Current Opinion in Neurobiology.

[14]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[15]  Si Wu,et al.  Dynamics of neural networks with continuous attractors , 2008 .

[16]  Kazuyuki Aihara,et al.  Bayesian Inference Explains Perception of Unity and Ventriloquism Aftereffect: Identification of Common Sources of Audiovisual Stimuli , 2007, Neural Computation.

[17]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[18]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[19]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[20]  K. Zhang,et al.  Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.