Encoding Multisensory Information in Modular Neural Networks

The brain is capable of integrating information in multiple sensory channels in a Bayesian optimal way. Based on a decentralized network model inspired by electrophysiological recordings, we consider the structural pre-requisites for optimal multisensory integration. In this architecture, same-channel feedforward and recurrent links encode the unisensory likelihoods, whereas reciprocal couplings connecting the different modules are shaped by the correlation in the joint prior probabilities. Moreover, the statistical relationship between the difference in the optimal network structures and the difference in the priors and the likelihoods clearly shows that the network can encode multisensory information in a distributed manner. Our results generate testable predictions for future experiments and are likely to be applicable to other artificial systems.

[1]  Si Wu,et al.  How the Prior Information Shapes Neural Networks for Optimal Multisensory Integration , 2017, ISNN.

[2]  Eero P. Simoncelli,et al.  Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics , 2011, Nature Neuroscience.

[3]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[4]  Si Wu,et al.  "Congruent" and "Opposite" Neurons: Sisters for Multisensory Integration and Segregation , 2016, NIPS.

[5]  Dora E Angelaki,et al.  Macaque Parieto-Insular Vestibular Cortex: Responses to Self-Motion and Optic Flow , 2010, Journal of Neuroscience.

[6]  Si Wu,et al.  A Moving Bump in a Continuous Manifold: A Comprehensive Study of the Tracking Dynamics of Continuous Attractor Neural Networks , 2008, Neural Computation.

[7]  Yong Gu,et al.  Evidence for a Causal Contribution of Macaque Vestibular, But Not Intraparietal, Cortex to Heading Perception , 2016, The Journal of Neuroscience.

[8]  Dora E Angelaki,et al.  Functional Specializations of the Ventral Intraparietal Area for Multisensory Heading Discrimination , 2013, The Journal of Neuroscience.

[9]  Ulrik R Beierholm,et al.  Sound-induced flash illusion as an optimal percept , 2005, Neuroreport.

[10]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[11]  M. Rasch,et al.  Decentralized Multisensory Information Integration in Neural Systems , 2016, The Journal of Neuroscience.

[12]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[13]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[14]  Kazuyuki Aihara,et al.  Bayesian Inference Explains Perception of Unity and Ventriloquism Aftereffect: Identification of Common Sources of Audiovisual Stimuli , 2007, Neural Computation.

[15]  David Alais,et al.  No direction-specific bimodal facilitation for audiovisual motion detection. , 2004, Brain research. Cognitive brain research.

[16]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[17]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[18]  Richard F Murray,et al.  Cue combination on the circle and the sphere. , 2010, Journal of vision.