How the Prior Information Shapes Neural Networks for Optimal Multisensory Integration

Extensive studies suggest that the brain integrates multisensory signals in a Bayesian optimal way. In this work, we consider how the couplings in a neural network model are shaped by the prior information when it performs optimal multisensory integration and encodes the whole profile of the posterior. To process stimuli of two modalities, a biologically plausible neural network model consists of two modules, one for each modality, and crosstalks between the two modules are carried out through feedforward cross-links and reciprocal connections. We found that the reciprocal couplings are crucial to optimal multisensory integration in that their pattern is shaped by the correlation in the joint prior distribution of sensory stimuli. Our results show that a decentralized architecture based on reciprocal connections is able to accommodate complex correlation structures across modalities and utilize this prior information in optimal multisensory integration.

[1]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[2]  M. Murray,et al.  Multisensory Integration: Flexible Use of General Operations , 2014, Neuron.

[3]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[4]  Kazuyuki Aihara,et al.  Bayesian Inference Explains Perception of Unity and Ventriloquism Aftereffect: Identification of Common Sources of Audiovisual Stimuli , 2007, Neural Computation.

[5]  David Alais,et al.  No direction-specific bimodal facilitation for audiovisual motion detection. , 2004, Brain research. Cognitive brain research.

[6]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[7]  G. DeAngelis,et al.  A Normalization Model of Multisensory Integration , 2011, Nature Neuroscience.

[8]  S. Morad,et al.  Ceramide-orchestrated signalling in cancer cells , 2012, Nature Reviews Cancer.

[9]  Mauro Ursino,et al.  A Neural Network Model of Ventriloquism Effect and Aftereffect , 2012, PloS one.

[10]  Brian J. Fischer,et al.  Owl's behavior and neural representation predicted by Bayesian inference , 2011, Nature Neuroscience.

[11]  M. Rasch,et al.  Decentralized Multisensory Information Integration in Neural Systems , 2016, The Journal of Neuroscience.

[12]  Abe Sklar,et al.  Random variables, joint distribution functions, and copulas , 1973, Kybernetika.

[13]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[14]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[15]  H. Sebastian Seung,et al.  Learning Continuous Attractors in Recurrent Networks , 1997, NIPS.

[16]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[17]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[18]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[19]  Si Wu,et al.  A Moving Bump in a Continuous Manifold: A Comprehensive Study of the Tracking Dynamics of Continuous Attractor Neural Networks , 2008, Neural Computation.

[20]  Ulrik R Beierholm,et al.  Sound-induced flash illusion as an optimal percept , 2005, Neuroreport.

[21]  Eero P. Simoncelli,et al.  Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics , 2011, Nature Neuroscience.

[22]  C. Sempi,et al.  Principles of Copula Theory , 2015 .

[23]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.