Sensory fusion: A neurocomputational approach

The merging of senses is one of the most impressive capacities of the brain. Understanding how this ability can be realized by biologically inspired neural circuits is of the greatest value not only in cognitive neuroscience, but also in neuroclinics (for the design of innovative rehabilitation procedures and neuroprostheses) and in many engineering problems. The present study describes two neural networks inspired by brain functioning, to simulate the main characteristics of audio-visual multisensory integration. The first, motivated by the present knowledge on the Superior Colliculus (a midbrain structure that drives reflexive movements) is based on a feedforward schema, and can simulate many properties of audio-visual integration. The second, inspired by perception in the cerebral cortex, also includes a feedback between the auditory and visual activities, and realizes a Bayesian inference, able to estimate stimuli positions and their causal structure in a near-optimal way.

[1]  M T Wallace,et al.  Mechanisms of within- and cross-modality suppression in the superior colliculus. , 1997, Journal of neurophysiology.

[2]  A. Pouget,et al.  Probabilistic brains: knowns and unknowns , 2013, Nature Neuroscience.

[3]  Mauro Ursino,et al.  A theoretical study of multisensory integration in the superior colliculus by a neural network model , 2008, Neural Networks.

[4]  Mark T. Wallace,et al.  The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus , 2002, Experimental Brain Research.

[5]  Mauro Ursino,et al.  Multisensory Bayesian Inference Depends on Synapse Maturation during Training: Theoretical Analysis and Neural Modeling Implementation , 2017, Neural Computation.

[6]  Benjamin A. Rowland,et al.  Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model , 2012, Biological Cybernetics.

[7]  Mauro Ursino,et al.  A neural network for learning the meaning of objects and words from a featural representation , 2015, Neural Networks.

[8]  Mauro Ursino,et al.  A Neural Network Model of Ventriloquism Effect and Aftereffect , 2012, PloS one.

[9]  Nadia Bolognini,et al.  Visual search improvement in hemianopic patients after audio-visual stimulation , 2005 .

[10]  Nadia Bolognini,et al.  A neurocomputational analysis of the sound-induced flash illusion , 2014, NeuroImage.

[11]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[12]  M. Wallace,et al.  Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. , 2005, Journal of neurophysiology.

[13]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[14]  M. Wallace,et al.  Unifying multisensory signals across time and space , 2004, Experimental Brain Research.