Multisensory Facilitation of Behavior in Monkeys: Effects of Stimulus Intensity

Multisensory stimuli can improve performance, facilitating RTs on sensorimotor tasks. This benefit is referred to as the redundant signals effect (RSE) and can exceed predictions on the basis of probability summation, indicative of integrative processes. Although an RSE exceeding probability summation has been repeatedly observed in humans and nonprimate animals, there are scant and inconsistent data from nonhuman primates performing similar protocols. Rather, existing paradigms have instead focused on saccadic eye movements. Moreover, the extant results in monkeys leave unresolved how stimulus synchronicity and intensity impact performance. Two trained monkeys performed a simple detection task involving arm movements to auditory, visual, or synchronous auditory–visual multisensory pairs. RSEs in excess of predictions on the basis of probability summation were observed and thus forcibly follow from neural response interactions. Parametric variation of auditory stimulus intensity revealed that in both animals, RT facilitation was limited to situations where the auditory stimulus intensity was below or up to 20 dB above perceptual threshold, despite the visual stimulus always being suprathreshold. No RT facilitation or even behavioral costs were obtained with auditory intensities 30–40 dB above threshold. The present study demonstrates the feasibility and the suitability of behaving monkeys for investigating links between psychophysical and neurophysiologic instantiations of multisensory interactions.

[1]  B. Stein,et al.  Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis , 1996, Journal of Cognitive Neuroscience.

[2]  Ankoor S. Shah,et al.  Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. , 2004, Journal of neurophysiology.

[3]  Eric-Jan Wagenmakers,et al.  The Effects of Accessory Stimuli on Information Processing: Evidence from Electrophysiology and a Diffusion Model Analysis , 2009, Journal of Cognitive Neuroscience.

[4]  B. Stein,et al.  Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. , 1986, Journal of neurophysiology.

[5]  M. Corballis Interhemispheric neural summation in the absence of the corpus callosum. , 1998, Brain : a journal of neurology.

[6]  Christophe Jouffrais,et al.  Single-unit responses in the auditory cortex of monkeys performing a conditional acousticomotor task , 2003, Experimental Brain Research.

[7]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[8]  D. Raab Statistical facilitation of simple reaction times. , 1962, Transactions of the New York Academy of Sciences.

[9]  A. Diederich,et al.  Bimodal and trimodal multisensory enhancement: Effects of stimulus onset and intensity on reaction time , 2004, Perception & psychophysics.

[10]  J. Todd,et al.  Reaction To Multiple Stimuli... , 2012 .

[11]  Lizabeth M Romanski,et al.  Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. , 2007, Cerebral cortex.

[12]  Gregor Thut,et al.  Selective integration of auditory-visual looming cues by humans , 2009, Neuropsychologia.

[13]  B. Fraysse,et al.  Evidence that cochlear-implanted deaf patients are better multisensory integrators , 2007, Proceedings of the National Academy of Sciences.

[14]  Christoph Kayser,et al.  Behavioral/systems/cognitive Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2022 .

[15]  N. Holmes The Principle of Inverse Effectiveness in Multisensory Integration: Some Statistical Considerations , 2009, Brain Topography.

[16]  Jeff Miller,et al.  Locus of the redundant-signals effect in bimodal divided attention: A neurophysiological analysis , 2001, Perception & psychophysics.

[17]  A John Van Opstal,et al.  Crossmodal integration in the primate superior colliculus underlying the preparation and initiation of saccadic eye movements. , 2005, Journal of neurophysiology.

[18]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[19]  Brigitte Röder,et al.  Multisensory processing in the redundant-target effect: A behavioral and event-related potential study , 2005, Perception & psychophysics.

[20]  R S Nickerson,et al.  Intersensory facilitation of reaction time: energy summation or preparation enhancement? , 1973, Psychological review.

[21]  Adele Diederich,et al.  Visual–tactile integration: does stimulus duration influence the relative amount of response enhancement? , 2006, Experimental Brain Research.

[22]  S. Celebrini,et al.  Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence , 2008, BMC Neuroscience.

[23]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[24]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[25]  A J Van Opstal,et al.  Auditory-visual interactions subserving goal-directed saccades in a complex scene. , 2002, Journal of neurophysiology.

[26]  Y. Lamarre,et al.  Fast ballistic arm movements triggered by visual, auditory, and somesthetic stimuli in the monkey. I. Activity of precentral cortical neurons. , 1983, Journal of neurophysiology.

[27]  M. Wallace,et al.  Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. , 2005, Journal of neurophysiology.

[28]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[29]  I. Nelken,et al.  Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex , 2006, Cerebral cortex.

[30]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.

[31]  D. Raab DIVISION OF PSYCHOLOGY: STATISTICAL FACILITATION OF SIMPLE REACTION TIMES* , 1962 .

[32]  Shinsuke Shimojo,et al.  Development of multisensory spatial integration and perception in humans. , 2006, Developmental science.

[33]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[34]  A. Puce,et al.  Neuronal oscillations and visual amplification of speech , 2008, Trends in Cognitive Sciences.

[35]  P F Seitz,et al.  The use of visible speech cues for improving auditory detection of spoken sentences. , 2000, The Journal of the Acoustical Society of America.

[36]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[37]  H. Scheich,et al.  Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys , 2005, The Journal of Neuroscience.

[38]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[39]  Nadia Bolognini,et al.  Multisensory-Mediated Auditory Localization , 2006, Perception.

[40]  T. Ono,et al.  Auditory thalamus integrates visual inputs into behavioral gains , 2005, Nature Neuroscience.

[41]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[42]  W. David Hairston,et al.  Multisensory enhancement of localization under conditions of induced myopia , 2003, Experimental Brain Research.

[43]  Céline Cappe,et al.  The Thalamocortical Projection Systems in Primate: An Anatomical Support for Multisensory and Sensorimotor Interplay , 2009, Cerebral cortex.

[44]  N. Logothetis,et al.  Neuroperception: Facial expressions linked to monkey calls , 2003, Nature.

[45]  Geoffrey Hall,et al.  Animal cognition , 1985, Nature.

[46]  Nikos K. Logothetis,et al.  Eye movements of monkey observers viewing vocalizing conspecifics , 2006, Cognition.

[47]  S. Kojima,et al.  Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes) , 2004, Animal Cognition.

[48]  Micah M. Murray,et al.  Early, Low-Level Auditory-Somatosensory Multisensory Interactions Impact Reaction Time Speed , 2009, Front. Integr. Neurosci..

[49]  John J. Foxe,et al.  Oscillatory beta activity predicts response speed during a multisensory audiovisual reaction time task: a high-density electrical mapping study. , 2005, Cerebral cortex.

[50]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[51]  John J. Foxe,et al.  Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness , 2011, NeuroImage.

[52]  John J. Foxe,et al.  Impaired multisensory processing in schizophrenia: Deficits in the visual enhancement of speech comprehension under noisy environmental conditions , 2007, Schizophrenia Research.

[53]  A. Opstal,et al.  Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus , 1998, Brain Research Bulletin.

[54]  M HERSHENSON,et al.  Reaction time as a measure of intersensory facilitation. , 1962, Journal of experimental psychology.

[55]  Arnaud Falchier,et al.  Multisensory connections of monkey auditory cerebral cortex , 2009, Hearing Research.

[56]  J. Groh,et al.  Visual- and saccade-related signals in the primate inferior colliculus , 2007, Proceedings of the National Academy of Sciences.

[57]  Asif A Ghazanfar,et al.  Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. , 2009, Journal of neurophysiology.

[58]  Mark T. Wallace,et al.  The Development of Cortical Multisensory Integration , 2006, The Journal of Neuroscience.

[59]  I. Kermadi,et al.  Neuronal activity in the primate supplementary motor area and the primary motor cortex in relation to spatio-temporal bimanual coordination. , 1998, Somatosensory & motor research.

[60]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[61]  BMC Neuroscience , 2003 .

[62]  Y. Sakurai,et al.  Functional role of the secondary visual cortex in multisensory facilitation in rats , 2008, Neuroscience.

[63]  Asif A Ghazanfar,et al.  Interactions between the Superior Temporal Sulcus and Auditory Cortex Mediate Dynamic Face/Voice Integration in Rhesus Monkeys , 2008, The Journal of Neuroscience.

[64]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[65]  E. Vatikiotis-Bateson,et al.  Eye movement of perceivers during audiovisualspeech perception , 1998, Perception & psychophysics.

[66]  Gregor Thut,et al.  Preperceptual and Stimulus-Selective Enhancement of Low-Level Human Visual Cortex Excitability by Sounds , 2009, Current Biology.

[67]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[68]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[69]  Bruno B Averbeck,et al.  Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex , 2006, The Journal of Neuroscience.

[70]  E. M. Rouiller,et al.  Multisensory anatomical pathways , 2009, Hearing Research.

[71]  S. Sterbing-D’Angelo,et al.  Behavioral/systems/cognitive Multisensory Space Representations in the Macaque Ventral Intraparietal Area , 2022 .

[72]  Sylvia Weir,et al.  Action perception , 1974 .

[73]  Terrence R Stanford,et al.  Cortex Mediates Multisensory But Not Unisensory Integration in Superior Colliculus , 2007, The Journal of Neuroscience.

[74]  Jean-Philippe Thiran,et al.  Multisensory interactions within human primary cortices revealed by BOLD dynamics. , 2007, Cerebral cortex.

[75]  Stefano Targher,et al.  Audiovisual integration in low vision individuals , 2012, Neuropsychologia.

[76]  L. Merabet,et al.  Occipital Transcranial Magnetic Stimulation Has Opposing Effects on Visual and Auditory Stimulus Detection: Implications for Multisensory Interactions , 2007, The Journal of Neuroscience.

[77]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[78]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[79]  P. Reuter-Lorenz,et al.  Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. , 1994, Journal of experimental psychology. Human perception and performance.

[80]  Istvan Ulbert,et al.  Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane , 2007, The Journal of comparative neurology.

[81]  John J. Foxe,et al.  Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study , 2001, Neuropsychologia.

[82]  Asif A Ghazanfar,et al.  Multisensory Integration of Looming Signals by Rhesus Monkeys , 2004, Neuron.

[83]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[84]  J. Groh,et al.  Representation of eye position in primate inferior colliculus. , 2006, Journal of neurophysiology.

[85]  W. David Hairston,et al.  Altered temporal profile of visual–auditory multisensory interactions in dyslexia , 2005, Experimental Brain Research.

[86]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[87]  Wei Ji Ma,et al.  Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space , 2009, PloS one.

[88]  M Giray,et al.  Motor coactivation revealed by response force in divided and focused attention. , 1993, Journal of experimental psychology. Human perception and performance.

[89]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[90]  J. Groh,et al.  Eye Position Affects Activity in Primary Auditory Cortex of Primates , 2003, Current Biology.

[91]  Brian L Allman,et al.  Subthreshold multisensory processing in cat auditory cortex , 2009, Neuroreport.

[92]  M A Meredith,et al.  The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. , 2001, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[93]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[94]  N. Logothetis,et al.  Frontiers in Integrative Neuroscience Integrative Neuroscience Directed Interactions between Auditory and Superior Temporal Cortices and Their Role in Sensory Integration , 2022 .

[95]  R. Goebel,et al.  Reduced Neural Integration of Letters and Speech Sounds Links Phonological and Reading Deficits in Adult Dyslexia , 2009, Current Biology.

[96]  P. Barone,et al.  Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey , 2005, The European journal of neuroscience.

[97]  M. Wallace,et al.  Enhanced multisensory integration in older adults , 2006, Neurobiology of Aging.

[98]  E. M. Rouiller,et al.  Thalamocortical and the dual pattern of corticothalamic projections of the posterior parietal cortex in macaque monkeys , 2007, Neuroscience.

[99]  J Miller,et al.  Absence of coactivation in the motor component: evidence from psychophysiological measures of target detection. , 1996, Journal of experimental psychology. Human perception and performance.