The Prediction of Impact of a Looming Stimulus onto the Body Is Subserved by Multisensory Integration Mechanisms

In the jungle, survival is highly correlated with the ability to detect and distinguish between an approaching predator and a putative prey. From an ecological perspective, a predator rapidly approaching its prey is a stronger cue for flight than a slowly moving predator. In the present study, we use functional magnetic resonance imaging in the nonhuman primate, to investigate the neural bases of the prediction of an impact to the body by a looming stimulus, i.e., the neural bases of the interaction between a dynamic visual stimulus approaching the body and its expected consequences onto an independent sensory modality, namely, touch. We identify a core cortical network of occipital, parietal, premotor, and prefrontal areas maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus on the faces compared with the activations observed for spatially or temporally incongruent tactile and dynamic visual cues. These activations reflect both an active integration of visual and tactile information and of spatial and temporal prediction information. The identified cortical network coincides with a well described multisensory visuotactile convergence and integration network suggested to play a key role in the definition of peripersonal space. These observations are discussed in the context of multisensory integration and spatial, temporal prediction and Bayesian causal inference. SIGNIFICANCE STATEMENT Looming stimuli have a particular ecological relevance as they are expected to come into contact with the body, evoking touch or pain sensations and possibly triggering an approach or escape behavior depending on their identity. Here, we identify the nonhuman primate functional network that is maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus. Our findings suggest that the integration of spatial and temporal predictive cues possibly rely on the same neural mechanisms that are involved in multisensory integration.

[1]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[2]  Tirin Moore,et al.  Selection and Maintenance of Spatial Information by Frontal Eye Field Neurons , 2009, The Journal of Neuroscience.

[3]  Suliann Ben Hamed,et al.  fMRI Cortical Correlates of Spontaneous Eye Blinks in the Nonhuman Primate. , 2015, Cerebral cortex.

[4]  Gregor Thut,et al.  Selective integration of auditory-visual looming cues by humans , 2009, Neuropsychologia.

[5]  Frank Bremmer,et al.  The Representation of Movement in Near Extra-Personal Space in the Macaque Ventral Intraparietal Area (VIP) , 1997 .

[6]  Sander Nieuwenhuis,et al.  Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory , 2011, Journal of Cognitive Neuroscience.

[7]  Heiko Hecht,et al.  Threatening pictures induce shortened time-to-contact estimates , 2012, Attention, perception & psychophysics.

[8]  Charles G. Gross,et al.  REVIEW ■ : Multiple Representations of Space in the Brain , 1995 .

[9]  Henry Kennedy,et al.  Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness , 2004, Cognitive, affective & behavioral neuroscience.

[10]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[11]  Stella F. Lourenco,et al.  Threat modulates perception of looming visual stimuli , 2012, Current Biology.

[12]  Dylan F. Cooke,et al.  Sensorimotor integration in the precentral gyrus: polysensory neurons and defensive movements. , 2004, Journal of neurophysiology.

[13]  Wim Vanduffel,et al.  The Retinotopic Organization of Macaque Occipitotemporal Cortex Anterior to V4 and Caudoventral to the Middle Temporal (MT) Cluster , 2014, The Journal of Neuroscience.

[14]  Timothy E. J. Behrens,et al.  Dissociable effects of surprise and model update in parietal and anterior cingulate cortex , 2013, Proceedings of the National Academy of Sciences.

[15]  M. Sereno,et al.  A human parietal face area contains aligned head-centered visual and tactile maps , 2006, Nature Neuroscience.

[16]  Craig E. L. Stark,et al.  When zero is not zero: The problem of ambiguous baseline conditions in fMRI , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[17]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: anatomic location and visual response properties. , 1993, Journal of neurophysiology.

[18]  R. E. Yoss,et al.  Pupil size and spontaneous pupillary waves associated with alertness, drowsiness, and sleep , 1970, Neurology.

[19]  Frank Bremmer,et al.  ã Federation of European Neuroscience Societies Heading encoding in the macaque ventral intraparietal area (VIP) , 2022 .

[20]  Laura Leuchs,et al.  Spontaneous pupil dilations during the resting state are associated with activation of the salience network , 2016, NeuroImage.

[21]  Justine C. Cléry,et al.  Neuronal bases of peripersonal and extrapersonal spaces, their plasticity and their dynamics: Knowns and unknowns , 2015, Neuropsychologia.

[22]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[23]  Angela J. Yu.  Change is in the eye of the beholder , 2012, Nature Neuroscience.

[24]  Justine Cléry,et al.  Whole brain mapping of visual and tactile convergence in the macaque monkey , 2015, NeuroImage.

[25]  Geert Crombez,et al.  What’s Coming Near? The Influence of Dynamical Visual Stimuli on Nociceptive Processing , 2016, PloS one.

[26]  M. Ernst,et al.  When Correlation Implies Causation in Multisensory Integration , 2012, Current Biology.

[27]  Marzio Gerbella,et al.  Multimodal architectonic subdivision of the rostral part (area F5) of the macaque ventral premotor cortex , 2009, The Journal of comparative neurology.

[28]  W. Ball,et al.  Infant Responses to Impending Collision: Optical and Real , 1971, Science.

[29]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[30]  Joanna E. Lewis,et al.  Fixation Not Required: Characterizing Oculomotor Attention Capture for Looming Stimuli , 2015, Attention, perception & psychophysics.

[31]  S. Sara,et al.  Network reset: a simplified overarching theory of locus coeruleus noradrenaline function , 2005, Trends in Neurosciences.

[32]  R. Desimone,et al.  High-Frequency, Long-Range Coupling Between Prefrontal and Visual Cortex During Attention , 2009, Science.

[33]  J. Duhamel,et al.  Differential Dynamics of Spatial Attention, Position, and Color Coding within the Parietofrontal Network , 2015, The Journal of Neuroscience.

[34]  S. Sara.  The locus coeruleus and noradrenergic modulation of cognition , 2009, Nature Reviews Neuroscience.

[35]  Etienne Olivier,et al.  A Deficit in Covert Attention after Parietal Cortex Inactivation in the Monkey , 2004, Neuron.

[36]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[37]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[38]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[39]  Murray Mm,et al.  The Use of fMRI to Assess Multisensory Integration -- The Neural Bases of Multisensory Processes , 2012 .

[40]  Ryan A. Stevenson,et al.  The Use of fMRI to Assess Multisensory Integration , 2012 .

[41]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[42]  S. Sirois,et al.  Pupillometry , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[43]  Steven E. Petersen,et al.  The mixed block/event-related design , 2012, NeuroImage.

[44]  V. Ekroll,et al.  Partial modal completion under occlusion: what do modal and amodal percepts represent? , 2015, Journal of vision.

[45]  D. Schacter,et al.  The Brain's Default Network , 2008, Annals of the New York Academy of Sciences.

[46]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[47]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[48]  Uta Noppeney,et al.  Temporal prediction errors in visual and auditory cortices , 2014, Current Biology.

[49]  J. A. Frost,et al.  Conceptual Processing during the Conscious Resting State: A Functional MRI Study , 1999, Journal of Cognitive Neuroscience.

[50]  Etienne Olivier,et al.  Multisensory integration in multiple reference frames in the posterior parietal cortex , 2004, Cognitive Processing.

[51]  Asif A Ghazanfar,et al.  Multisensory Integration of Looming Signals by Rhesus Monkeys , 2004, Neuron.

[52]  Alan Cowey,et al.  Defensive responses to looming visual stimuli in monkeys with unilateral striate cortex ablation , 1992, Neuropsychologia.

[53]  J. Bradshaw,et al.  Pupil Size as a Measure of Arousal during Information Processing , 1967, Nature.

[54]  Luc H. Arnal,et al.  Transitions in neural oscillations reflect prediction errors generated in audiovisual speech , 2011, Nature Neuroscience.

[55]  Stefan Van der Stigchel,et al.  Approaching threat modulates visuotactile interactions in peripersonal space , 2016, Experimental Brain Research.

[56]  G. Aston-Jones,et al.  Locus coeruleus activity in monkey: Phasic and tonic changes are associated with altered vigilance , 1994, Brain Research Bulletin.

[57]  R. O’Connell,et al.  Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans. , 2011, Psychophysiology.

[58]  M. Petrides,et al.  Orofacial somatomotor responses in the macaque monkey homologue of Broca's area , 2005, Nature.

[59]  S. Ben Hamed,et al.  Representation of the visual field in the lateral intraparietal area of macaque monkeys: a quantitative receptive field analysis , 2001, Experimental Brain Research.

[60]  A. Serino,et al.  Dynamic Sounds Capture the Boundaries of Peripersonal Space Representation in Humans , 2012, PloS one.

[61]  M. Koss,et al.  Pupillary dilation as an index of central nervous system alpha 2-adrenoceptor activation. , 1986, Journal of pharmacological methods.

[62]  Dylan F. Cooke,et al.  Parieto-frontal interactions, personal space, and defensive behavior , 2006, Neuropsychologia.

[63]  Suliann Ben Hamed,et al.  A Functional Hierarchy within the Parietofrontal Network in Stimulus Selection and Attention Control , 2013, The Journal of Neuroscience.

[64]  Amélie J. Reynaud,et al.  Boosting Norepinephrine Transmission Triggers Flexible Reconfiguration of Brain Networks at Rest , 2016, Cerebral cortex.

[65]  Hans-Jochen Heinze,et al.  Neural basis of multisensory looming signals , 2013, NeuroImage.

[66]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[67]  M. Bradley,et al.  The pupil as a measure of emotional arousal and autonomic activation. , 2008, Psychophysiology.

[68]  G. DeAngelis,et al.  How Can Single Sensory Neurons Predict Behavior? , 2015, Neuron.

[69]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[70]  James A. Caviness,et al.  Persistent Fear Responses in Rhesus Monkeys to the Optical Stimulus of "Looming" , 1962, Science.

[71]  C. Gross,et al.  Coding of visual space by premotor neurons. , 1994, Science.

[72]  G. Orban,et al.  Visual Motion Processing Investigated Using Contrast Agent-Enhanced fMRI in Awake Behaving Monkeys , 2001, Neuron.

[73]  D. Heeger,et al.  In this issue , 2002, Nature Reviews Drug Discovery.

[74]  A. Dale,et al.  Selective averaging of rapidly presented individual trials using fMRI , 1997, Human brain mapping.

[75]  Ulrik R. Beierholm,et al.  Probability Matching as a Computational Strategy Used in Perception , 2010, PLoS Comput. Biol..

[76]  Uta Noppeney,et al.  Sensory reliability shapes perceptual inference via two mechanisms. , 2015, Journal of vision.

[77]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[78]  U. Noppeney,et al.  Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception , 2015, PLoS biology.

[79]  E. Szabadi,et al.  Functional Neuroanatomy of the Noradrenergic Locus Coeruleus: Its Roles in the Regulation of Arousal and Autonomic Function Part II: Physiological and Pharmacological Manipulations and Pathological Alterations of Locus Coeruleus Activity in Humans , 2008, Current neuropharmacology.

[80]  Jorge A. Santos,et al.  Keeping you at arm’s length: modifying peripersonal space influences interpersonal distance , 2017, Psychological research.

[81]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[82]  D. Simons,et al.  Moving and looming stimuli capture attention , 2003, Perception & psychophysics.

[83]  Mark S. Gilzenrat,et al.  Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function , 2010, Cognitive, affective & behavioral neuroscience.

[84]  Jeff Moher,et al.  Goal-directed action is automatically biased towards looming motion , 2015, Vision Research.

[85]  H. C. Dijkerman,et al.  Visuo-tactile interactions are dependent on the predictive value of the visual stimulus , 2015, Neuropsychologia.

[86]  M. Koss.  Pupillary dilation as an index of central nervous system α2-adrenoceptor activation , 1986 .

[87]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[88]  Jonathan D. Cohen,et al.  The effects of neural gain on attention and learning , 2013, Nature Neuroscience.

[89]  G. Rizzolatti,et al.  Coding of peripersonal space in inferior premotor cortex (area F4). , 1996, Journal of neurophysiology.

[90]  Uta Noppeney,et al.  Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices , 2016, Current Biology.

[91]  Valeria I. Petkova,et al.  Integration of visual and tactile signals from the hand in the human brain: an FMRI study. , 2011, Journal of neurophysiology.

[92]  François Klam,et al.  ã Federation of European Neuroscience Societies Visual±vestibular interactive responses in the macaque ventral intraparietal area (VIP) , 2022 .

[93]  Uta Noppeney,et al.  The contributions of transient and sustained response codes to audiovisual integration. , 2011, Cerebral cortex.

[94]  Suliann Ben Hamed,et al.  Tactile representation of the head and shoulders assessed by fMRI in the nonhuman primate. , 2016, Journal of neurophysiology.

[95]  Stephen J. Gotts,et al.  Cell-Type-Specific Synchronization of Neural Activity in FEF with V4 during Attention , 2012, Neuron.

[96]  Asif A. Ghazanfar,et al.  Integration of Bimodal Looming Signals through Neuronal Coherence in the Temporal Lobe , 2008, Current Biology.

[97]  M. Schölvinck,et al.  Tracking brain arousal fluctuations with fMRI , 2016, Proceedings of the National Academy of Sciences.

[98]  W. Einhäuser,et al.  Pupil Dilation Signals Surprise: Evidence for Noradrenaline’s Role in Decision Making , 2011, Front. Neurosci..

[99]  G Rizzolatti,et al.  The Space Around Us , 1997, Science.

[100]  F Bremmer,et al.  Stages of self-motion processing in primate posterior parietal cortex. , 2000, International review of neurobiology.

[101]  Marianne Latinus,et al.  Cerebral correlates and statistical criteria of cross-modal face and voice integration. , 2011, Seeing and perceiving.

[102]  Suliann Ben Hamed,et al.  Impact Prediction by Looming Visual Stimuli Enhances Tactile Detection , 2015, The Journal of Neuroscience.

[103]  Stephen J. Gotts,et al.  Cell-Type-Specific Synchronization of Neural Activity in FEF with V 4 during Attention , 2022 .

[104]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[105]  T. Stanford,et al.  Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness , 2009, Experimental Brain Research.

[106]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[107]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[108]  T. Takeuchi,et al.  Pupillometric evidence for the locus coeruleus-noradrenaline system facilitating attentional processing of action-triggered visual stimuli , 2015, Front. Psychol..

[109]  Christopher R Fetsch,et al.  Neural correlates of reliability-based cue weighting during multisensory integration , 2011, Nature Neuroscience.

[110]  Suliann Ben Hamed,et al.  Multimodal Convergence within the Intraparietal Sulcus of the Macaque Monkey , 2013, The Journal of Neuroscience.

[111]  M. Murray,et al.  Multisensory Integration: Flexible Use of General Operations , 2014, Neuron.

[112]  A. Toga,et al.  The Rhesus Monkey Brain in Stereotaxic Coordinates , 1999 .

[113]  H M Simpson,et al.  Pupillary Changes during a Decision-Making Task , 1969, Perceptual and motor skills.

[114]  M. Graziano,et al.  Complex Movements Evoked by Microstimulation of Precentral Cortex , 2002, Neuron.

[115]  Paul J. Laurienti,et al.  On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies , 2005, Experimental Brain Research.