Task-irrelevant sounds influence both temporal order and apparent-motion judgments about tactile stimuli applied to crossed and uncrossed hands

It has been suggested that judgments about the temporal–spatial order of successive tactile stimuli depend on the perceived direction of apparent motion between them. Here we manipulated tactile apparent-motion percepts by presenting a brief, task-irrelevant auditory stimulus temporally in-between pairs of tactile stimuli. The tactile stimuli were applied one to each hand, with varying stimulus onset asynchronies (SOAs). Participants reported the location of the first stimulus (temporal order judgments: TOJs) while adopting both crossed and uncrossed hand postures, so we could scrutinize skin-based, anatomical, and external reference frames. With crossed hands, the sound improved TOJ performance at short (≤300 ms) and at long (>300 ms) SOAs. When the hands were uncrossed, the sound induced a decrease in TOJ performance, but only at short SOAs. A second experiment confirmed that the auditory stimulus indeed modulated tactile apparent motion perception under these conditions. Perceived apparent motion directions were more ambiguous with crossed than with uncrossed hands, probably indicating competing spatial codes in the crossed posture. However, irrespective of posture, the additional sound tended to impair potentially anatomically coded motion direction discrimination at a short SOA of 80 ms, but it significantly enhanced externally coded apparent motion perception at a long SOA of 500 ms. Anatomically coded motion signals imply incorrect TOJ responses with crossed hands, but correct responses when the hands are uncrossed; externally coded motion signals always point toward the correct TOJ response. Thus, taken together, these results suggest that apparent-motion signals are likely taken into account when tactile temporal–spatial information is reconstructed.

[1]  J. Vroomen,et al.  Temporal ventriloquism: sound modulates the flash-lag effect. , 2004, Journal of experimental psychology. Human perception and performance.

[2]  Lihan Chen,et al.  Mutual influences of intermodal visual/tactile apparent motion and auditory motion with uncrossed and crossed arms. , 2013, Multisensory research.

[3]  Rolf Ulrich,et al.  Multimodal integration of time. , 2014, Experimental psychology.

[4]  Ole Jensen,et al.  Parietal Oscillations Code Nonvisual Reach Targets Relative to Gaze and Body , 2013, The Journal of Neuroscience.

[5]  S. Getzmann,et al.  Audiovisual influences on the perception of visual apparent motion: exploring the effect of a single sound. , 2008, Acta psychologica.

[6]  R. L. Knoll,et al.  The Perception of Temporal Order: Fundamental Issues and a General Model , 1973 .

[7]  Spatial references and audio-tactile interaction in cross-modal dynamic capture. , 2014, Multisensory research.

[8]  S. Soto-Faraco,et al.  Somatosensory saccades reveal the timing of tactile spatial remapping , 2011, Neuropsychologia.

[9]  J. Craig,et al.  The effect of motion on tactile and visual temporal order judgments , 2003, Perception & psychophysics.

[10]  S. Soto-Faraco,et al.  Changing Reference Frames during the Encoding of Tactile Events , 2008, Current Biology.

[11]  Alan Kingstone,et al.  Congruency effects between auditory and tactile motion: Extending the phenomenon of cross-modal dynamic capture , 2004, Cognitive, affective & behavioral neuroscience.

[12]  B. Röder,et al.  Event‐related potential evidence for a dynamic (re‐)weighting of somatotopic and external coordinates of touch during visual–tactile interactions , 2015, The European journal of neuroscience.

[13]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[14]  B. Röder,et al.  Integration of anatomical and external response mappings explains crossing effects in tactile localization: A probabilistic modeling approach , 2015, Psychonomic Bulletin & Review.

[15]  Brigitte Röder,et al.  Common Anatomical and External Coding for Hands and Feet in Tactile Attention: Evidence from Event-related Potentials , 2010, Journal of Cognitive Neuroscience.

[16]  Shinsuke Shimojo,et al.  Beyond perceptual modality: Auditory effects on visual perception , 2001 .

[17]  A. Engel,et al.  Tactile remapping: from coordinate transformation to integration in sensorimotor processing , 2015, Trends in Cognitive Sciences.

[18]  C. Spence,et al.  Confusing the mind by crossing the hands. , 2002, Brain research. Cognitive brain research.

[19]  Gene R Stoner,et al.  Auditory modulation of visual apparent motion with short spatial and temporal intervals. , 2010, Journal of vision.

[20]  M. Longo,et al.  A three-dimensional spatial characterization of the crossed-hands deficit , 2016, Cognition.

[21]  C. Spence,et al.  Back-to-front: Improved tactile discrimination performance in the space you cannot see , 2006, Neuroscience Letters.

[22]  B. Röder,et al.  Flexibly weighted integration of tactile reference frames , 2015, Neuropsychologia.

[23]  Brigitte Röder,et al.  Processing load impairs coordinate integration for the localization of touch , 2014, Attention, Perception, & Psychophysics.

[24]  S. Kitazawa,et al.  Effects of visual stimuli on temporal order judgments of unimanual finger stimuli , 2007, Experimental Brain Research.

[25]  Orit Baruch,et al.  Space and Time: An Impact of Spatial Separation, Apparent Motion, and Perceptual Grouping on TOJ Performance , 2013, Perception.

[26]  S. Kitazawa,et al.  Neural correlates of tactile temporal-order judgment in humans: an fMRI study. , 2013, Cerebral cortex.

[27]  Zhuanghua Shi,et al.  Interaction of Perceptual Grouping and Crossmodal Temporal Capture in Tactile Apparent-Motion , 2011, PloS one.

[28]  Charles Spence,et al.  Spatiotemporal interactions between audition and touch depend on hand posture , 2005, Experimental Brain Research.

[29]  J. Lewald,et al.  Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. , 2003, Brain research. Cognitive brain research.

[30]  Xiaolin Zhou,et al.  Capture of intermodal visual/tactile apparent motion by moving and static sound. , 2011, Seeing and perceiving.

[31]  B. Röder,et al.  Multiple spatial representations determine touch localization on the fingers. , 2014, Journal of experimental psychology. Human perception and performance.

[32]  Jon Driver,et al.  Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound , 2008, Current Biology.

[33]  S. Kitazawa,et al.  Effects of handedness on tactile temporal order judgment , 2004, Neuropsychologia.

[34]  Zachary P. Barnett,et al.  Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance , 2012, Experimental Brain Research.

[35]  Stephan Getzmann,et al.  The Effect of Brief Auditory Stimuli on Visual Apparent Motion , 2007, Perception.

[36]  Tobias Heed,et al.  Using time to investigate space: a review of tactile temporal order judgments as a window onto spatial processing in touch , 2014, Front. Psychol..

[37]  T. Heed,et al.  Towards explaining spatial touch perception: Weighted integration of multiple location codes , 2016, Cognitive neuropsychology.

[38]  J. Kirman,et al.  Tactile apparent movement: The effects of interstimulus onset interval and stimulus duration , 1974 .

[39]  James C Craig,et al.  The effect of hand position and pattern motion on temporal order judgments , 2003, Perception & psychophysics.

[40]  J. Craig,et al.  Temporal order and tactile patterns , 1990, Perception & psychophysics.

[41]  C. Spence,et al.  Effect of posture change on tactile perception: impaired direction discrimination performance with interleaved fingers , 2005, Experimental Brain Research.

[42]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[43]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[44]  J. Lewald,et al.  Auditory-visual temporal integration as a function of distance: no compensation for sound-transmission time in human perception , 2004, Neuroscience Letters.

[45]  S. Kitazawa,et al.  Reversal of subjective temporal order due to arm crossing , 2001, Nature Neuroscience.

[46]  M. Barnett-Cowan,et al.  Crossing the hands is more confusing for females than males , 2010, Experimental Brain Research.

[47]  Charles Spence,et al.  Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: A signal detection study , 2007, Cognition.

[48]  Zhuanghua Shi,et al.  Auditory temporal modulation of the visual Ternus effect: the influence of time interval , 2010, Experimental Brain Research.

[49]  Susumu Tachi,et al.  Directional remapping in tactile inter-finger apparent motion: a motion aftereffect study , 2011, Experimental Brain Research.

[50]  T Z Strybel,et al.  A comparison of the effects of spatial separation on apparent motion in the auditory and visual modalities , 1990, Perception & psychophysics.

[51]  S. Kitazawa,et al.  Reversal of subjective temporal order due to sensory and motor integrations , 2006 .

[52]  Ole Jensen,et al.  Multiple Reference Frames in Cortical Oscillatory Activity during Tactile Remapping for Saccades , 2011, The Journal of Neuroscience.

[53]  Charles Spence,et al.  The modulation of crossmodal integration by unimodal perceptual grouping: a visuotactile apparent motion study , 2006, Experimental Brain Research.

[54]  James C Craig,et al.  Visual Motion Interferes with Tactile Motion Perception , 2006, Perception.

[55]  Brigitte Röder,et al.  Disentangling the External Reference Frames Relevant to Tactile Localization , 2016, PloS one.

[56]  T. Heed,et al.  Reach Trajectories Characterize Tactile Localization for Sensorimotor Decision Making , 2015, The Journal of Neuroscience.