Evidence for Amodal Representations after Bimodal Learning: Integration of Haptic-Visual Layouts into a Common Spatial Image

Abstract Participants learned circular layouts of six objects presented haptically or visually, then indicated the direction from a start target to an end target of the same or different modality (intramodal versus intermodal). When objects from the two modalities were learned separately, superior performance for intramodal trials indicated a cost of switching between modalities. When a bimodal layout intermixing modalities was learned, intra- and intermodal trials did not differ reliably. These findings indicate that a spatial image, independent of input modality, can be formed when inputs are spatially and temporally congruent, but not when modalities are temporally segregated in learning.

[1]  Stephen M. Kosslyn,et al.  Structure and Strategy in Image Generation , 1981, Cogn. Sci..

[2]  B. Tversky Distortions in memory for maps , 1981, Cognitive Psychology.

[3]  S Lakatos,et al.  Haptic Underestimation of Angular Extent , 1998, Perception.

[4]  Roberta L Klatzky,et al.  Spatial updating of locations specified by 3-d sound and spatial language. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[5]  W H Warren,et al.  Auditory perception of breaking and bouncing events: a case study in ecological acoustics. , 1984, Journal of experimental psychology. Human perception and performance.

[6]  R Blake,et al.  Visual form created solely from temporal structure. , 1999, Science.

[7]  Jonathan W. Kelly,et al.  Sensorimotor alignment effects in the learning environment and in novel environments. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[8]  S Appelle,et al.  Visual and haptic angle perception in the matching task. , 1971, The American journal of psychology.

[9]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[10]  R. Klatzky,et al.  Assessing auditory distance perception using perceptually directed action , 1998, Perception & psychophysics.

[11]  Roberta L. Klatzky,et al.  Functional Equivalence of Spatial Images Produced by Perception and Spatial Language , 2007 .

[12]  Marios N. Avraamides,et al.  Does Body Orientation Matter When Reasoning about Depicted or Described Scenes? , 2008, Spatial Cognition.

[13]  S. Palmer Perceptual Grouping: It's Later Than You Think , 2002 .

[14]  Roberta L. Klatzky,et al.  Encoding, learning, and spatial updating of multiple object locations specified by 3-D sound, spatial language, and vision , 2003, Experimental Brain Research.

[15]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[16]  T. McNamara,et al.  Orientation and perspective dependence in route and survey learning. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[17]  Andrew T. Woods,et al.  Visual, haptic and crossmodal recognition of scenes , 2005, Experimental Brain Research.

[18]  E. Macaluso,et al.  Multisensory spatial interactions: a window onto functional integration in the human brain , 2005, Trends in Neurosciences.

[19]  Michael Tlauka,et al.  Orientation specificity occurs in both small- and large-scale imagined routes presented as verbal descriptions. , 1999 .

[20]  Roberta L Klatzky,et al.  Functional equivalence of spatial representations derived from vision and language: evidence from allocentric judgments. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[21]  Randall Stilla,et al.  Selective visuo‐haptic processing of shape and texture , 2008, Human brain mapping.

[22]  Fiona N. Newell,et al.  Visual and haptic representations of scenes are updated with observer movement , 2005, Experimental Brain Research.

[23]  G. Miller,et al.  Language and Perception , 1976 .

[24]  R. Jackendoff Consciousness and the Computational Mind , 1987 .

[25]  P.O. Barber,et al.  Encoding Direction in Manipulatory Space and the Role of Visual Experience , 1988 .

[26]  Nicholas A Giudice,et al.  Journal of Experimental Psychology : Learning , Memory , and Cognition Functional Equivalence of Spatial Images From Touch and Vision : Evidence From Spatial Updating in Blind and Sighted Individuals , 2011 .

[27]  D H Ashmead,et al.  Contribution of listeners' approaching motion to auditory distance perception. , 1995, Journal of experimental psychology. Human perception and performance.

[28]  David J. Bryant,et al.  Representing Space in Language and Perception , 1997 .

[29]  S. Kosslyn Image and mind , 1982 .

[30]  Mark Holllns,et al.  Spatial updating in blind and sighted people , 1988, Perception & psychophysics.

[31]  E Abravanel Intersensory Integration of Selected Spatial Dimensions: Extension to an Adult Sample , 1971, Perceptual and motor skills.