Visual cues and virtual touch: Role of visual stimuli and intersensory integration in cross-modal haptic illusions and the sense of presence.

Summary Intermodal integration (sometimes referred to as intersensory integration) may be a key psychological mechanism contributing to a sense of presence in virtual environments. Sensorimotor processes associated with multimodal integration may integrate perceptual cues and motor actions into a coherent experience and relatively consistent model of objects and spaces. When the cues come from virtual environments, intermodal integration may generate a sense of presence in a coherent virtual world. Because stimuli from virtual environments frequently fail to provide coherent and consistent cues, evidence of the role of intermodal integration might be found in intersensory illusions, the results of the user’s attempt to integrate an inconsistent environment. Secondly, if intermodal integration plays a role in the generation of presence, then intersensory illusions should be correlated with the illusion of presence in a coherent virtual world.

[1]  D. Cross,et al.  The relation between size and apparent heaviness , 1975 .

[2]  R. Welch.  Perceptual modification : adapting to altered sensory environments / Robert B. Welch , 1978 .

[3]  L. Marks.  The Unity of the Senses , 1978 .

[4]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[5]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[6]  R. Cytowič.  Synesthesia: A Union of the Senses , 1989 .

[7]  B. Stein,et al.  The Merging of the Senses , 1993 .

[8]  S. Lederman,et al.  The role of haptic versus visual volume cues in the size-weight illusion , 1993, Perception & psychophysics.

[9]  Kees Overbeeke,et al.  Designing in virtual reality: perception-action coupling and affordances , 1995 .

[10]  Kees C. J. Overbeeke,et al.  Visual resolution and spatial performance: the trade-off between resolution and interactivity , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[11]  Frank Biocca,et al.  The Cyborg's Dilemma: Progressive Embodiment in Virtual Environments , 2006, J. Comput. Mediat. Commun..

[12]  Frank Biocca,et al.  Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays , 1998, Presence.

[13]  M. Graziano.  Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[14]  C. Spence,et al.  Visual Capture of Touch: Out-of-the-Body Experiences With Rubber Gloves , 2000, Psychological science.

[15]  Abderrahmane Kheddar,et al.  Pseudo-haptic feedback: can isometric input devices simulate force feedback? , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[16]  Frank Biocca,et al.  Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions , 2001, Presence: Teleoperators & Virtual Environments.

[17]  Jonathan Freeman,et al.  A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory , 2001, Presence: Teleoperators & Virtual Environments.