Sensory Contributions to Spatial Knowledge of Real and Virtual Environments

Most sensory systems are able to inform people about the spatial structure of their environment, their place in that environment, and their movement through it. We discuss these various sources of sensory information by dividing them into three general categories: external (vision, audition, somatosensory), internal (vestibular, kinesthetic) and efferent (efference copy, attention). Research on the roles of these sensory systems in the creation of environmental knowledge has shown, with few exceptions, that information from a single sensory modality is often sufficient for acquiring at least rudimentary knowledge of one’s immediate environment and one’s movement through it. After briefly discussing the ways in which sources of sensory information commonly covary in everyday life, we examine the types and quality of sensory information available from contemporary virtual environments, including desktop, CAVE, and HMD-based systems. Because none of these computer mediated systems is yet able to present a perfectly full and veridical sensory experience to its user, it is important for researchers and VE developers to understand the circumstances, tasks, and goals for which different sensory information sources are most critical. We review research on these topics, as well as research on how the omission, limitation, or distortion of different information sources may affect the perception and behavior of users. Finally, we discuss situations in which various types of virtual environment systems may be more or less useful.