The senses and virtual environments

Abstract Designers and developers of immersive 3D virtual environments typically aim to faithfully recreate real-world scenarios yet, traditional emphasis is placed on recreating the visual, and to a much lesser extent, auditory scenes, while ignoring the other senses despite their importance in the real world. However, simulation all of the senses in real-time with a high level of fidelity is still not feasible and recent work has shown that high fidelity reproduction of the real world doesn’t necessarily lead to the greatest user experience and outcomes. In the real world, senses interact with one another and alter each other’s processing and ultimately perception. An understanding of this multi-modal interactivity can, therefore, inform our development of virtual environments. Through perceptual-based rendering, we can exploit multimodal effects to reduce computational requirements while creating more compelling scenarios that take advantage of current technology. In this article, we provide a brief review of fidelity, multimodal interactions and perceptual-based rendering in the virtual domain. We emphasize the perception of reality in the virtual world is subjective and not necessarily a reproduction of reality, but rather a combination of personal experiences including those pertaining to media consumption, which can ultimately lead to better user experiences.