Presence studies as an evaluation method for user experiences in multimodal virtual environments

In this paper, we propose a methodology to design and evaluate environmental sounds for virtual environments. We propose to combine physically modeled sound events with recorded soundscapes. Physical models are used to provide feedback to users’ actions, while soundscapes reproduce the characteristic soundmarks of an environment. In this particular case, physical models are used to simulate the act of walking in the botanical garden of the city of Prague, while soundscapes are used to reproduce the particular sound of the garden. The auditory feedback designed was combined with a photorealistic reproduction of the same garden. A between-subject experiment was conducted where 126 subjects participated, involving six different experimental conditions, including both uni and bi-modal stimuli (auditory and visual). The auditory stimuli consisted of several combinations of auditory feedback, including static sound sources as well as self-induced interactive sounds simulated using physical models. Results show that subjects’ motion in the environment is significantly enhanced when dynamic sound sources and sound of ego-motion are rendered in the environment.

[1]  R. T. Verrillo,et al.  Investigation of Some Parameters of the Cutaneous Threshold for Vibration , 1962 .

[2]  B. Watters.  Impact‐Noise Characteristics of Female Hard‐Heeled Foot Traffic , 1965 .

[3]  F. W. Galbraith,et al.  Ground Loading from Footsteps , 1970 .

[4]  D. Cress.  Terrain Considerations and Data Base Development for the Design and Testing of Devices to Detect Intruder-Induced Ground Motion , 1978 .

[5]  Byron Reeves,et al.  MESSAGE COMPLEXITY AND ATTENTION TO TELEVISION , 1985 .

[6]  S. Bolanowski,et al.  The effects of skin temperature on the psychophysical responses to vibration on glabrous and hairy skin. , 1986, The Journal of the Acoustical Society of America.

[7]  Sidney C. Port,et al.  Probability, Random Variables, and Stochastic Processes—Second Edition (Athanasios Papoulis) , 1986 .

[8]  Robert J. Logan,et al.  Perception of acoustic source characteristics: walking sounds. , 1991, The Journal of the Acoustical Society of America.

[9]  Barry Arons,et al.  A Review of The Cocktail Party Effect , 1992 .

[10]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[11]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[12]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[13]  E. Tan.  Film-induced affect as a witness emotion , 1995 .

[14]  Ge Wu,et al.  The effects of surface compliance on foot pressure in stance , 1996 .

[15]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[16]  R. Caussé,et al.  The representation of auditory source characteristics: simple geometric form. , 1996, Perception & psychophysics.

[17]  E. Lorch,et al.  Preschool children's attention to television: visual attention and probe response times. , 1997, Journal of experimental child psychology.

[18]  Perry R. Cook,et al.  Physically Informed Sonic Modeling (PhISM): Synthesis of percussive sounds , 1997 .

[19]  Ville Pulkki,et al.  Virtual Sound Source Positioning Using Vector Base Amplitude Panning , 1997 .

[20]  T. Maurer,et al.  A comparison of Likert scale and traditional measures of self-efficacy. , 1998 .

[21]  Barry Peterson,et al.  Quantitative Measures of Presence in Virtual Environments: The Roles of Attention and Spatial Comprehension , 1999, Cyberpsychology Behav. Soc. Netw..

[22]  Steven van de Par,et al.  Auditory-visual interaction: from fundamental research in cognitive psychology to (possible) applications , 1999, Electronic Imaging.

[23]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[24]  Mel Slater,et al.  A Virtual Presence Counter , 2000, Presence: Teleoperators & Virtual Environments.

[25]  M. Lombard,et al.  Presence and Television: The Role of Screen Size. , 2000 .

[26]  Michael Zyda,et al.  Interactions in Perceived Quality of Auditory-Visual Displays , 2000, Presence: Teleoperators & Virtual Environments.

[27]  Wijnand A. IJsselsteijn,et al.  Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli , 2000, Presence: Teleoperators & Virtual Environments.

[28]  Wijnand A. IJsselsteijn,et al.  Presence: concept, determinants, and measurement , 2000, Electronic Imaging.

[29]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[30]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[31]  Davide Rocchesso,et al.  MODELING COLLISION SOUNDS: NON-LINEAR CONTACT FORCE , 2001 .

[32]  Jonathan Freeman,et al.  Here, there and everywhere: The effects of multichannel audio on presence , 2001 .

[33]  Ville Pulkki,et al.  Spatial sound generation and perception by amplitude panning techniques , 2001 .

[34]  G. Pfurtscheller,et al.  Presence Research and EEG , 2002 .

[35]  Richard D. Sanders,et al.  The effect of sound delivery methods on a user's sense of presence in a virtual environment , 2002 .

[36]  Philip H. Marsden,et al.  Designing Auditory Spaces to Support Sense of Place: The Role of Expectation , 2002 .

[37]  Davide Rocchesso,et al.  The Sounding Object , 2002 .

[38]  Dong Pyo Jang,et al.  Analysis of Physiological Response to Two Virtual Environments: Driving and Flying Simulation , 2002, Cyberpsychology Behav. Soc. Netw..

[39]  L. Richards,et al.  Effects of age, step direction, and reaction condition on the ability to step quickly. , 2002, The journals of gerontology. Series A, Biological sciences and medical sciences.

[40]  Perry R. Cook,et al.  Modeling Bill's Gait: Analysis and Parametric Synthesis of Walking Sounds , 2002 .

[41]  S. Dane,et al.  SEX AND HANDEDNESS DIFFERENCES IN EYE-HAND VISUAL REACTION TIMES IN HANDBALL PLAYERS , 2003, The International journal of neuroscience.

[42]  Daniel Västfjäll,et al.  The Subjective Sense of Presence, Emotion Recognition, and Experienced Emotions in Auditory Virtual Environments , 2003, Cyberpsychology Behav. Soc. Netw..

[43]  Federico Fontana,et al.  Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds , 2003 .

[44]  Phil Turner,et al.  Re - Creating the Botanics: Towards a Sense of Place in Virtual Environments , 2003 .

[45]  M. Sile O'Modhrain,et al.  PebbleBox and CrumbleBag: Tactile Interfaces for Granular Synthesis , 2004, NIME.

[46]  OmniPres project IST-2001-39237 Deliverable 5 Measuring Presence : A Guide to Current Measurement Approaches , 2004 .

[47]  B. Bioulac,et al.  Age, performance and sleep deprivation , 2004, Journal of sleep research.

[48]  Bill Kapralos,et al.  Auditory Cues in the Perception of Self Motion , 2004 .

[49]  Daniel Västfjäll,et al.  Perception of Self-motion and Presence in Auditory Virtual Environments , 2004 .

[50]  S. McAdams,et al.  The psychomechanics of simulated sound sources: material properties of impacted bars. , 2004, The Journal of the Acoustical Society of America.

[51]  Daniel Västfjäll,et al.  Travelling without moving: Auditory scene cues for translational self-motion , 2005 .

[52]  Davide Rocchesso,et al.  Continuous sonic feedback from a rolling ball , 2005, IEEE MultiMedia.

[53]  Frans Mäyrä,et al.  Fundamental Components of the Gameplay Experience: Analysing Immersion , 2005, DiGRA Conference.

[54]  Eunmi Oh,et al.  Classification and identification of recorded and synthesized impact sounds by practiced listeners, musicians, and nonmusicians. , 2005, The Journal of the Acoustical Society of America.

[55]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[56]  P. Vorderer,et al.  The value of reaction-time measures for presence research: Empirical findings and future perspectives , 2005 .

[57]  J. Sabatier,et al.  Vibration and sound signatures of human footsteps in buildings. , 2006, The Journal of the Acoustical Society of America.

[58]  Rolf Nordahl.  Increasing the Motion of Users in Photo-realistic Virtual Environments by Utilising Auditory Rendering of the Environment and Ego-motion , 2006 .

[59]  S. Handel.  Perceptual Coherence: Hearing and Seeing , 2006 .

[60]  Bruno L. Giordano,et al.  Walking and playing: what's the origin of emotional expressiveness in music? , 2006 .

[61]  Atau Tanaka,et al.  Mobile Music Technology: Report on an Emerging Community , 2006, NIME.

[62]  Bruno L. Giordano,et al.  Material identification of real impact sounds: effects of size variation in steel, glass, wood, and plexiglass plates. , 2006, The Journal of the Acoustical Society of America.

[63]  I. Deary,et al.  Age and sex differences in reaction time in adulthood: results from the United Kingdom Health and Lifestyle Survey. , 2006, Psychology and aging.

[64]  Y. Visell,et al.  An Architectural Platform for Audio-Haptic Simulation in Walking , 2007 .

[65]  Peter Fröhlich,et al.  Mobile spatial interaction , 2007, CHI Extended Abstracts.

[66]  Roberto Bresin,et al.  Sound Design for Affective Interaction , 2007, ACII.

[67]  Ching-Ju Liu,et al.  Individual differences in source identification from synthesized impact sounds. , 2006, The Journal of the Acoustical Society of America.

[68]  Bruno L. Giordano,et al.  Non‐visual identification of walking grounds , 2008 .

[69]  Rolf Nordahl,et al.  On the Use of Presence Measurements to Evaluate Computer Games , 2008 .

[70]  Stephen McAdams,et al.  A Vibrotactile Device for Display of Virtual Ground Materials in Walking , 2008, EuroHaptics.

[71]  J Hollerbach,et al.  Locomotion Interfaces and Rendering , 2008 .

[72]  R. Lutfi.  Human Sound Source Identification , 2008 .

[73]  Lynette A. Jones,et al.  Tactile Displays: Guidance for Their Design and Application , 2008, Hum. Factors.

[74]  R. Pastore,et al.  Auditory event perception: The source—perception loop for posture in human gait , 2008, Perception & psychophysics.

[75]  R. Nordahl,et al.  On the use of adjustable distraction as a measure to determine sustained attention during movie clips , 2009 .

[76]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.