The Plausibility of a String Quartet Performance in Virtual Reality

We describe an experiment that explores the contribution of auditory and other features to the illusion of plausibility in a virtual environment that depicts the performance of a string quartet. ‘Plausibility’ refers to the component of presence that is the illusion that the perceived events in the virtual environment are really happening. The features studied were: Gaze (the musicians ignored the participant, the musicians sometimes looked towards and followed the participant's movements), Sound Spatialization (Mono, Stereo, Spatial), Auralization (no sound reflections, reflections corresponding to a room larger than the one perceived, reflections that exactly matched the virtual room), and Environment (no sound from outside of the room, birdsong and wind corresponding to the outside scene). We adopted the methodology based on color matching theory, where 20 participants were first able to assess their feeling of plausibility in the environment with each of the four features at their highest setting. Then five times participants started from a low setting on all features and were able to make transitions from one system configuration to another until they matched their original feeling of plausibility. From these transitions a Markov transition matrix was constructed, and also probabilities of a match conditional on feature configuration. The results show that Environment and Gaze were individually the most important factors influencing the level of plausibility. The highest probability transitions were to improve Environment and Gaze, and then Auralization and Spatialization. We present this work as both a contribution to the methodology of assessing presence without questionnaires, and showing how various aspects of a musical performance can influence plausibility.

[1]  Michael Meehan,et al.  Physiological measures of presence in stressful virtual environments , 2002, SIGGRAPH.

[2]  Paul Zimmons,et al.  The influence of rendering quality on presence and task performance in a virtual environment , 2003, IEEE Virtual Reality, 2003. Proceedings..

[3]  Woodrow Barfield,et al.  Presence in virtual environments as a function of type of input device and display update rate , 1998 .

[4]  Samuel Karlin,et al.  A First Course on Stochastic Processes , 1968 .

[5]  J. Loomis,et al.  Interpersonal Distance in Immersive Virtual Environments , 2003, Personality & social psychology bulletin.

[6]  F L Wightman,et al.  Resolution of front-back ambiguity in spatial hearing by listener and source movement. , 1999, The Journal of the Acoustical Society of America.

[7]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[8]  Karsten Bormann,et al.  Presence and the Utility of Audio Spatialization , 2005, Presence: Teleoperators & Virtual Environments.

[9]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[10]  Robin Wolff,et al.  Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments , 2008, CSCW.

[11]  Mel Slater,et al.  Body Centred Interaction in Immersive Virtual Environments , 1994 .

[12]  S. Delacroix,et al.  The Responses of Medical General Practitioners to Unreasonable Patient Demand for Antibiotics - A Study of Medical Ethics Using Immersive Virtual Reality , 2016, PloS one.

[13]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.

[14]  Lawrence W. Stark,et al.  The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[15]  Nicola Döring,et al.  Integration of spatial sound in immersive virtual environments an experimental study on effects of spatial sound on presence , 2013, 2013 IEEE Virtual Reality (VR).

[16]  M. Slater,et al.  Bystander Responses to a Violent Incident in an Immersive Virtual Environment , 2013, PloS one.

[17]  Woodrow Barfield,et al.  Presence as a Function of Update Rate Within Virtual Environments , 1995 .

[18]  Eric Kolstad,et al.  The Effects of Virtual Reality, Augmented Reality, and Motion Parallax on Egocentric Depth Perception , 2008, 2008 IEEE Virtual Reality Conference.

[19]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[20]  Dinesh Manocha,et al.  WAVE: Interactive Wave-based Sound Propagation for Virtual Environments , 2015, IEEE Transactions on Visualization and Computer Graphics.

[21]  Mel Slater,et al.  Depth of Presence in Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[22]  Michael P. Snow,et al.  Empirical Models Based on Free-Modulus Magnitude Estimation of Perceived Presence in Virtual Environments , 1998, Hum. Factors.

[23]  Dinesh Manocha,et al.  Psychoacoustic Characterization of Propagation Effects in Virtual Environments , 2016, ACM Trans. Appl. Percept..

[24]  Marcelo M. Wanderley,et al.  Enriched Multimodal Representations of Music Performances: Online Access and Visualization , 2017, IEEE MultiMedia.

[25]  Stephen R. Ellis,et al.  Presence of Mind:A Reaction to Thomas Sheridan's Further Musings on the Psychophysics of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[26]  David B. Kaber,et al.  Telepresence , 1998, Hum. Factors.

[27]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[28]  Mel Slater,et al.  Visual Realism Enhances Realistic Response in an Immersive Virtual Environment , 2009, IEEE Computer Graphics and Applications.

[29]  Robert Höldrich,et al.  A LIBRARY FOR REALTIME 3D BINAURAL SOUND REPRODUCTION IN PURE DATA (PD) , 2005 .

[30]  Carrie Heeter,et al.  Being There: The Subjective Experience of Presence , 1992, Presence: Teleoperators & Virtual Environments.

[31]  Joaquim A. Jorge,et al.  Combining EEG Data with Place and Plausibility Responses as an Approach to Measuring Presence in Outdoor Virtual Environments , 2014, PRESENCE: Teleoperators and Virtual Environments.

[32]  Thomas B. Sheridan.  Further Musings on the Psychophysics of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[33]  Mel Slater,et al.  Simulating virtual environments within virtual environments as the basis for a psychophysics of presence , 2010, ACM Trans. Graph..

[34]  Woodrow Barfield,et al.  The effect of update rate on the sense of presence within virtual environments , 1995, Virtual Reality.

[35]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[36]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[37]  Carrie Heater,et al.  Being There: The Subjective Experience of Presence , 1992, Presence: Teleoperators & Virtual Environments.

[38]  Jean-Jacques Embrechts,et al.  Adding 3D sound to 3D cinema: Identification and evaluation of different reproduction techniques , 2010, 2010 International Conference on Audio, Language and Image Processing.

[39]  Jack M. Loomis,et al.  Distal Attribution and Presence , 1992, Presence: Teleoperators & Virtual Environments.

[40]  Mel Slater,et al.  A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments , 1997, Presence: Teleoperators & Virtual Environments.

[41]  Woodrow Barfield,et al.  Presence within Virtual Environments as a Function of Visual Display Parameters , 1996, Presence: Teleoperators & Virtual Environments.

[42]  Woodrow Barfield,et al.  The Sense of Presence within Auditory Virtual Environments , 1996, Presence: Teleoperators & Virtual Environments.

[43]  Neff Walker,et al.  Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[44]  Marcelo M. Wanderley,et al.  Online Access and Visualization of Enriched Multimodal Representations of Music Performance Recordings: the Quartet Dataset and the Repovizz System , 2022 .

[45]  Andy Clark,et al.  Commentary on J.K O’Regan and A Noe: A sensorimotor account of vision and visual consciousness , 2001 .

[46]  J. Kingman.  A FIRST COURSE IN STOCHASTIC PROCESSES , 1967 .

[47]  Sylvia Weir,et al.  Action perception , 1974 .

[48]  M. Whitton,et al.  Effect of latency on presence in stressful virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..