Being in a Virtual World: Presence, Environment, Salience, Sound

We present a theoretical model of how presence is achieved in virtual worlds, from which future frameworks prescribing the sensory design of such worlds can be constructed. Drawing on the study of cognitive metaphors and the concept that sound is an emergent perception we offer an account of the environment as a salient and dynamic construct that functions as a metonym for the nonself. Separating environment from world, we discuss the role of sound in the forming of the environment and argue that it is this environment that establishes the means for presence because it is the process behind the construction of the environment that individuates self from nonself and such differentiation is a defining feature of the concept of presence.

[1]  G. Lakoff.  The Neural Theory of Metaphor , 2009 .

[2]  J. Graftieaux.  [The uncanny]. , 2011, Annales francaises d'anesthesie et de reanimation.

[3]  J. Waterworth,et al.  Distributed Embodiment : Real Presence in Virtual Bodies , 2013 .

[4]  Alice Deignan,et al.  Metaphor and Corpus Linguistics , 2005 .

[5]  R. Gibbs.  The Cambridge Handbook of Metaphor and Thought , 2008 .

[6]  Eric Clarke,et al.  Ways of Listening: An Ecological Approach to the Perception of Musical Meaning , 2005 .

[7]  Jennifer Ann Rode,et al.  Using a large projection screen as an alternative to head-mounted displays for virtual environments , 2000, CHI.

[8]  Taylor Carman,et al.  The Body in Husserl and Merleau-Ponty , 1999 .

[9]  Mark Grimshaw,et al.  Sonic Virtuality: Sound as Emergent Perception , 2015 .

[10]  A. McMahan.  Immersion, Engagement, and Presence: A Method for Analyzing 3-D Video Games , 2013 .

[11]  M. Grimshaw,et al.  The Oxford handbook of virtuality , 2014 .

[12]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[13]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[14]  Mads Walther-Hansen,et al.  The Force Dynamic Structure of the Phonographic Container: How Sound Engineers Conceptualise the 'Inside' of the Mix , 2014 .

[15]  Frans Mäyrä,et al.  Fundamental Components of the Gameplay Experience: Analysing Immersion , 2005, DiGRA Conference.

[16]  D. H. Warren,et al.  The role of visual-auditory “compellingness” in the ventriloquism effect: Implications for transitivity among the spatial senses , 1981, Perception & psychophysics.

[17]  T. Ziemke.  What’s that Thing Called Embodiment? , 2003 .

[18]  Joseph M. Williams.  Synaesthetic Adjectives: A Possible Law of Semantic Change , 1976 .

[19]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[20]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[21]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[22]  Paul Sambre,et al.  Gilles Fauconnier & Mark Turner, " The way we think: conceptual blending and the mind's hidden complexities" , 2002 .

[23]  Maurice Merleau-Ponty.  Phenomenology of Perception , 1964 .

[24]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[25]  Jeffrey R. Binder,et al.  The Neural Career of Sensory-motor Metaphors , 2011, Journal of Cognitive Neuroscience.

[26]  Mel Slater,et al.  A note on presence terminology , 2003 .

[27]  Wijnand A. IJsselsteijn,et al.  Presence: Where Are We? , 2001, Cyberpsychology Behav. Soc. Netw..

[28]  A. Bregman,et al.  Demonstrations of auditory scene analysis : the perceptual organization of sound , 1995 .

[29]  Mark Grimshaw,et al.  Presence through sound , 2017 .

[30]  Thor Magnusson,et al.  Of Epistemic Tools: musical instruments as cognitive extensions , 2009, Organised Sound.

[31]  Mark Grimshaw.  A brief argument for, and summary of, the concept of Sonic Virtuality , 2015 .

[32]  Mark Grimshaw,et al.  The Sound of the Smell of my Shoes , 2015, AM '15.

[33]  Eric Clarke,et al.  Ways of Listening , 2005 .

[34]  Lauren Davidson,et al.  A Sense of Space , 2016 .

[35]  Alan Chalmers.  Level of Realism , 2014 .

[36]  N. Presmeg.  The body in the mind: The bodily basis of meaning, imagination and reason , 1992 .

[37]  Robert J. Zatorre,et al.  There’s more to auditory cortex than meets the ear , 2007, Hearing Research.

[38]  The Digital Eye : Image Metrics Attempts to Leap the Uncanny Valley , 2011 .

[39]  Mark Grimshaw,et al.  Imagining sound , 2014, Audio Mostly Conference.

[40]  Gordon Calleja,et al.  Immersion in Virtual Worlds , 2014 .

[41]  G. Fauconnier,et al.  The Way We Think: Conceptual Blending and the Mind''s Hidden Complexities. Basic Books , 2002 .

[42]  G. Lakoff.  Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[43]  Mel Slater,et al.  Presence and The Sixth Sense , 2002, Presence: Teleoperators & Virtual Environments.

[44]  Mark Grimshaw,et al.  The Uncanny Wall , 2011, Int. J. Arts Technol..

[45]  Z. Kövecses,et al.  Metaphor: A Practical Introduction , 2002 .

[46]  Mads Walther-Hansen,et al.  Balancing Audio: Towards a Cognitive Structure of Sound Interaction in Music Production , 2015, CMMR.

[47]  A. Clark.  Expecting the World: Perception, Prediction, and the Origins of Human Knowledge , 2013 .