The Role of Agency in Ludoacoustic Immersion: Experiencing Recorded Sound and Music in Situational Context

In sound-fx and music, expressive artifacts of the recording contribute to forming expectations on how an object (sound source) is interacting with an environment (room) while also accounting for the associated resonances occurring in that object. By switching attentional focus between action and resonance in object and environment as a result of comparing expectations to incoming stimuli, the context of a virtual situation is simulated by referencing to syntax of body-object-environment interaction. This virtual syntax may be partially projected onto the situational context of the user, leading to antecedents of immersion that depend on emotional arousal and personality traits of the listener. After having outlined a conceptual framework describing the mediation and agency detection of sonic expression within the acoustic properties of situational contexts, the paper provides an outlook on how these agents may be translated to meaningful structures that are yet to be studied in video games.

[1]  C. Allesch,et al.  Understanding our experience of music: What kind of psychology do we need? , 2006 .

[2]  Mark Grimshaw,et al.  Sound And Immersion In The First-Person Shooter , 2007, Int. J. Intell. Games Simul..

[3]  D. Västfjäll,et al.  An experience sampling study of emotional reactions to music: listener, music, and situation. , 2008, Emotion.

[4]  A N Popper,et al.  Evolution of the ear and hearing: issues and questions. , 1997, Brain, behavior and evolution.

[5]  Jean Decety,et al.  Film music influences how viewers relate to movie characters. , 2011 .

[6]  D. Berlyne,et al.  Aesthetics and psychobiology , 1975 .

[7]  Thomas F Münte,et al.  Unforgettable film music: The role of emotion in episodic long-term memory for music , 2008, BMC Neuroscience.

[8]  David Jones,et al.  Play it again, Sam , 1990, Nature.

[9]  P. Janata.  Music and the self , 2009 .

[10]  Mendel Kleiner,et al.  Auditory-Induced Presence in Mixed Reality Environments and Related Technology , 2010, The Engineering of Mixed Reality Systems.

[11]  R. Gregory.  Perceptions as hypotheses. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[12]  Mark R. Anderson,et al.  Direct comparison of the impact of head tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. , 2001, Journal of the Audio Engineering Society. Audio Engineering Society.

[13]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[14]  Hans-Peter Gasselseder.  Re-orchestrating Game Drama: The Immersive Experience of Dynamic Music in Video Games , 2013, EVA.

[15]  Jens Blauert,et al.  Spaces Speak, Are You Listening? Experiencing Aural Architecture , 2007 .

[16]  Marc Leman.  Music, Gesture, and the Formation of Embodied Meaning , 2009 .

[17]  R. Schubotz.  Prediction of external events with our motor system: towards a new framework , 2007, Trends in Cognitive Sciences.

[18]  Emery Schubert,et al.  Cognitive Styles of Music Listening , 2008 .

[19]  M. Jeannerod.  The mechanism of self-recognition in humans , 2003, Behavioural Brain Research.

[20]  Philip H. Marsden,et al.  Designing Auditory Spaces to Support Sense of Place: The Role of Expectation , 2002 .

[21]  Robert J. Zatorre,et al.  Mental Concerts: Musical Imagery and Auditory Cortex , 2005, Neuron.

[22]  Mel Slater,et al.  Presence and The Sixth Sense , 2002, Presence: Teleoperators & Virtual Environments.

[23]  Marc Jeannerod,et al.  The sense of agency and its disturbances in schizophrenia: a reappraisal , 2008, Experimental Brain Research.

[24]  D. Wegner,et al.  Blaming God for Our Pain: Human Suffering and the Divine Mind , 2010, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[25]  Rolf A. Zwaan.  Situation Models , 1999 .

[26]  P. Larsson,et al.  Title: Auditory-induced Presence in Mediated Environments and Related Technology , 2005 .

[27]  Roberto Bresin,et al.  Emotion rendering in music: Range and characteristic values of seven musical variables , 2011, Cortex.

[28]  J. Bruner,et al.  On the perception of incongruity; a paradigm. , 1949, Journal of personality.

[29]  S. Gallagher.  Philosophical conceptions of the self: implications for cognitive science , 2000, Trends in Cognitive Sciences.

[30]  William W. Gaver,et al.  Play it again, Sam: On Liking Music , 1987 .

[31]  Yôiti Suzuki,et al.  Effects of visual information on auditory presence , 2003 .

[32]  J. Decety,et al.  Effect of subjective perspective taking during simulation of action: a PET investigation of agency , 2001, Nature Neuroscience.

[33]  J. Bruner.  Actual minds, possible worlds , 1985 .

[34]  John E. McEneaney.  Agency Effects in Human–Computer Interaction , 2013, Int. J. Hum. Comput. Interact..

[35]  Johnny Wingstedt,et al.  Young adolescents' usage of narrative functions of media music by manipulation of musical expression , 2008 .

[36]  Jack M. Loomis,et al.  Limited Field of View of Head-Mounted Displays Is Not the Cause of Distance Underestimation in Virtual Environments , 2004, Presence: Teleoperators & Virtual Environments.

[37]  Marc Woodbury-Smith,et al.  The Essential Difference: Male And Female Brains And The Truth About Autism , 2003 .

[38]  L. Barsalou.  Flexibility, Structure, and Linguistic Vagary in Concepts: Manifestations of a Compositional System of Perceptual Symbols , 2019, Theories of Memory.

[39]  Oliver Vitouch,et al.  When Your Ear Sets the Stage: Musical Context Effects in Film Perception , 2001 .

[40]  Marilyn Gail Boltz,et al.  Audiovisual Interactions: The Impact of Visual Information on Music Perception and Memory , 2009 .

[41]  H. Brenton,et al.  The Uncanny Valley : does it exist ? , 2005 .