The Potential Disconnect between Time Perception and Immersion: Effects of Music on VR Player Experience

How much music contributes to player experience (PX) in virtual reality (VR) games remains unclear in the games user research literature. A core factor of PX in VR games that has not been studied before (in relation to audio or otherwise) is time perception. Thus, we provide the first empirical exploration of how music affects time perception in a VR game. In a user study (N=64), we investigated the effects of music on PX and time perception (operationalized as retrospective time estimation). Participants retrospectively perceived time to pass significantly quicker in the VR game when music was present, but reported no difference in PX components, including immersion. This contributes to ongoing discourse on the surprising lack of music effects in VR games. Moreover, our results highlight the need to re-conceptualize our understanding of the relationship between time perception and immersion in games.

[1]  Katja Rogers,et al.  Game Atmosphere: Effects of Audiovisual Thematic Cohesion on Player Experience and Psychophysiology , 2020, CHI PLAY.

[2]  A. Maratos,et al.  Music therapy for depression. , 2008, The Cochrane database of systematic reviews.

[3]  D. Eagleman,et al.  The Effect of Predictability on Subjective Duration , 2007, PloS one.

[4]  A. Imran Nordin,et al.  The Effect of Time Manipulation on Immersion in Digital Games , 2017, IVIC.

[5]  V. Pouthas,et al.  The role of physiological arousal in time perception: Psychophysiological evidence from an emotion regulation paradigm , 2011, Brain and Cognition.

[6]  Inger Ekman.  On the desire to not kill your players: Rethinking sound in pervasive and mixed reality games , 2013, FDG.

[7]  Christoph Klimmt,et al.  Effects of soundtrack music on the video game experience , 2018, Media Psychology.

[8]  Teresa Lynch,et al.  Nothing to Fear? An Analysis of College Students' Fear Experiences With Video Games , 2015 .

[9]  Diego Klabjan,et al.  Rapid Prediction of Player Retention in Free-to-Play Mobile Games , 2016, AIIDE.

[10]  S. Droit-Volet,et al.  Fear and time: Fear speeds up the internal clock , 2015, Behavioural Processes.

[11]  Paul A. Cairns,et al.  Time perception, immersion and music in videogames , 2010, BCS HCI.

[12]  Marcelo Bussotti Reyes,et al.  Effects of Chronic Video Game Use on Time Perception: Differences Between Sub- and Multi-Second Intervals , 2013, Cyberpsychology Behav. Soc. Netw..

[13]  Mark D. Griffiths,et al.  Experiences of Time Loss among Videogame Players: An Empirical Study , 2007, Cyberpsychology Behav. Soc. Netw..

[14]  Martin P. Paulus,et al.  Psychological and Neural Mechanisms of Subjective Time Dilation , 2011, Front. Neurosci..

[15]  Yoshihiro Shimomura,et al.  Effects of monochromatic light on time sense for short intervals. , 2007, Journal of physiological anthropology.

[16]  Peter Wolf,et al.  Towards a system of customized video game mechanics based on player personality: Relating the Big Five personality traits with difficulty adaptation in a first-person shooter game , 2016, Entertain. Comput..

[17]  Masayuki Numao,et al.  An analysis of player affect transitions in survival horror games , 2014, Journal on Multimodal User Interfaces.

[18]  Jih-Hsuan Tammy Lin,et al.  Fear in virtual reality (VR): Fear elements, coping reactions, immediate and next-day fright responses toward a survival horror zombie virtual reality game , 2017, Comput. Hum. Behav..

[19]  G. Cassidy,et al.  The effects of music on time perception and performance of a driving game. , 2010, Scandinavian journal of psychology.

[20]  Simon Grondin,et al.  An Ecological Approach to Prospective and Retrospective Timing of Long Durations: A Study Involving Gamers , 2010, PloS one.

[21]  Jiulin Zhang,et al.  The Influence of Background Music of Video Games on Immersion , 2015 .

[22]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[23]  Katja Rogers,et al.  Effects of Background Music on Risk-Taking and General Player Experience , 2019, CHI PLAY.

[24]  M. Csíkszentmihályi,et al.  The Concept of Flow , 2014 .

[25]  Elena Márquez Segura,et al.  ExerCube vs. Personal Trainer: Evaluating a Holistic, Immersive, and Adaptive Fitness Game Setup , 2019, CHI.

[26]  Adrian C. North,et al.  Musical preferences during and after relaxation and exercise. , 2000, The American journal of psychology.

[27]  Daniel L. King,et al.  The Role of Structural Characteristics in Problematic Video Game Play: An Empirical Study , 2011, International Journal of Mental Health and Addiction.

[28]  Matthias Rauterberg,et al.  Incongruity-Based Adaptive Game Balancing , 2009, ACG.

[29]  Karen Collins,et al.  Subjective Measures of the Influence of Music Customization on the Video Game Play Experience: A Pilot Study , 2011, Game Stud..

[30]  Daniel Västfjäll,et al.  Music, subjective well-being and health : The role of everyday emotions , 2012 .

[31]  G. Cassidy,et al.  The effects of music choice on task performance: A study of the impact of self-selected and experimenter-selected music on driving game performance and experience , 2009 .

[32]  P. Laukka,et al.  Uses of music and psychological well-being among the elderly , 2007 .

[33]  P. Hancock,et al.  How cognitive load affects duration judgments: A meta-analytic review. , 2010, Acta psychologica.

[34]  M. Csíkszentmihályi,et al.  Optimal experience: Psychological studies of flow in consciousness. , 1988 .

[35]  Paul A. Cairns,et al.  Attention, time perception and immersion in games , 2013, CHI Extended Abstracts.

[36]  Farrokh Habibzadeh,et al.  Common statistical mistakes in manuscripts submitted to biomedical journals , 2013 .

[37]  Anke Huckauf,et al.  Effects of Neutral and Fearful Mood on Duration Estimation of Neutral and Fearful Face Stimuli , 2016 .

[38]  Julian Frommel,et al.  Exploring Interaction Fidelity in Virtual Reality: Object Manipulation and Whole-Body Movements , 2019, CHI.

[39]  David M. Eagleman,et al.  Does Time Really Slow Down during a Frightening Event? , 2007, PloS one.

[40]  LinJih-Hsuan Tammy.  Fear in virtual reality (VR) , 2017 .

[41]  M. Csíkszentmihályi.  Flow and the Foundations of Positive Psychology , 2014 .

[42]  David Bashwiner.  Musical analysis for multimedia: A perspective from music theory. , 2013 .

[43]  R M Church,et al.  Scalar Timing in Memory , 1984, Annals of the New York Academy of Sciences.

[44]  John E. Schlerf,et al.  Dedicated and intrinsic models of time perception , 2008, Trends in Cognitive Sciences.

[45]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[46]  Thomas Bliesener,et al.  The Effect of Computer Gaming on Subsequent Time Perception , 2009 .

[47]  Manfred Tscheligi,et al.  CHI '04 Extended Abstracts on Human Factors in Computing Systems , 2004, CHI 2004.

[48]  Sylvie Droit-Volet,et al.  Time, Emotion and the Embodiment of Timing , 2013 .

[49]  Thomas Hess,et al.  Immersion, Presence, Interactivity: Towards a Joint Understanding of Factors Influencing Virtual Reality Acceptance and Use , 2017, AMCIS.

[50]  E. Bigand,et al.  Time Changes with the Embodiment of Another’s Body Posture , 2011, PloS one.

[51]  A. J. Delong,et al.  Phenomenological space-time: toward an experiential relativity. , 1981, Science.

[52]  Gerd Bruder,et al.  Who turned the clock? Effects of Manipulated Zeitgebers, Cognitive Load and Immersion on Time Estimation , 2016, IEEE Transactions on Visualization and Computer Graphics.

[53]  M. Grimshaw,et al.  Game Sound Technology and Player Interaction: Concepts and Developments , 2010 .

[54]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[55]  L. Rüschendorf,et al.  On the Perception of Time , 2009, Gerontology.

[56]  Annabel J. Cohen,et al.  Future research directions for music and sound in multimedia , 2013 .

[57]  Daniel M. Johnson,et al.  Design and Preliminary Validation of The Player Experience Inventory , 2016, CHI PLAY.

[58]  V. N. Stratton,et al.  The Effect of Background Music on Verbal Interaction in Groups , 1984 .

[59]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[60]  R. Lozano.  It’s About Time: The Illusions of Time Perception and Travel in Immersive Virtual Reality , 2016 .

[61]  Ting-Jui Chou,et al.  The Role of Flow Experience in Cyber-Game Addiction , 2003, Cyberpsychology Behav. Soc. Netw..

[62]  Alex Copello,et al.  The Impact of Massively Multiplayer Online Role Playing Games (MMORPGs) on Psychological Wellbeing and the Role of Play Motivations and Problematic Use , 2014, International Journal of Mental Health and Addiction.

[63]  Edwin H. Blake,et al.  The Effects of Mediation in a Storytelling Virtual Environment , 2003, International Conference on Virtual Storytelling.

[64]  M. Treisman.  Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[65]  Annabel J. Cohen.  Congruence-Association Model of music and multimedia: Origin and evolution , 2013 .

[66]  Simon Grondin,et al.  Video games and the perception of very long durations by adolescents , 2009, Comput. Hum. Behav..

[67]  Michael Weber,et al.  Vanishing Importance: Studying Immersive Effects of Game Audio Perception on Player Experiences in Virtual Reality , 2018, CHI.

[68]  E. Bigand,et al.  Time flies with music whatever its emotional valence. , 2010, Acta psychologica.

[69]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[70]  Susan M. Schneider,et al.  Effect of virtual reality on time perception in patients receiving chemotherapy , 2011, Supportive Care in Cancer.

[71]  Enrico Rukzio,et al.  Demo of the Matrix Has You: Realizing Slow Motion in Full-Body Virtual Reality , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[72]  Aliimran Nordin,et al.  Immersion and players' time perception in digital games , 2014 .

[73]  Frans Mäyrä,et al.  Fundamental Components of the Gameplay Experience: Analysing Immersion , 2005, DiGRA Conference.

[74]  Mendel Kleiner,et al.  Auditory-Induced Presence in Mixed Reality Environments and Related Technology , 2010, The Engineering of Mixed Reality Systems.

[75]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[76]  Katja Rogers,et al.  Audio Habits and Motivations in Video Game Players , 2019, Audio Mostly Conference.

[77]  Marilyn Gail Boltz,et al.  Audiovisual Interactions: The Impact of Visual Information on Music Perception and Memory , 2009 .

[78]  Daniel M. Johnson,et al.  Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences , 2020, Int. J. Hum. Comput. Stud..

[79]  Neff Walker,et al.  Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[80]  Gerd Bruder,et al.  Threefolded motion perception during immersive walkthroughs , 2014, VRST '14.

[81]  PEI-LUEN PATRICK RAU,et al.  Time Distortion for Expert and Novice Online Game Players , 2006, Cyberpsychology Behav. Soc. Netw..

[82]  J. Webster,et al.  The Dimensionality and Correlates of Flow in Human-Computer Interactions. , 1993 .

[83]  W. Meck,et al.  How emotions colour our perception of time , 2007, Trends in Cognitive Sciences.

[84]  R. Ragot,et al.  How emotional auditory stimuli modulate time perception. , 2007, Emotion.

[85]  Kristine Jørgensen,et al.  Left in the dark: playing computer games with the sound turned off , 2008 .

[86]  Marilyn Tremaine.  CHI '01 Extended Abstracts on Human Factors in Computing Systems , 2001, CHI Extended Abstracts.

[87]  Siu-Lan Tan,et al.  Effects of Built-in Audio versus Unrelated Background Music on Performance In an Adventure Role-Playing Game , 2010, Int. J. Gaming Comput. Mediat. Simulations.

[88]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[89]  Jacqueline Bp,et al.  Developing and Using a Case Formulation to Guide Cognitive-Behavior Therapy , 2014 .