User Experience in an Interactive Music Virtual Reality System: An Exploratory Study

The Objects VR interface and study explores interactive music and virtual reality, focusing on user experience, understanding of musical functionality, and interaction issues. Our system offers spatio-temporal music interaction using 3D geometric shapes and their designed relationships. Control is provided by tracking of the hands, and the experience is rendered across a head-mounted display with binaural sound presented over headphones. The evaluation of the system uses a mixed methods approach based on semi-structured interviews, surveys and video-based interaction analysis. On average the system was positively received in terms of interview self-report, metrics for spatial presence and creative support. Interaction analysis and interview thematic analysis also revealed instances of frustration with interaction and levels of confusion with system functionality. Our results allow reflection on design criteria and discussion of implications for facilitating music engagement in virtual reality. Finally our work discusses the effectiveness of measures with respect to future evaluation of novel interactive music systems in virtual reality.

[1]  Sidney Fels,et al.  Collaborative Musical Experiences for Novices , 2003 .

[2]  Kenton O'Hara,et al.  On the naturalness of touchless: Putting the “interaction” back into NUI , 2013, TCHI.

[3]  Christoph Klimmt,et al.  A Process Model of the Formation of Spatial Presence Experiences , 2007 .

[4]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[5]  Yvonne Rogers,et al.  Introduction to the special issue on the theory and practice of embodied interaction in HCI and interaction design , 2013, TCHI.

[6]  Stefania Serafin,et al.  Virtual Reality Instruments capable of changing Dimensions in Real-time , 2007 .

[7]  Andrew Johnston.  Beyond Evaluation: Linking Practice and Theory in New Musical Interface Design , 2011, NIME.

[8]  Kjetil Falkenberg Hansen,et al.  Artistically Directed Prototyping in Development and in Practice , 2012 .

[9]  Diogo Cabral,et al.  InspirationWall: Supporting Idea Generation Through Automatic Information Exploration , 2015, Creativity & Cognition.

[10]  Zachary Seldess.  MIAP: Manifold-Interface Amplitude Panning in Max/MSP and Pure Data , 2014 .

[11]  Charles Hill,et al.  What do Prototypes Prototype , 1997 .

[12]  Nicole Fruehauf.  Flow The Psychology Of Optimal Experience , 2016 .

[13]  Pierre Guillot,et al.  The HOA library, review and prospects , 2014, ICMC.

[14]  Jodi Forlizzi,et al.  Understanding experience in interactive systems , 2004, DIS '04.

[15]  Nick Bryan-Kinns,et al.  Designing collaborative musical experiences for broad audiences , 2013, Creativity & Cognition.

[16]  Martin Hachet,et al.  DRILE: An Immersive Environment for Hierarchical Live-Looping , 2010, NIME.

[17]  Penda Tomlinson,et al.  Reconciling Csikszentmihalyi's Broader Flow Theory: With Meaning and Value in Digital Games , 2016, Trans. Digit. Games Res. Assoc..

[18]  Christoph Klimmt,et al.  The Spatial Presence Experience Scale (SPES) , 2016, J. Media Psychol. Theor. Methods Appl..

[19]  Dan Stowell,et al.  Evaluation of live human-computer music-making: Quantitative and qualitative approaches , 2009, Int. J. Hum. Comput. Stud..

[20]  Nalini Ambady,et al.  Fluid Movement and Creativity Pilot Experiments , 2022 .

[21]  Celine Latulipe,et al.  Quantifying the Creativity Support of Digital Tools through the Creativity Support Index , 2014, ACM Trans. Comput. Hum. Interact..

[22]  Christian Heath,et al.  Collaboration and controlCrisis management and multimedia technology in London Underground Line Control Rooms , 1992, Computer Supported Cooperative Work (CSCW).

[23]  Florian Thalmann,et al.  Crossroads: Interactive Music Systems Transforming Performance, Production and Listening , 2016 .

[24]  Alistair G. Sutcliffe,et al.  Heuristic evaluation of virtual reality applications , 2004, Interact. Comput..

[25]  William Ryan,et al.  Evaluating Interactive Entertainment using Breakdown: Understanding Embodied Learning in Video Games , 2009, DiGRA Conference.

[26]  Sidney Fels,et al.  Mapping virtual object manipulation to sound variation , 1997 .

[27]  M. Csíkszentmihályi.  Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990. , 1990 .

[28]  Jeremy R. Cooperstock,et al.  A SPATIAL INTERFACE FOR AUDIO AND MUSIC PRODUCTION , 2006 .

[29]  Rolf Inge Godøy,et al.  Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus , 2006, Organised Sound.

[30]  John Zimmerman,et al.  An analysis and critique of Research through Design: towards a formalization of a research approach , 2010, Conference on Designing Interactive Systems.

[31]  Thomas W. Schubert.  A New Conception of Spatial Presence: Once Again, with Feeling , 2009 .

[32]  Kristina Andersen,et al.  The Instrument as the Source of New in New Music , 2017, Design Issues.

[33]  David Byrne,et al.  How Music Works , 2012 .

[34]  Alexander Travis Adams,et al.  SonicExplorer: fluid exploration of audio parameters , 2014, CHI.

[35]  Xavier Rodet,et al.  Study of haptic and visual interaction for sound and music control in the phase project , 2005 .

[36]  Mel Slater,et al.  Body Centred Interaction in Immersive Virtual Environments , 1994 .

[37]  Karthik Ramani,et al.  MobiSweep: Exploring Spatial Design Ideation Using a Smartphone as a Hand-held Reference Plane , 2016, Tangible and Embedded Interaction.

[38]  Austin Henderson,et al.  Interaction Analysis: Foundations and Practice , 1995 .

[39]  Doug A. Bowman,et al.  Testbed Evaluation of Virtual Environment Interaction Techniques , 2001, Presence Teleoperators Virtual Environ..

[40]  Brian Magerko,et al.  Interaction-based Authoring for Scalable Co-creative Agents , 2015, ICCC.