Virtual reality musical instruments: Guidelines for multisensory interaction design

The rapid development and availability of low cost technologies has created a wide interest in virtual reality (VR), but how to design and evaluate multisensory interactions in VR remains as a challenge. In this paper, we focus on virtual reality musical instruments, present an overview of our design and evaluation guidelines, and examine historical case studies. Our main contribution is to inform the design and evaluation of the future VRMIs and consider the challenges.

[1]  Mel Slater,et al.  Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play , 2013, IEEE Transactions on Visualization and Computer Graphics.

[2]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  Cumhur Erkut,et al.  A Structured Design and Evaluation Model with Application to Rhythmic Interaction Displays , 2011, NIME.

[4]  Perttu Hämäläinen,et al.  EFFECT OF LATENCY ON PLAYING ACCURACY OF TWO GESTURE CONTROLLED CONTINUOUS SOUND INSTRUMENTS WITHOUT TACTILE FEEDBACK , 2004 .

[5]  Jaron Lanier,et al.  Reality built for two: a virtual reality tool , 1990, I3D '90.

[6]  Tapio Takala,et al.  Experiments with Virtual Reality Instruments , 2005, NIME.

[7]  Cumhur Erkut,et al.  Design and Evaluation of Human–Computer Rhythmic Interaction in a Tutoring System , 2011, Computer Music Journal.

[8]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[9]  Rainer Brunn.  The sound of one hand , 1995 .

[10]  F. Brooks,et al.  Physiological Reaction as an Objective Measure of Presence in Virtual Environments , 2001 .

[11]  Cumhur Erkut,et al.  Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions , 2016, Computer Music Journal.

[12]  Teemu Mäki-Patola,et al.  Physics-based modeling of musical instruments for interactive virtual reality , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[13]  Stefania Serafin,et al.  Virtual Reality Instruments capable of changing Dimensions in Real-time , 2007 .

[14]  Mel Slater,et al.  Public Speaking in Virtual Reality: Facing an Audience of Avatars , 1999, IEEE Computer Graphics and Applications.