Virtual Slide Guitar

This article describes a system for virtual slide guitar playing. From the control point of view, it can be seen as a successor of the virtual air guitar (VAG) developed at Helsinki University of Technology (TKK) a few years ago (Karjalainen et al. 2006). The original VAG allows the user to control an electric guitar synthesizer by mimicking guitar-playing gestures. In the current work, the same gesture-control approach that was found successful in the VAG is used: a computer-visionbased system, in which a camera detects the player's hands and a computer tracks the hand movements and converts them into control data, such as pluck events and string length. Sound synthesis in the virtual slide guitar application is based on an energy-compensated time-varying digital waveguide model of a guitar with new extensions to generate contact sounds caused by the slide tube touching the strings. Video files showing the virtual slide guitar in action can be found on the accompanying Web page (www.acoustics.hut.fi/publications/papers/vsg/) and the forthcoming 2008 Computer Music Journal Sound and Video Anthology DVD. Although the current implementation uses a camera-based user interface, it can also be controlled by other humanmachine interfaces or computer programs. Excellent reviews on gestural control of music synthesis have been written by Paradiso (1997) and by Wanderley and Depalle (2004). Camera-based gesture analysis has become very sophisticated owing to increased computing power and new methodologies provided by research, as exemplified by the EyesWeb platform (Camurri et al. 2000, 2005; Gorman et al. 2007). Digital waveguide modeling is mature technology, which can be applied to high-quality synthesis of various musical instruments (Smith 1992; Valimaki et al. 2006). For more information on waveguide synthesis of string instruments, see Valimaki et al. (1996), Karjalainen, Valimaki, and Tolonen (1998), and Karjalainen et al. (2006). The roots of this study are in an early prototype of the TKK virtual air guitar, which here is called the rubber-band virtual air guitar. It was a simplified gesture-control system, in which the distance between the hands was detected and di-

[1]  Julius O. Smith,et al.  Physical Modeling Using Digital Waveguides , 1992 .

[2]  J Pakarinen,et al.  Analysis of handling noises on wound strings. , 2007, The Journal of the Acoustical Society of America.

[3]  Davide Rocchesso,et al.  The Sounding Object , 2002 .

[4]  Unto K. Laine,et al.  Splitting the Unit Delay - Tools for fractional delay filter design , 1996 .

[5]  James R. Hopgood,et al.  Nonconcurrent multiple speakers tracking based on extended Kalman particle filter , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Shuji Hashimoto,et al.  EyesWeb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems , 2000, Computer Music Journal.

[7]  Hazarathaiah Malepati,et al.  Speech and Audio Processing , 2010 .

[8]  Marc Le Brun,et al.  Digital Waveshaping Synthesis , 1979 .

[9]  J. A. Paradiso.  Electronic money: toward a virtual wallet , 1997 .

[10]  Marcelo M. Wanderley,et al.  Gestural control of sound synthesis , 2004, Proceedings of the IEEE.

[11]  Matti Karjalainen,et al.  Virtual Air Guitar , 2006 .

[12]  Federico Fontana,et al.  Physics-based models for the acoustic representation of space in virtual environments , 2003 .

[13]  Perry R. Cook,et al.  Physically Informed Sonic Modeling (PhISM): Synthesis of percussive sounds , 1997 .

[14]  Marc Leman,et al.  Communicating expressiveness and affect in multimodal interactive systems , 2005, IEEE MultiMedia.

[15]  Vesa Välimäki,et al.  Development and Calibration of a Guitar Synthesizer , 1997 .

[16]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[17]  Davide Rocchesso,et al.  Sounding Objects , 2003, IEEE Multim..

[18]  Perry R. Cook,et al.  Physically Informed Sonic Modeling (PhISM): Percussive Synthesis , 1996, International Conference on Mathematics and Computing.

[19]  Miller Puckette,et al.  Pure Data , 1997, ICMC.

[20]  Matti Karjalainen,et al.  Energy behavior in time-varying fractional delay filters for physical modeling synthesis of musical instruments , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[21]  Davide Rocchesso,et al.  Interactive Simulation of rigid body interaction with friction-induced sound generation , 2005, IEEE Transactions on Speech and Audio Processing.

[22]  Richard Kronland-Martinet,et al.  Analysis-synthesis of impact sounds by real-time dynamic filtering , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[23]  Unto K. Laine,et al.  Splitting the unit delay [FIR/all pass filters design] , 1996, IEEE Signal Process. Mag..

[24]  Richard Kronland-Martinet,et al.  A Percussive Sound Synthesizer Based on Physical and Perceptual Attributes , 2006, Computer Music Journal.

[25]  M. Leman,et al.  Model-based sound synthesis of the guqin. , 2006, The Journal of the Acoustical Society of America.

[26]  Vesa Välimäki,et al.  Physical Modeling of Plucked String Instruments with Application to Real-Time Sound Synthesis , 1996 .

[27]  Margrit Betke,et al.  A Camera-Based Music-Making Tool for Physical Rehabilitation , 2005, Computer Music Journal.

[28]  Cumhur Erkut,et al.  Synthesis of Hand Clapping Sounds , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[29]  Matti Karjalainen,et al.  Plucked-string models: From the Karplus-Strong algorithm to digital waveguides and beyond , 1998 .

[30]  M. Karjalainen,et al.  Discrete-time modelling of musical instruments , 2005 .

[31]  Davide Rocchesso,et al.  Continuous sonic feedback from a rolling ball , 2005, IEEE MultiMedia.

[32]  Daniel Arfib.  Digital Synthesis of Complex Spectra by means of Multiplication of Non-linear Distorted Sine Waves , 1978, ICMC.