Physically based sonic interaction synthesis for computer games

In this paper we describe a platform in which sounds synthesized in real-time by using physical models are integrated in a multimodal environment. We focus in particular on sound effects created by actions of the player in the environment such as waking on different surfaces and hitting different objects. The sound effects are implemented as extensions to the real-time sound synthesis engine Max/MSP.1 An 8-channel soundscape is spatialized using the vector based amplitude panning (VBAP) algorithm developed by VIlle Pulkki [17]. The sonic environment is connected through TCP/IP to Virtools.2

[1]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[2]  Ville Pulkki.  Generic panning tools for MAX/MSP , 2000, ICMC.

[3]  Joseph A. Paradiso,et al.  Design and implementation of expressive footwear , 2000, IBM Syst. J..

[4]  Julius O. Smith,et al.  Physical Modeling Using Digital Waveguides , 1992 .

[5]  James F. O'Brien,et al.  Synthesizing Sounds from Physically Based Motion , 2001, SIGGRAPH Video Review on Animation Theater Program.

[6]  Claude Cadoz,et al.  Physical Models for Music and Animated Image: The Use of CORDIS-ANIMA in "ESQUISSES"; a Music Film by ACROE , 1994, ICMC.

[7]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[8]  Tina Blaine.  The Convergence of Alternate Controllers and Musical Interfaces in Interactive Entertainment , 2005, NIME.

[9]  Davide Rocchesso,et al.  Physically-based Sounding Objects, as We Develop Them Today , 2004 .

[10]  William W. Gaver.  How Do We Hear in the World?: Explorations in Ecological Acoustics , 1993 .

[11]  Chen Shen,et al.  Synthesizing sounds from rigid-body simulations , 2002, SCA '02.

[12]  Tapio Takala,et al.  Sound rendering , 1992, SIGGRAPH.

[13]  Stefania Serafin,et al.  A simple practical approach to a wireless data acquisition board , 2006, NIME.

[14]  Rolf Nordahl.  Design and evaluation of a multi-modal footstep controller with VR-applications , 2005 .

[15]  Jean-Marie Adrien,et al.  The missing link: modal synthesis , 1991 .

[16]  Perry R. Cook,et al.  Real Sound Synthesis for Interactive Applications , 2002 .

[17]  Tapio Takala,et al.  An integrated approach to motion and sound , 1995, Comput. Animat. Virtual Worlds.

[18]  Perry R. Cook,et al.  Physically Informed Sonic Modeling (PhISM): Synthesis of percussive sounds , 1997 .