Real-time spatial processing of sounds for music, multimedia and interactive human-computer interfaces

Abstract. This paper gives an overview of the principles and methods for synthesizing complex 3D sound scenes by processing multiple individual source signals. Signal-processing techniques for directional sound encoding and rendering over loudspeakers or headphones are reviewed, as well as algorithms and interface models for synthesizing and dynamically controling room reverberation and distance effects. A real-time modular spatial-sound-processing software system, called Spat, is presented. It allows reproducing and controling the localization of sound sources in three dimensions and the reverberation of sounds in an existing or virtual space. A particular aim of the Spatialisateur project is to provide direct and computationally efficient control over perceptually relevant parameters describing the interaction of each sound source with the virtual space, irrespective of the chosen reproduction format over loudspeakers or headphones. The advantages of this approach are illustrated in practical contexts, including professional audio, computer music, multimodal immersive simulation systems, and architectural acoustics.

[1]  Manfred R. Schroeder,et al.  Natural Sounding Artificial Reverberation , 1962 .

[2]  John M. Chowning,et al.  THE SIMULATION OF MOVING SOUND SOURCES , 1970 .

[3]  M. Schroeder.  Computer Models for Concert Hall Acoustics , 1973 .

[4]  Georg Plenge,et al.  Localization of Lateral Phantom Sources , 1976 .

[5]  James A. Moorer,et al.  About This Reverberation Business , 1978 .

[6]  Miller S. Puckette,et al.  Designing Multi-Channel Reverberators , 1982 .

[7]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[8]  Michael A. Gerzon,et al.  Ambisonics in Multichannel Broadcasting and Video , 1985 .

[9]  William L. Martens,et al.  Image Model Reverberation from Recirculating Delays , 1986 .

[10]  Scott H. Foster,et al.  A Virtual Display System for Conveying Three-Dimensional Acoustic Information , 1988 .

[11]  Duane H. Cooper,et al.  Prospects for Transaural Recording , 1989 .

[12]  Catherine Lavandier.  Validation perceptive d'un modèle objectif de caractérisation de la qualité acoustique des salles , 1989 .

[13]  David Griesinger,et al.  Practical Processors and Programs for Digital Reverberation , 1989 .

[14]  David G. Malham.  Ambisonics-A Technique for Low-Cost, High-Precision, Three-Dimensional Sound Diffusion , 1990, ICMC.

[15]  O. Warusfel,et al.  ÉTUDE DES PARAMÈTRES LIÉS À LA PRISE DE SON POUR LES APPLICATIONS D'ACOUSTIQUE VIRTUELLE , 1990 .

[16]  Miller S. Puckette,et al.  Combining Event and Signal Processing in the MAX Graphical Programming Environment , 1991 .

[17]  S.H. Foster,et al.  Real Time Synthesis of Complex Acoustic Environments , 1991, Final Program and Paper Summaries 1991 IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics.

[18]  Henrik Møller.  Fundamentals of binaural technology , 1991 .

[19]  Jean-Marc Jot,et al.  Digital Delay Networks for Designing Artificial Reverberators , 1991 .

[20]  Michael A. Gerzon,et al.  Psychoacoustic Decoders for Multispeaker Stereo and Surround Sound , 1992 .

[21]  Felipe Orduña-Bustamante,et al.  Multi-Channel Signal Processing Techniques in the Reproduction of Sound , 1992 .

[22]  Duane H. Cooper,et al.  Generalized Transaural Stereo , 1992 .

[23]  Gérard Assayag,et al.  Spatializer: From Room Acoustics to Virtual Acoustics , 1992, ICMC.

[24]  Günther Theile,et al.  The New Sound Format: 3/2-Stereo , 1993 .

[25]  F L Wightman,et al.  Localization using nonindividualized head-related transfer functions. , 1993, The Journal of the Acoustical Society of America.

[26]  A. Berkhout,et al.  Acoustic control by wave field synthesis , 1993 .

[27]  Jean-Marc Jot,et al.  Binaural Concert Hall Simulation in Real Time , 1993 .

[28]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[29]  David Stanley Mcgrath,et al.  Convolution Processing for Realistic Reverberation , 1995 .

[30]  Michael A. Casey,et al.  Vision-Steered Beam Forming and Transaural Rendering for the Artificial Life Interactive Video Environment (ALIVE) , 1995 .

[31]  David G. Malham,et al.  3-D Sound Spatialization using Ambisonic Techniques , 1995 .

[32]  Michael Cohen,et al.  The design of multidimensional sound interfaces , 1995 .

[33]  François Déchelle,et al.  The IRCAM Real-Time Platform and Applications , 1995, ICMC.

[34]  William G. Gardner,et al.  Efficient Convolution without Input/Output Delay , 1995 .

[35]  Jean-Marc Jot,et al.  Digital Signal Processing Issues in the Context of Binaural and Transaural Stereophony , 1995 .

[36]  Chris Travis.  A Virtual Reality Perspective on Headphone Audio , 1996 .

[37]  Jean-Marc Jot,et al.  Analysis and synthesis of room reverberation based on a statistical time-frequency model , 1997 .

[38]  Jean-Marc Jot,et al.  Efficient models for reverberation and distance rendering in computer music and virtual audio reality , 1997, ICMC.

[39]  W. G. Gardner,et al.  3-D Audio Using Loudspeakers , 1998 .