Virtual Reality System with Integrated Sound Field Simulation and Reproduction

A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.

[1]  H. Steven Colburn,et al.  Role of spectral detail in sound-source localization , 1998, Nature.

[2]  M. Vorländer.  Simulation of the transient and steady‐state sound propagation in rooms using a new combined ray‐tracing/image‐source algorithm , 1989 .

[3]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[4]  Russell L. Storms,et al.  NPSNET-3D Sound Server. An Effective Use of the Auditory Channel. , 1995 .

[5]  Ronald Azuma,et al.  A frequency-domain analysis of head-motion prediction , 1995, SIGGRAPH.

[6]  Jont B. Allen,et al.  Image method for efficiently simulating small‐room acoustics , 1976 .

[7]  W. G. Gardner,et al.  3-D Audio Using Loudspeakers , 1998 .

[8]  R. Schmacher,et al.  Study for Applying Computer-Generated Images to Visual Simulation: (510842009-001) , 1969 .

[9]  U. Svensson,et al.  An analytic secondary source model of edge diffraction impulse responses , 1999 .

[10]  P. Nelson,et al.  Measurement of transient response of rooms and comparison with geometrical acoustic models , 1999 .

[11]  M. Vorländer,et al.  Definition and measurement of random-incidence scattering coefficients , 2000 .

[12]  Hareo Hamada,et al.  Local sound field reproduction using two closely spaced loudspeakers , 1998 .

[13]  Graham Naylor,et al.  ODEON—Another hybrid room acoustical model , 1993 .

[14]  Heinrich Kuttruff,et al.  Room acoustics , 1973 .

[15]  E. H. Dooijes,et al.  Spatial audio in graphical applications , 1994 .

[16]  Tim Brookes,et al.  The Effect of Non-Symmetrical Left/Right Recording Pinnae on the Perceived Externalisation of Binaural Recordings. , 2005 .

[17]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[18]  Markus H. Gross,et al.  Spatialized audio rendering for immersive virtual environments , 2002, VRST '02.

[19]  Ming Ouhyoung,et al.  A 3D tracking experiment on latency and its compensation methods in virtual environments , 1995, UIST '95.

[20]  Tapio Lokki,et al.  Creating Interactive Virtual Acoustic Environments , 1999 .

[21]  George Drettakis,et al.  Perceptual audio rendering of complex virtual environments , 2004, ACM Trans. Graph..

[22]  H. Kuttruff.  A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries , 1995 .

[23]  Jens Holger Rindel.  Evaluation of room acoustic qualities and defects by use of auralization , 2004 .

[24]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[25]  Lauri Savioja,et al.  Modeling Techniques for Virtual Acoustics , 1999 .

[26]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[27]  Dirk Schröder,et al.  Real-time processing of image sources using binary space partitioning , 2006 .

[28]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[29]  Thomas A. Funkhouser,et al.  Real-time acoustic modeling for distributed virtual environments , 1999, SIGGRAPH.

[30]  Tobias Lentz.  Dynamic crosstalk cancellation for binaural synthesis in virtual reality environments , 2006 .

[31]  Thomas A. Funkhouser,et al.  Modeling acoustics in virtual environments using the uniform theory of diffraction , 2001, SIGGRAPH.

[32]  Joseph J. LaViola,et al.  A testbed for studying and choosing predictive tracking algorithms in virtual environments , 2003, IPT/EGVE.

[33]  J. Borish.  Extension of the image model to arbitrary polyhedra , 1984 .

[34]  D. Ross Jeffery,et al.  CASE: a testbed for modeling, measurement and management , 1992, CACM.

[35]  Michael Vorländer,et al.  Beugungssimulation ohne Rechenzeitexplosion: Die Methode der quantisierten Pyramidenstrahlen : ein neues Berechnungsverfahren für Raumakustik und Lärmimmissionsprognose ; Vergleiche, Ansätze, Lösungen , 2004 .

[36]  Henrik Møller.  Reproduction of artificial head recordings through loudspeakers , 1989 .

[37]  Oliver Schmitz,et al.  Realisation of an Adaptive Cross-talk Cancellation System for a Moving Listener , 2002 .

[38]  Ingo Assenmacher,et al.  Performance of Spatial Audio Using Dynamic Cross-Talk Cancellation , 2005 .

[39]  David A. Burgess,et al.  An Architecture for Spatial Audio Servers , 1993 .

[40]  Tomas Akenine-Möller,et al.  Real-Time Rendering, Second Edition , 2002 .

[41]  Renate Heinz,et al.  Binaural room simulation based on an image source model with addition of statistical methods to include the diffuse sound scattering of walls and to predict the reverberant tail , 1993 .

[42]  Michael Vorländer,et al.  Spatial Variation of Lateral Measures in Different Concert Halls , 2004 .

[43]  Ronald Azuma,et al.  Improving static and dynamic registration in an optical see-through HMD , 1994, SIGGRAPH.

[44]  C. L. Christensen.  A new scattering method that combines roughness and diffraction effects , 2005 .

[45]  Thomas Funkhouser,et al.  A beam tracing method for interactive architectural acoustics. , 2004, The Journal of the Acoustical Society of America.

[46]  Per-Anders Forsberg.  Fully discrete ray tracing , 1985 .

[47]  Y. W. Lam,et al.  A comparison of three diffuse reflection modeling methods used in room acoustics computer models , 1996 .

[48]  John S. Bradley,et al.  The influence of late arriving energy on spatial impression , 1995 .

[49]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[50]  Sophia Antipolis,et al.  SCALABLE PERCEPTUAL MIXING AND FILTERING OF AUDIO SIGNALS USING AN AUGMENTED SPECTRAL REPRESENTATION , 2005 .

[51]  Durand R. Begault,et al.  Challenges to the Successful Implementation of 3-D Sound , 1991 .

[52]  Michael Vorländer,et al.  A Fast Reverberation Estimator for Virtual Environments , 2007 .

[53]  D. Ward.  On the performance of acoustic crosstalk cancellation in a reverberant environment , 2001 .

[54]  Tomas Akenine-Möller,et al.  Real-time rendering , 1997 .

[55]  Douglas S. Brungart,et al.  Auditory localization of a nearby point source , 1996 .

[56]  Renate Heinz.  Entwicklung und Beurteilung von computergestützten Methoden zur binauralen Raumsimulation , 1994 .

[57]  Bengt-Inge Dalenbäck.  Room acoustic prediction based on a unified treatment of diffuse and specular reflection , 1996 .

[58]  Joseph J. LaViola.  A testbed for studying and choosing predictive tracking algorithms in virtual environments , 2003 .

[59]  William G. Gardner,et al.  Efficient Convolution without Input/Output Delay , 1995 .

[60]  Tyrone L. Vincent,et al.  Three-Dimensional Motion and Structure Estimation Using Inertial Sensors and Computer Vision for Augmented Reality , 2002, Presence: Teleoperators & Virtual Environments.

[61]  Wulf Pompetzki.  Psychoakustische Verifikation von Computermodellen zur binauralen Raumsimulation , 1993 .

[62]  I. Bork,et al.  A comparison of room simulation software - The 2nd round robin on room acoustical computer simulation , 2000 .

[63]  Benjamin B. Bauer,et al.  Stereophonic Earphones and Binaural Loudspeakers , 1961 .