Simulation and Auralization of Concert Halls / Opera Houses: Paper ISMRA2016-14 Augmented auralization: Complementing auralizations with immersive virtual reality technologies

A framework conceived and designed to enable ecological assessments of theater, concert hall, and auditorium acoustics is presented. Coupling real-time convolution based auralization and 3D visualization with the inclusion of a 3D audio-visual (3D-AV) recorded performance provides for an animated visual anchor, allowing for appropriate orientation and distance variations perception, improving the sense of presence in the simulation. Auralizations are rendered either via an Ambisonic speaker array or binaural headphones using a Max/MSP audio engine. The 3D visualization is handled by BlenderVR, an open-source framework for Virtual Reality (VR). The framework is designed to facilitate multi-platform operation and seamless porting of the rendering over Head-Mounted Display (HMD), a portable one-projector CAVE system, or any other VR architecture. A use-case is presented with the historic Théâtre de l’Athénée in Paris. The acoustics from various seats are auralized using High-Order Ambisonic room impulse responses calculated using the Geometrical Acoustics (GA) software CATT-Acoustic on a GA model of the theater validated in a previous study. A 3D-AV theatrical performance is filmed with a depth sensor (Kinect 2), the resulting volumetric video is encrusted on the virtual stage allowing the same performance to be played in different room configurations without the need to construct and animate CGI avatars. Diverse application scenarios are discussed. The tools developed to support this framework are Open Source to promote research and development in augmented auralization.