Comparing perceived auditory width to the visual image of a performing ensemble in contrasting bi-modal environments.

Despite many studies investigating auditory spatial impressions in rooms, few have addressed the impact of simultaneous visual cues on localization and the perception of spaciousness. The current research presents an immersive audiovisual environment in which participants were instructed to make auditory width judgments in dynamic bi-modal settings. The results of these psychophysical tests suggest the importance of congruent audio visual presentation to the ecological interpretation of an auditory scene. Supporting data were accumulated in five rooms of ascending volumes and varying reverberation times. Participants were given an audiovisual matching test in which they were instructed to pan the auditory width of a performing ensemble to a varying set of audio and visual cues in rooms. Results show that both auditory and visual factors affect the collected responses and that the two sensory modalities coincide in distinct interactions. The greatest differences between the panned audio stimuli given a fixed visual width were found in the physical space with the largest volume and the greatest source distance. These results suggest, in this specific instance, a predominance of auditory cues in the spatial analysis of the bi-modal scene.

[1]  C. Spence,et al.  Perceptual effects of cross-modal stimulation , 2022 .

[2]  Jonas Braasch,et al.  A Loudspeaker-Based Projection Technique for Spatial Music Applications Using Virtual Microphone Control , 2008, Computer Music Journal.

[3]  Francis Rumsey,et al.  Spatial quality evaluation for reproduced sound: terminology, meaning and a scene-based paradigm , 2002 .

[4]  J Blauert,et al.  Auditory spaciousness: some further psychoacoustic analyses. , 1986, The Journal of the Acoustical Society of America.

[5]  B. Stein,et al.  The Merging of the Senses , 1993 .

[6]  Jonas Braasch,et al.  Subjective Expectation Adjustments of Early-to-Late Reverberant Energy Ratio and Reverberation Time to Match Visual Environmental Cues of a Musical Performance , 2008 .

[7]  A. H. Marshall,et al.  Spatial impression due to early lateral reflections in concert halls: The derivation of a physical measure , 1981 .

[8]  Densil Cabrera,et al.  Auditory Versus Visual Spatial Impression: A Study of Two Auditoria , 2004, ICAD.

[9]  G. A. Soulodre,et al.  Objective measures of listener envelopment , 1995 .

[10]  John S. Bradley,et al.  The influence of late arriving energy on spatial impression , 1995 .

[11]  Jonas Braasch,et al.  Subjective scaling of spatial room acoustic parameters influenced by visual environmental cues. , 2010, The Journal of the Acoustical Society of America.

[12]  AUDITORY-VISUAL INTERACTION IN REAL AND VIRTUAL ROOMS , 2002 .

[13]  Leo L. Beranek,et al.  Interaural cross‐correlation, lateral fraction, and low‐ and high‐frequency sound levels as measures of acoustical quality in concert halls , 1995 .

[14]  Paul T. Calamia,et al.  Cross‐modal perception of room acoustics , 2006 .

[15]  W R Thurlow,et al.  Certain Determinants of the “Ventriloquism Effect” , 1973, Perceptual and motor skills.

[16]  P. Bertelson,et al.  The ventriloquist effect does not depend on the direction of deliberate visual attention , 2000, Perception & psychophysics.

[17]  M. Barron The subjective effects of first reflections in concert halls—The need for lateral reflections , 1971 .

[18]  A. H. Marshall,et al.  A note on the importance of room cross-section in concert halls , 1967 .