The Technology of Binaural Listening & Understanding : Paper ICA 2016-445 Exploiting envelope fluctuations to achieve robust extraction and intelligent integration of binaural cues

The human auditory system achieves remarkably robust communication performance, even in complex environments featuring multiple talkers, distracting noises, echoes, and reverberation. Although the neural mechanisms of this facility are not well understood, many studies point to the importance of binaural and spatial cues present at sound onset or during other fluctuations of the temporal envelope. Specifically, transient increases in the amplitude envelope appear to trigger the sampling of binaural information, independent of binaural-cue type or frequency range. This paper begins with a review of the psychophysical and neural evidence for such a triggering process, and an exploration of signal-processing algorithms that mimic and/or exploit that process. Such algorithms can be applied in two key directions of importance to communication acoustics: First, temporal envelopes are used to guide the strategic application of spatial cues in spatial sound synthesis for human listeners. Second, temporal fluctuations are used to guide the extraction of spatial cues from binaural recordings and intelligently group those cues into temporally and spatially coherent binaural proto-objects. These applications provide critical tests of the triggering hypothesis, the general role of temporal envelope fluctuations in binaural hearing, and the neural mechanisms of integrated binaural perception. Further, they provide powerful tools for the design of efficient audio communication systems and devices that interface with human participants in real or virtual spatial settings. Supported by NIH DC011548.