Automatic adaptive speech separation using beamformer-output-ratio for voice activity classification

This paper focuses on the practical challenge of adaptation control for speech separation systems. Adaptive beamforming methods, such as minimum variance distortionless response (MDVR), can effectively extract the desired speech signal from interference and noise. However, to avoid the signal cancellation problem, the beamformer adaptation is halted when the desired speaker is active. An automated scheme for this adaptation requires classifying speakers' voice activity status, which remains a challenge for multi-speaker environments. In this paper, we propose a novel approach to identify voice activities for two speakers based on a new metric, called the beamformer-output-ratio (BOR). Statistical properties of the BOR are studied and used to develop a hypothesis-based method for voice activity classification. The method is further refined using an algorithm detecting incorrect beamformer adaptation by analysing changes in the output power of a blind adapting MVDR beamformer. Based on the new methods, we construct an automatic adaptive beamforming system to simultaneously separate speech for two speakers. The speech separation module of the system uses MVDR beamformers whose adaptation is guided by the voice activity classification. Our methods can lead to, in some cases, 20% reduction in voice activity classification error, and 8dB improvement on the output SINR. The results are verified on both synthesised signals and realistic recordings. HighlightsWe design an automated adaptive beamforming system to extract speech of two speakers.The quantity BOR and its roles in active speaker identification are introduced.The BOR-VAC method is developed, in both generic form and practical realisation.We model the beamformer output power behaviour to detect incorrect adaptation.The proposed systems are tested in both real and synthesised recordings.