Assessing Speaker Engagement in 2-Person Debates: Overlap Detection in United States Presidential Debates

Co-channel speech recordings typically contain significant amounts of overlap in which the intelligibility and quality of the desired speech is degraded by interference from a competing talker. Convolutive Non-negative Matrix Factorization (CNMF) has been shown to be a successful approach in detecting overlap by extracting specific acoustic basis dimensions for each speaker from an audio stream. While the results of CNMF have been successful, it requires isolated single speech recordings for each speaker to derive their corresponding bases functions/dimensions. In our previous work, The Teager-Kaiser Energy Operator (TEO)-based Pyknogram has been introduced which does not require prior information concerning the speakers. In this study, Pyknogram and CNMF based solutions for overlap detection within audio streams have been examined using the GRID dataset. TEO-based Pyknogram is shown to achieve a relative 8-10% lower Equal Error Rate (EER) compared to CNMF features. Another drawback of CNMF is that its performance drops considerably when dealing with spontaneous speech that has not been considered for extracting bases in the training step. In addition to the experiments on GRID corpus, a secondary evaluation is also performed based on naturalistic audio streams with overlap. Specifically, we collected a real-world audio database of US Presidential debates stemming from the last 12 years that are challenging due to overlap, changing Signal to Interference Ratio (SIR), and environmental noise, etc. Our experiments indicate that TEO-based Pyknogram is well suited for detecting overlap in challenging real world scenarios such as the US presidential debates.