Generalized Viterbi-based models for time-series segmentation and clustering applied to speaker diarization

Abstract Speaker diarization is a problem of separating unknown speakers in a conversation into homogeneous parts in the speaker sense. State-of-the-art diarization systems are based on i-vector methodologies. However, these approaches require large quantities of training data, which must be obtained from an environment that is similar to that of the conversation being diarized. In this paper we present a diarization system that does not require such training data but instead can suffice with some development data for parameter-tuning. This system is a generalization of the well-known hidden Markov model (HMM), a popular clustering algorithm trained by Viterbi statistics. Our proposed model, referred to as a hidden distortion model (HDM), is based on state distortion models and transition costs, for which probabilistic calculations are not mandatory, in contrast to the case of HMM. We provide a mathematical basis for our approach, and we demonstrate that Viterbi-based HMM can be seen as a special case of HDM. This proximity allows us to apply similar approaches for state-model training when the new paradigm is used to learn sequence dependencies. We carry out diarizations of two-speaker telephone conversations in order to evaluate the performance of HDM. When applied to conversations from the LDC CALLHOME database, HDM improves on the performance of a baseline HMM system by about 26% (relative improvement). Moreover, when applied to the NIST 2005 database, it yields a small improvement over the HMM system.