Speech Separation via Parallel Factor Analysis of Cross-Frequency Covariance Tensor

This paper considers separation of convolutive speech mixtures in frequency-domain within a tensorial framework. By assuming that components associated with neighboring frequency bins of the same source are still correlated, a set of cross-frequency covariance tensors with trilinear structure are established, and an algorithm consisting of consecutive parallel factor (PARAFAC) decompositions is developed. Each PARAFAC decompositon used in the proposed method can simultaneously estimate two neighboring frequency responses, one of which is a common factor with the subsequent crossfrequency covariance tensor, and thus could be used to align the permutations of the estimates in all the PARAFAC decompositions. In addition, the issue of identifiability is addressed, and simulations with synthetic speech signals are provided to verify the efficacy of the proposed method.