Solving the indeterminations of blind source separation of convolutive speech mixtures

Looking at the speaker's face seems useful for hearing a speech signal better and extracting it from competing sources before identification. We present a novel algorithm plugging the audiovisual coherence of speech signals, estimated by statistical tools, on audio blind source separation (BSS) algorithms in the difficult case of convolutive mixtures. The algorithm mainly works in the frequency (transform) domain, where the convolutive mixture becomes an additive mixture for each frequency channel. Frequency by frequency separation is made by an audio BSS algorithm, and the audiovisual information is used to solve the standard source permutation and scale factor problems at the output of the separation stage, for each frequency. The proposed method is shown to be efficient in the case of 2/spl times/2 convolutive mixtures.