Noise compensation in a multi-modal verification system

In this paper we propose an adaptive multi-modal verification system comprised of a modified minimum cost Bayesian classifier (MCBC) and a method to find the reliability of the speech expert for various noisy conditions. The modified MCBC takes into account the reliability of each modality expert, allowing the de-emphasis of the contribution of opinions from the expert affected by noise. Reliability of the speech expert is found without directly modeling the noisy speech or finding the reliability a priori for various conditions of the speech signal. Experiments on the digit database show the total error to be reduced by 78% when compared to a non-adaptive system.

[1]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[2]  Kuldip K. Paliwal,et al.  Adaptive Multi-Modal Person Verification System , 2000 .

[3]  Jen-Tzung Chien,et al.  A novel projection-based likelihood measure for noisy speech recognition , 1998, Speech Commun..

[4]  Brian Hanson,et al.  Regression features for recognition of speech in quiet and in noise , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[5]  E. Mayoraz,et al.  Fusion of face and speech data for person identity verification , 1999, IEEE Trans. Neural Networks.

[6]  Douglas A. Reynolds,et al.  Speaker identification and verification using Gaussian mixture speaker models , 1995, Speech Commun..

[7]  Kuldip K. Paliwal,et al.  Multi-modal person verification system based on face profiles and speech , 1999, ISSPA '99. Proceedings of the Fifth International Symposium on Signal Processing and its Applications (IEEE Cat. No.99EX359).

[8]  T. Moon.  The expectation-maximization algorithm , 1996, IEEE Signal Process. Mag..

[9]  Chin-Hui Lee,et al.  Speaker verification using normalized log-likelihood score , 1996, IEEE Trans. Speech Audio Process..