Fuzzy integrals for the aggregation of confidence measures in speech recognition

This paper presents a study on merging confidence measures using fuzzy logic. Instead of the previous approaches using the notion of probability, we propose to observe the uncertainty of the recognition hypotheses and the notion of possibility thanks to fuzzy reasoning. Four different confidence measures are developed, coming from different parts of a speech recognizer. Various merging methods are studied to improve the performance of the confidence measures. The methods are evaluated in terms of Confidence Error Rate (CER) and in terms of their Detection Error Tradeoff (DET) curves on a French broadcast news corpus. They are compared to some fuzzy logic aggregation techniques among which the technique based on the Choquet Integral yields to a significant improvement in terms of CER.

[1]  Jean-Luc Marichal,et al.  Aggregation of interacting criteria by means of the discrete Choquet integral , 2002 .

[2]  Paul Deléglise,et al.  Automatic Detection of Well Recognized Words in Automatic Speech Transcriptions , 2006, LREC.

[3]  Gunnar Evermann,et al.  Large vocabulary decoding and confidence estimation using word posterior probabilities , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[4]  Michel Grabisch,et al.  Classification by fuzzy integral: performance and tests , 1994, CVPR 1994.

[5]  Paul Deléglise,et al.  The LIUM speech transcription system: a CMU Sphinx III-based system for French broadcast news , 2005, INTERSPEECH.

[6]  José Bernardo Mariño Acebal,et al.  Fuzzy reasoning in confidence evaluation of speech recognition , 1999 .

[7]  Andreas Stolcke,et al.  Finding consensus in speech recognition: word error minimization and other applications of confusion networks , 2000, Comput. Speech Lang..

[8]  Alvin F. Martin,et al.  The DET curve in assessment of detection task performance , 1997, EUROSPEECH.

[9]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[10]  Wayne H. Ward,et al.  Confidence measures for spoken dialogue systems , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[11]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decisionmaking , 1988, IEEE Trans. Syst. Man Cybern..

[12]  J. Mendel.  Fuzzy logic systems for engineering: a tutorial , 1995, Proc. IEEE.

[13]  Gunnar Evermann,et al.  Posterior probability decoding, confidence estimation and system combination , 2000 .

[14]  G. Choquet.  Theory of capacities , 1954 .

[15]  D. Dubois,et al.  On Possibility/Probability Transformations , 1993 .

[16]  L. Shapley.  A Value for n-person Games , 1988 .

[17]  Rong Zhang,et al.  Word level confidence annotation using combinations of features , 2001, INTERSPEECH.

[18]  John W. Seaman,et al.  The efficacy of fuzzy representations of uncertainty , 1994, IEEE Trans. Fuzzy Syst..

[19]  M. Grabisch.  The application of fuzzy integrals in multicriteria decision making , 1996 .

[20]  Guillaume Gravier,et al.  The ESTER phase II evaluation campaign for the rich transcription of French broadcast news , 2005, INTERSPEECH.

[21]  Hermann Ney,et al.  Unsupervised training of acoustic models for large vocabulary continuous speech recognition , 2005, IEEE Transactions on Speech and Audio Processing.

[22]  Ralf Schlüter,et al.  Using word probabilities as confidence measures , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[23]  Michel Grabisch,et al.  A new algorithm for identifying fuzzy measures and its application to pattern recognition , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[24]  M. Sugeno,et al.  A theory of fuzzy measures: Representations, the Choquet integral, and null sets , 1991 .

[25]  Hui Jiang,et al.  Confidence measures for speech recognition: A survey , 2005, Speech Commun..

[26]  Delphine Charlet,et al.  On combining confidence measures for improved rejection of incorrect data , 2001, INTERSPEECH.

[27]  Hagen Soltau,et al.  Confidence measure based language identification , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).