An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification

In this paper, we describe a relation between classification systems and information transmission systems. By looking at the classification systems from this perspective, we propose a method of classifier weight estimation for the linear (LIN-OP) and logarithmic opinion pool (LOG-OP) type classifier combination schemes for which some tools from information theory are used. These weights provide contextual information about the classifiers such as class dependent classifier reliability and global classifier reliability. A measure for decision consensus among the classifiers is also proposed which is formulated as a multiplicative part of the classifier weights. A method of selecting the classifiers which provide complementary information for the combination operation is given. Using the proposed method, two classifiers are selected to be used in the combination operation. Simulation experiments in closed set speaker identification have shown that the method of weight estimation described in this paper improved the identification rates of both linear and logarithmic opinion type combination schemes. A comparison between the proposed method and some other methods of weight selection is also given at the end of the paper.

[1]  Jon Atli Benediktsson,et al.  Consensus theoretic classification methods , 1992, IEEE Trans. Syst. Man Cybern..

[2]  Kevin R. Farrell.  Text-dependent speaker verification using data fusion , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[3]  Pramod K. Varshney,et al.  An information theoretic approach to the distributed detection problem , 1989, IEEE Trans. Inf. Theory.

[4]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[5]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[6]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[7]  B. P. Lathi,et al.  Modern Digital and Analog Communication Systems , 1983 .

[8]  B. V. K. Vijaya Kumar,et al.  Unified decision combination framework , 1998, Pattern Recognit..

[9]  Ching Y. Suen,et al.  Optimal combinations of pattern classifiers , 1995, Pattern Recognit. Lett..

[10]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[11]  Douglas A. Reynolds,et al.  Robust text-independent speaker identification using Gaussian mixture speaker models , 1995, IEEE Trans. Speech Audio Process..

[12]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[13]  H. Gish,et al.  Text-independent speaker identification , 1994, IEEE Signal Processing Magazine.

[14]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Ke Chen,et al.  A method of combining multiple probabilistic classifiers through soft competition on different feature sets , 1998, Neurocomputing.

[16]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Horst Bunke,et al.  Lipreading: A classifier combination approach , 1997, Pattern Recognit. Lett..

[18]  Jr. J.P. Campbell,et al.  Speaker recognition: a tutorial , 1997, Proc. IEEE.

[19]  I. K. Sethi,et al.  Hierarchical Classifier Design Using Mutual Information , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Pramod K. Varshney,et al.  Distributed Detection and Data Fusion , 1996 .

[21]  Carmen García-Mateo,et al.  On the use of acoustic segmentation in speaker identification , 1997, EUROSPEECH.

[22]  Harry E. Stephanou,et al.  Measuring Consensus Effectiveness by a Generalized Entropy Criterion , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[25]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[26]  Roberto Brunelli,et al.  Person identification using multiple cues , 1995, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Sridha Sridharan,et al.  Telephone based speaker recognition using multiple binary classifier and Gaussian mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[28]  B. Atal.  Effectiveness of linear prediction characteristics of the speech wave for automatic speaker identification and verification. , 1974, The Journal of the Acoustical Society of America.

[29]  Frank K. Soong,et al.  On the use of instantaneous and transitional spectral information in speaker recognition , 1988, IEEE Trans. Acoust. Speech Signal Process..

[30]  Robert McEliece,et al.  The Theory of Information and Coding: Information theory , 2002 .

[31]  Isabelle Bloch.  Information combination operators for data fusion: a comparative review with classification , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[32]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[33]  Rui Zhang,et al.  Adaptive confidence transform based classifier combination for Chinese character recognition , 1998, Pattern Recognit. Lett..

[34]  Johan Lindberg,et al.  Guidelines for experiments on the POLYCOST database , 1996 .

[35]  Vlasta Radová,et al.  An approach to speaker identification using multiple classifiers , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[36]  Mübeccel Demirekler,et al.  On the use of supra model information from multiple classifiers for robust speaker identification , 1999, EUROSPEECH.

[37]  Kagan Tumer,et al.  Analysis of decision boundaries in linearly combined neural classifiers , 1996, Pattern Recognit..

[38]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[39]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[40]  Sadaoki Furui,et al.  Recent advances in speaker recognition , 1997, Pattern Recognit. Lett..