论文信息 - A Comparison of Criteria for Maximum Entropy/ Minimum Divergence Feature Selection

A Comparison of Criteria for Maximum Entropy/ Minimum Divergence Feature Selection

In this paper we study the gain, a naturally-arising statistic from the theory of MEMD modeling [2], as a figure of merit for selecting features for an MEMD language model. We compare the gain with two popular alternatives-empirical activation and mutual information-and argue that the gain is the preferred statistic, on the grounds that it directly measures a feature's contribution to improving upon the base modeL

Adam L. Berger | Harry Printz

[1] Andreas Stolcke,et al. Structure and performance of a dependency language model , 1997, EUROSPEECH.

[2] David M. Magerman. Natural Language Parsing as Statistical Pattern Recognition , 1994, ArXiv.

[3] Ronald Rosenfeld,et al. Adaptive Statistical Language Modeling; A Maximum Entropy Approach , 1994 .

[4] John D. Lafferty,et al. Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5] John D. Lafferty,et al. A Model of Lexical Attraction and Repulsion , 1997, ACL.

[6] Adwait Ratnaparkhi,et al. A Maximum Entropy Model for Prepositional Phrase Attachment , 1994, HLT.

[7] Adam L. Berger,et al. A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[8] Ronald Rosenfeld,et al. Trigger-based language models: a maximum entropy approach , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9] Salim Roukos,et al. MDI adaptation of language models across corpora , 1997, EUROSPEECH.

[10] Harry Printz. Fast computation of maximum entropy / minimum divergence feature gain , 1998, ICSLP.

[11] John D. Lafferty,et al. Inference and Estimation of a Long-Range Trigram Model , 1994, ICGI.