Three Issues in Modern Language Modeling

In this paper we discuss three issues in modern language modeling. The first one is the question of a quality measure for language models, the second is language model smoothing and the third is the question of how to build good long-range language models. In all three cases some results are given indicating possible directions of further research.

[1]  Harry Printz.  Fast computation of maximum entropy / minimum divergence feature gain , 1998, ICSLP.

[2]  Ronald Rosenfeld,et al.  A maximum entropy approach to adaptive statistical language modelling , 1996, Comput. Speech Lang..

[3]  Mari Ostendorf,et al.  Analyzing and predicting language model improvements , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[4]  Eugene Charniak,et al.  Immediate-Head Parsing for Language Models , 2001, ACL.

[5]  G. Zipf,et al.  The Psycho-Biology of Language , 1936 .

[6]  Mari Ostendorf,et al.  A new metric for stochastic language model evaluation , 1999, EUROSPEECH.

[7]  Ronald Rosenfeld,et al.  Topic adaptation for language modeling using unnormalized exponential models , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[8]  Frederick Jelinek,et al.  Exploiting Syntactic Structure for Language Modeling , 1998, ACL.

[9]  Dietrich Klakow,et al.  Log-linear interpolation of language models , 1998, ICSLP.

[10]  Stanley F. Chen,et al.  Evaluation Metrics For Language Models , 1998 .

[11]  Hermann Ney,et al.  On the Use of Grammar Based Language Models for Statistical Machine Translation , 2000, IWPT.

[12]  K. A. Semendyayev,et al.  Handbook of mathematics , 1985 .

[13]  Philip Clarkson,et al.  Towards improved language model evaluation measures , 1999, EUROSPEECH.

[14]  Hermann Ney,et al.  Improved backing-off for M-gram language modeling , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[15]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[16]  Dietrich Klakow,et al.  Testing the correlation of word error rate and perplexity , 2002, Speech Commun..

[17]  Dietrich Klakow,et al.  Language model adaptation using dynamic marginals , 1997, EUROSPEECH.

[18]  Peder A. Olsen,et al.  Theory and practice of acoustic confusability , 2002, Comput. Speech Lang..

[19]  Christoph Neukirchen,et al.  Generation and expansion of word graphs using long span context information , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).