Impact of Web based language modeling on speech understanding

Data sparseness in building statistical language models for spoken dialog systems is a critical problem. In a previous paper we addressed this issue by exploiting the World Wide Web (WWW) and other external data sources in a financial transaction domain. In this paper, we evaluate the impact of improved speech recognition due to Web-based language model (WebLM) on the speech understanding performance in a new domain. As speech understanding system we use a natural language call-routing system. Experimental results show that the WebLM improves the speech recognition performance by 1.7% to 2.7% across varying amounts of in-domain data. The improvements in action classification (AC) performance were modest yet consistent ranging from 0.3% to 0.8%

[1]  Ronald Rosenfeld,et al.  Improving trigram language modeling with the World Wide Web , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[2]  Ronald Rosenfeld,et al.  A survey of smoothing techniques for ME models , 2000, IEEE Trans. Speech Audio Process..

[3]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[4]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Alexander I. Rudnicky.  Language Modeling with Limited Domain Data , 1995 .

[6]  Frank Keller,et al.  The Web as a Baseline: Evaluating the Performance of Unsupervised Web-based Models for a Range of NLP Tasks , 2004, NAACL.

[7]  Geoffrey Zweig,et al.  Toward domain-independent conversational speech recognition , 2003, INTERSPEECH.

[8]  R. Rosenfeld,et al.  Two decades of statistical language modeling: where do we go from here? , 2000, Proceedings of the IEEE.

[9]  Cheng Wu,et al.  Language model estimation for optimizing end-to-end performance of a natural language call routing system , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[10]  Andreas Stolcke,et al.  Getting More Mileage from Web Text Sources for Conversational Speech Language Modeling using Class-Dependent Mixtures , 2003, NAACL.

[11]  Ruhi Sarikaya,et al.  Rapid language model development using external resources for new spoken dialog domains , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[12]  Robert Miller,et al.  Just-in-time language modelling , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).