Benchmark Tests for the DARPA Spoken Language Program

This paper documents benchmark tests implemented within the DARPA Spoken Language Program during the period November, 1992 - January, 1993. Tests were conducted using the Wall Street Journal-based Continuous Speech Recognition (WSJ-CSR) corpus and the Air Travel Information System (ATIS) corpus collected by the Multi-site ATIS Data COllection Working (MADCOW) Group. The WSJ-CSR tests consist of tests of large vocabulary (lexicons of 5,000 to more than 20,000 words) continuous speech recognition systems. The ATIS tests consist of tests of (1) ATIS-domain spontaneous speech (lexicons typically less than 2,000 words), (2) natural language understanding, and (3) spoken language understanding. These tests were reported on and discussed in detail at the Spoken Language Systems Technology Workshop held at the Massachusetts Institute of Technology, January 20-22, 1993.

[1]  Mei-Yuh Hwang,et al.  An Overview of the SPHINX-II Speech Recognition System , 1993, HLT.

[2]  Mitch Weintraub,et al.  Progressive-Search Algorithms For Large-Vocabulary Speech Recognition , 1993, HLT.

[3]  Lewis M. Norton,et al.  A Portable Approach To Last Resort Parsing And Interpretation , 1993, HLT.

[4]  Alexander I. Rudnicky,et al.  Multi-Site Data Collection and Evaluation in Spoken Language Understanding , 1993, HLT.

[5]  David S Pallet.  Performance assessment of automatic speech recognizers , 1985 .

[6]  Richard M. Stern,et al.  Efficient Cepstral Normalization for Robust Speech Recognition , 1993, HLT.

[7]  Min-Shiang Hwang,et al.  An Improved Search Algorithm for Continuous Speech Rec-ognition , 1993 .

[8]  Mei-Yuh Hwang,et al.  An improved search algorithm using incremental knowledge for continuous speech recognition , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Jonathan G. Fiscus,et al.  DARPA February 1992 ATIS Benchmark Test Results , 1992, HLT.

[10]  Mei-Yuh Hwang,et al.  The SPHINX-II speech recognition system: an overview , 1993, Comput. Speech Lang..

[11]  Hermann Ney,et al.  Large vocabulary continuous speech recognition of Wall Street Journal data , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Mitch Weintraub,et al.  Large-vocabulary dictation using SRI's DECIPHER speech recognition system: progressive search techniques , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[13]  Pascale Fung,et al.  The BBN/HARC spoken language understanding system , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  David S. Pallett.  DARPA February 1992 Pilot Corpus CSR "Dry Run" Benchmark Test Results , 1992, HLT.

[15]  Pascale Fung,et al.  Design and performance of HARC, the BBN spoken language understanding system , 1992, ICSLP.

[16]  George Zavaliagkos,et al.  Comparative Experiments on Large Vocabulary Speech Recognition , 1993, HLT.

[17]  Douglas B. Paul,et al.  The Lincoln large-vocabulary stack-decoder HMM CSR , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Robert C. Moore,et al.  Gemini: a natural language system for spoken-language understanding , 1993 .

[19]  Barry N. Taylor,et al.  Guidelines for Evaluating and Expressing the Uncertainty of Nist Measurement Results , 2017 .