1997 BROADCAST NEWS BENCHMARK TEST RESULTS: ENGLISH AND NON-ENGLISH

This paper documents use of Broadcast News test materials in DARPA-sponsored Automatic Speech Recognition (ASR) Benchmark Tests conducted late in 1997. This year's English-language tests differed from last year's in that statistical selection procedures were used in selecting a three-hour test set comprised of 158 story-length segments, in contrast to last year's two-hour test set which was comprised of 4 half-hour segments. The increased number of segments is intended to provide a better statistical sampling of story-length segments and a statistically-equivalent reserve test set for a future evaluation. The lowest word-error rate reported this year was 16.2%, contrasting with last year's lowest word error rate of 27.1%. In part, this apparent im provement is due to the much greater proportion of well-recognized F0 data present in the test set. This, in turn, is due to an effort to "balance" the test pool to match the properties of the training data. New this year was the completion of tests in languages other than English Mandarin and Spanish.