Two Sessions were devoted to introducing the DARPA SLS Common Task: the Air Travel Information System (ATIS). The first of these (in the late afternoon) provided an overview of the ATIS task and presented a summary of results reported to NIST in the first of a projected series of ATIS benchmark tests, while the second (immediately after dinner) served to describe different approaches in implementing the common task at several sites: BBN, CMU, MIT/LCS, SRI and Unisys. The two sessions were filled with a sense of excitement and appreciation of the precedent-setting nature of these tests, and revealed strong differences of opinion over details of implementation of the common task and implementation of the test, as well as frustration with the limitations on available time and the amount of ATIS domain material at the present time.
[1]
Lewis M. Norton,et al.
Beyond Class A: A Proposal for Automatic Evaluation of Discourse
,
1990,
HLT.
[2]
Madeleine Bates,et al.
A Proposal for SLS Evaluation
,
1989,
HLT.
[3]
George R. Doddington,et al.
The ATIS Spoken Language Systems Pilot Corpus
,
1990,
HLT.
[4]
P. J. Price,et al.
Evaluation of Spoken Language Systems: the ATIS Domain
,
1990,
HLT.
[5]
Jonathan G. Fiscus,et al.
DARPA ATIS Test Results June 1990
,
1990,
HLT.
[6]
John Makhoul,et al.
Developing an Evaluation Methodology for Spoken Language Systems
,
1990,
HLT.