Semantic Evaluation for Spoken-Language Systems

Development has begun on a semantic evaluation (SemEval) methodology and infrastructure for the ARPA Spoken Language Program. SemEval is an attempt to define a task-independent technology-based evaluation for language-understanding systems consisting of three parts: word-sense identification, predicate-argument structure determination, and identification of coreference relations. An initial spoken-language SemEval on ATIS data is planned for November/December 1994, concurrent with the next ATIS CAS (database answer) evaluation.