Evaluation and Assessment of a Domain-Independent Natural Language Query System

This paper is a brief description of TEAM, a project whose goal was to design an experimental natural-language interface that could be transported to existing database systems by people who already possessed expertise in their use. In presenting this overview, we have concentrated on those design aspects that were most constrained by the requirements of transportability. 1 A Functional Description A natural-language interface (NLI) to a computer database provides users with the capability of obtaining information stored in the database by querying the system in a natural language (e.g., English). The use of natural languages as a means of communication with computer systems allows users to frame a question or a statement in the way they think about the information being discussed, thereby freeing them from the need to know how the computer stores or processes the information. However, most existing NLI systems have been designed specifically to treat queries that are constrained in three ways: (1) they concern a single application domain; (2) they pertain to information in a single database; (3) they handle only a single task, namely, database query.’ Constructing a system for a new domain or database requires a new effort almost equal to the original one in magnitude. Transportable NLIs that can easily be adapted to new domains or databases are potentially much more useful than domainor database-specific systems. However, because many of the tech niques already developed for custom-built systems preclude automatic adaptation of the systems to new domains, the construction of transportable systems poses a number of technical and theo retical problems. In describing the transportable NLI system called TEAM (Transportable English database Access Medium), that was the focus and objective of a four-year project, this article em phasizes those choices in system design imposed by the requirement of transportability.2 For some problems, the design decisions incorporated in TEAM are generally applicable to a wider range of natural-language processing systems; for others, we were forced to take a more limited approach. 1.1 Transportability One of the major challenges faced in building NLIs is to provide the information needed by the system to bridge the gap between the way the user thinks about the domain of discourse and the way the computer handles the information it possesses about the domain. Existing databases employ ‘This constraint is more limiting in many ways than the other two. For example, queries are typically treated largely in isolation; very few features of dialogue are handled. Since this remains a constraint in TEAM it will not be discussed further in this article. 2Space limitations have compelled us to omit many of the specific problems faced in this research; for a fuller treatment, please see the journal article tGros85I.