A Question-Answering System for AP Chemistry: Assessing KR&R Technologies

Basic research in knowledge representation and reasoning (KR&R) has steadily advanced over the years, but it has been difficult to assess the capability of fielded systems derived from this research. In this paper, we present a knowledge-based question-answering system that we developed as part of a broader effort by Vulcan Inc. to assess KR&R technologies, and the result of its assessment. The challenge problem presented significant new challenges for knowledge representation, compared with earlier such assessments, due to the wide variability of question types that the system was expected to answer. Our solution integrated several modern KR&R technologies, in particular semantically well-defined frame systems, automatic classification methods, reusable ontologies, a methodology for knowledge base construction, and a novel extension of methods for explanation generation. The resulting system exhibited high performance, achieving scores for both accuracy and explanation which were comparable to human performance on similar tests. While there are qualifications to this result, it is a significant achievement and an informative data point about the state of the art in KR&R, and reflects significant progress by the field.

[1]  Ronald J. Brachman,et al.  An Overview of the KL-ONE Knowledge Representation System , 1985, Cogn. Sci..

[2]  Kenneth D. Forbus,et al.  Using Qualitative Physics to Build Articulate Software for Thermodynamics Education: A Preliminary Report , 1994, Interact. Learn. Environ..

[3]  Peter Clark,et al.  A library of generic concepts for composing knowledge bases , 2001, K-CAP '01.

[4]  Deborah L. McGuinness,et al.  Infrastructure for Web Explanations , 2003, SEMWEB.

[5]  Theodore L. Brown Chemistry: The Central Science , 1981 .

[6]  Paul R. Cohen,et al.  Experimental Evaluation of Subject Matter Expert-oriented Knowledge Base Authoring Tools , 2002 .

[7]  Peter Clark,et al.  A web-based ontology browsing and editing system , 2002, AAAI/IAAI.

[8]  斉藤 康己,et al.  Douglas B. Lenat and R. V. Guha : Building Large Knowledge-Based Systems, Representation and Inference in the Cyc Project, Addison-Wesley (1990). , 1990 .

[9]  Peter Clark,et al.  Building Concept Representations from Reusable Components , 1997, AAAI/IAAI.

[10]  Steffen Staab,et al.  Towards a Quantitative, Platform-Independent Analysis of Knowledge Systems , 2004, KR.

[11]  Peter Clark,et al.  A Knowledge-Based Approach to Question-Answering , 1999 .

[12]  Paul R. Cohen,et al.  The DARPA High-Performance Knowledge Bases Project , 1998, AI Mag..

[13]  Gordon S. Novak Conversion of Units of Measurement , 1995, IEEE Trans. Software Eng..

[14]  Daniel G. Bobrow,et al.  On Overview of KRL, a Knowledge Representation Language , 1976, Cogn. Sci..

[15]  Bruce W. Porter,et al.  AI Research in the Context of a Multifunctional Knowledge Base: The BotanyKnowledge Base Project , 1988 .

[16]  Alan Bundy,et al.  MECHO: A program to solve mechanics problems , 1979 .

[17]  Ramanathan V. Guha,et al.  Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project , 1990 .

[18]  Yolanda Gil,et al.  An integrated environment for knowledge acquisition , 2001, IUI '01.

[19]  Dieter Fensel,et al.  Ontobroker: Ontology Based Access to Distributed and Semi-Structured Information , 1999, DS-8.

[20]  Gordon S. Novak Representations of Knowledge in a Program for Solving Physics Problems , 1977, IJCAI.

[21]  Peter Clark,et al.  Knowledge entry as the graphical assembly of components , 2001, K-CAP '01.

[22]  Gordon S. Novak,et al.  Understanding Natural Language with Diagrams , 1990, AAAI.

[23]  William R. Swartout,et al.  XPLAIN: A System for Creating and Explaining Expert Consulting Programs , 1983, Artif. Intell..

[24]  Jürgen Angele Operationalisierung des Modells der Expertise mit KARL , 1993, DISKI.

[25]  Michael Kifer,et al.  Logical foundations of object-oriented and frame-based languages , 1995, JACM.