Diagnostic Models for Procedural Bugs in Basic Mathematical Skills

A new diagnostic modeling system for automatically synthesizing a deep-structure model of a student's misconceptions or bugs in his basic mathematical skills provides a mechanism for explaining why a student is making a mistake as opposed to simply identifying the mistake. This report is divided into four sections: The first provides examples of the problems that must be handled by a diagnostic model. It then introduces procedural networks as a general framework for representing the knowledge underlying a skill. The challenge in designing this representation is to find one that facilitates the discovery of misconceptions or bugs existing in a particular student's encoding of this knowledge. The second section discusses some of the pedagogical issues that have emerged from the use of diagnostic models within an instructional system. This discussion is framed in the context of a computer-based tutoring/gaming system developed to teach students and student teachers how to diagnose bugs strategically as well as how to provide a better understanding of the underlying structure of arithmetic skills. The third section describes our uses of an executable network as a tool for automatically diagnosing student behavior, for automatically generating “diagnostic” tests, and for judging the diagnostic quality of a given exam. Included in this section is a discussion of the success of this system in diagnosing 1300 school students from a data base of 20,000 test items. The last section discusses future research directions. Development of the general framework of Diagnostic Models which underlies this research was supported, in part, by the Advanced Research Projects Agency, Air Force Human Resources Laboratory, Army Research Institute for Behavioral and Social Sciences, and Navy Personnel Research and Development Center under Contract No. MDA903-76-C-0108.

[1]  John Seely Brown.  Aspects of a Theory for Automated Student Modelling. , 1977 .

[2]  L. S. Cox.  Systematic Errors in the Four Vertical Algorithms in Normal and Handicapped Populations. , 1975 .

[3]  Derek H. Sleeman,et al.  APRIL: a flexible production rule interpreter , 1977, SGAR.

[4]  John Seely Brown,et al.  Artificial Intelligence and Learning Strategies. , 1978 .

[5]  Tommie A. West.  Diagnosing Pupil Errors: Looking for Patterns. , 1971 .

[6]  J. A. Easley,et al.  Teaching by Listening--Toward a New Day in Math Classes. , 1975 .

[7]  W. Eric L. Grimson,et al.  Annotated Production Systems: A Model for Skill Acquisition , 1977, IJCAI.

[8]  Ira P. Goldstein.  Understanding simple picture programs , 1974 .

[9]  Z. P. Dienes,et al.  Learning Logic Logical Games , 1974 .

[10]  Barbara,et al.  The Radio Mathematics Project, Nicaragua, 1974-1975 , 1976 .

[11]  Ann D. Rubin,et al.  Hypothesis Formation and Evaluation in Medical Diagnosis , 1975 .

[12]  Ira P. Goldstein,et al.  Overview of a Linguistic Theory of Design , 1977 .

[13]  John Seely Brown,et al.  Semantic Grammar: A Technique for Constructing Natural Language Interfaces to Instructional Systems. , 1977 .

[14]  Allan M. Collins,et al.  Natural Semantics in Artificial Intelligence , 1973, IJCAI.

[15]  Allan Collins,et al.  Analysis and Synthesis of Tutorial Dialogues , 1975 .

[16]  Richard M. Young,et al.  Mixtures of strategies in structurally adaptive production systems: examples from seriation and subtraction , 1977, SGAR.

[17]  Ira P. Goldstein,et al.  The computer as coach: As athletic paradigm for intellectual education , 1977, ACM '77.

[18]  John Seely Brown,et al.  A Paradigmatic Example of an Artificially Intelligent Instructional System. , 1978 .

[19]  R. Krauss,et al.  Social and Nonsocial Speech , 1977 .

[20]  John Seely Brown,et al.  SYSTEMATIC UNDERSTANDING: Synthesis, Analysis, and Contingent Knowledge in Specialized Understanding Systems , 1975 .

[21]  Earl David Sacerdoti,et al.  A Structure for Plans and Behavior , 1977 .

[22]  John Seely Brown.  Reactive Learning Environment for Computer Assisted Electronics Instruction. , 1976 .

[23]  John A. Self,et al.  Student Models in Computer-Aided Instruction , 1974, Int. J. Man Mach. Stud..

[24]  Howard E. Shrobe,et al.  Initial Report on a Lisp Programmer's Apprentice , 1978, IEEE Transactions on Software Engineering.

[25]  Richard C. Atkinson,et al.  A Rationale and Description of the Basic Instructional Program , 1974 .

[26]  Patrick Suppes,et al.  The Nicaragua radio mathematics project. , 1975 .