An Assessment of Constraint-Based Tutors: A Response to Mitrovic and Ohlsson's Critique of "A Comparison of Model-Tracing and Constraint-Based Intelligent Tutoring Paradigms"

Two approaches to building intelligent tutoring systems are the well-established model-tracing paradigm and the relatively newer constraint-based paradigm. Proponents of the constraint-based paradigm claim that it affords performance at levels comparable to that of model-tracing tutors, but with significantly less development effort. We have built both a model-tracing and constraint-based tutor for the same problem domain (statistical hypothesis testing) and report on our findings with the goals of evaluating proponents' claims, more generally contrasting and comparing the two approaches, and providing guidance for others interested in building intelligent tutoring systems. Principally we conclude that two characteristics of the problem domain are key in distinguishing the appropriateness of the approaches for a given problem domain. First, the constraint-based paradigm is feasible only for domains in which the solution itself is rich in information. There is no such restriction for model tracing. Second, model tracing demonstrates superiority with respect to the ability to provide targeted, high-quality remediation; this superiority increases with the complexity of the solution process goal structure. Finally, we observe that the development effort required to build a model-tracing tutor is greater than that for building a constraint-based tutor. This increased effort is a function of additional design requirements that are responsible for the improved remediation.