The Cognitive Tutor Authoring Tools (CTAT): Preliminary Evaluation of Efficiency Gains

Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.

[1]  Vincent Aleven,et al.  Improving Intercultural Competence by Predicting in French Film , 2005 .

[2]  Kenneth R. Koedinger,et al.  Toward a Rapid Development Environment for Cognitive Tutors , 2003 .

[3]  Antonija Mitrovic,et al.  A Knowledge Acquisition System for Constraint-based Intelligent Tutoring Systems , 2005, AIED.

[4]  Yeliz Eseryel.  Proceedings of E-LEARN 2002 , 2002 .

[5]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Five Years of Evaluations , 2005, AIED.

[6]  Neil T. Heffernan,et al.  Blending Assessment and Instructional Assisting , 2005, AIED.

[7]  Lars Bollen,et al.  Cognitive tutoring of collaboration: developmental and empirical steps towards realization , 2005, CSCL.

[8]  Kenneth R. Koedinger,et al.  Applying Programming by Demonstration in an Intelligent Authoring Tool for Cognitive Tutors , 2005 .

[9]  Neil T. Heffernan,et al.  Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration , 2004, Intelligent Tutoring Systems.

[10]  Carolyn Penstein Rosé,et al.  Authoring plug-in tutor agents by demonstration: Rapid, rapid tutor development , 2005, AIED.

[11]  Beverly Park Woolf,et al.  Building a Community Memory for Intelligent Tutoring Systems , 1987, AAAI.

[12]  Neil T. Heffernan,et al.  The eXtensible Tutor Architecture: A New Foundation for ITS , 2005, AIED.

[13]  Tom Murray,et al.  Authoring Tools for Advanced Technology Learning Environments , 2003 .

[14]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[15]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[16]  Kenneth R. Koedinger,et al.  Studying the Effects of Personalized Language and Worked Examples in the Context of a Web-Based Intelligent Tutor , 2006, Intelligent Tutoring Systems.

[17]  Shaaron Ainsworth,et al.  Evaluating a Mixed-Initiative Authoring Environment: Is REDEEM for Real? , 2005, AIED.

[18]  Tom Murray,et al.  Authoring Intelligent Tutoring Systems: An analysis of the state of the art , 1999 .