Assessing the Impact of Positive Feedback in Constraint-Based Tutors

Most existing Intelligent Tutoring Systems (ITSs) are built around cognitive learning theories, such as Ohlsson's theory of learning from performance errors and Anderson's ACT theories of skill acquisition, which focus primarily on providing negative feedback, facilitating learning by correcting errors. Research into the behavior of expert tutors suggest that experienced tutors use positive feedback quite extensively and successfully. This paper investigates positive feedback; learning by capturing and responding to correct behavior, supported by cognitive learning theories. Our aim is to develop and implement a systematic approach to delivering positive feedback in ITSs. We report on an evaluation study done in the context of SQL-Tutor, in which the control group used the original version of the system giving only negative feedback, while the experimental group received both negative and positive feedback. Results show that the experimental group students needed significantly less time to solve the same number of problems, in fewer attempts compared to those in the control group. Students in the experimental group also learn approximately the same number of concepts as students in the control group, but in much less time. This indicates that positive feedback facilitates learning and improves the effectiveness of learning in ITSs.

[1]  Dolores Karolick,et al.  Cognitive Styles and Feedback in Computer-Assisted Instruction , 1995 .

[2]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[3]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[4]  Xiaoming Wang,et al.  Chapter 8 , 2003, The Sermons and Liturgy of Saint James.

[5]  Patricia Charlton,et al.  Conference on Artificial Intelligence in Education , 2009 .

[6]  C. McGuire,et al.  Office of Educational Research and Improvement , 1999 .

[7]  D. Gilman.  Comparison of Several Feedback Methods for Correcting Errors by Computer-Assisted Instruction. , 1969 .

[8]  Bert Bredeweg,et al.  Student Modelling: The Key to Individualized Knowledge-Based Instruction , 2010, NATO ASI Series.

[9]  Kurt VanLehn,et al.  A model of the self-explanation effect. , 1992 .

[10]  Mia Stern,et al.  Applications of AI in education , 1996, CROS.

[11]  Ramez Elmasri,et al.  Fundamentals of Database Systems , 1989 .

[12]  Susanne P. Lajoie,et al.  Apprenticeship training in the workplace: computer-coached practice environment as a new form of apprenticeship , 1989 .

[13]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[14]  Peter Brusilovsky,et al.  Creating More Versatile Intelligent Learning Environments with a Component-Based Architecture , 1998, Intelligent Tutoring Systems.

[15]  Antonija Mitrovic,et al.  An Intelligent SQL Tutor on the Web , 2003, Int. J. Artif. Intell. Educ..

[16]  Susanne Narciss,et al.  How to design informative tutoring feedback for multi-media learning , 2004 .

[17]  Antonija Mitrovic,et al.  A Knowledge-Based Teaching System for SQL , 1998 .

[18]  Antonija Mitrovic,et al.  Using Learning Curves to Mine Student Models , 2005, User Modeling.

[19]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[20]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[21]  Ramez Elmasri,et al.  Fundamentals of Database Systems, 5th Edition , 2006 .

[22]  Pierre Tchounikine,et al.  Modelling the Knowledge-Based Components of a Learning Environment within the Task/Method Paradigm , 1998, Intelligent Tutoring Systems.

[23]  Antonija Mitrovic,et al.  Evaluation of a Constraint-Based Tutor for a Database Language , 1999 .

[24]  Nelishia Pillay.  Developing intelligent programming tutors for novice programmers , 2003, SGCS.

[25]  Arthur C. Graesser,et al.  Teaching Tactics and Dialog in AutoTutor , 2001 .

[26]  Alenka Kavcic.  Dealing with Uncertainty of User Knowledge : Fuzzy User Modelling in Educational Hypermedia , 1998 .

[27]  Chen-Lin C. Kulik,et al.  Timing of Feedback and Verbal Learning , 1988 .

[28]  Antonija Mitrovic,et al.  Feedback Micro-engineering in EER-Tutor , 2005, AIED.

[29]  B. F. Skinner.  The technology of teaching / B.F. Skinner , 1968 .

[30]  R. W. Kulhavy,et al.  Feedback in written instruction: The place of response certitude , 1989 .

[31]  Neil T. Heffernan,et al.  Intelligent tutoring systems have forgotten the tutor: adding a cognitive model of human tutors , 1998, CHI Conference Summary.

[32]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[33]  Claude Frasson,et al.  An Actor Based Architecture for Intelligent Tutoring Systems , 1996, Intelligent Tutoring Systems.

[34]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[35]  Neil T. Heffernan,et al.  An Intelligent Tutoring System Incorporating a Model of an Experienced Human Tutor , 2002, Intelligent Tutoring Systems.

[36]  Lael J. Schooler,et al.  The Disruptive Potential of Immediate Feedback , 2008 .

[37]  Richard E. Mayer,et al.  Meaningful Design for Meaningful Learning: Applying Cognitive Theory to Multimedia Explanations , 2000 .

[38]  Roger Nkambou,et al.  Designing intelligent tutoring systems: a multiagent Planning Approach , 2001, SCOU.

[39]  John R. Anderson.  The Architecture of Cognition , 1983 .

[40]  Moffat Mathews.  Investigating the Effectiveness of Problem Templates on Learning in Intelligent Tutoring Systems BSc Honours Report , 2006 .

[41]  Davide Fossati,et al.  Beyond the code-and-count analysis of tutoring dialogues , 2007, AIED.

[42]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[43]  Roger Nkambou,et al.  Integrating WWW resources in an intelligent tutoring system , 1996 .

[44]  Stellan Ohlsson,et al.  Constraint-Based Student Modeling , 1994 .

[45]  Rosemary Luckin,et al.  Artificial Intelligence in Education, Building Technology Rich Learning Contexts That Work, Proceedings of the 13th International Conference on Artificial Intelligence in Education, AIED 2007, July 9-13, 2007, Los Angeles, California, USA , 2007, AIED.

[46]  Kurt VanLehn,et al.  Andes: A Coached Problem Solving Environment for Physics , 2000, Intelligent Tutoring Systems.

[47]  A. Paivio.  Mental Representations: A Dual Coding Approach , 1986 .

[48]  Stellan Ohlsson,et al.  Positive and negative verbal feedback for Intelligent Tutoring Systems , 2005, AIED.

[49]  Kurt VanLehn,et al.  Andes: An Intelligent Tutor for Classical Physics , 2000 .

[50]  Tanja Mitrovic,et al.  Constraint-based tutors: a success story , 2001, AIED.

[51]  Reva Freedman.  Atlas: A Plan Manager for Mixed-Initiative, Multimodal Dialogue , 1999 .

[52]  Richard E. Mayer,et al.  A Cognitive Theory of Multimedia Learning: Implications for Design Principles , 2001 .

[53]  Kenneth R. Koedinger,et al.  Intelligent Cognitive Tutors as Modeling Tool and Instructional Model , 1998 .

[54]  R. Mayer.  Cognitive Theory and the Design of Multimedia Instruction: An Example of the Two‐Way Street Between Cognition and Instruction , 2002 .

[55]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[56]  Stellan Ohlsson,et al.  Learning from Performance Errors. , 1996 .

[57]  Claus Zinn,et al.  Modeling Human Teaching Tactics in a Computer Tutor , 2000 .

[58]  Xin Lu,et al.  Expert Tutoring and Natural Language Feedback in Intelligent Tutoring Systems , 2006, ICCE.