Automatic modeling of procedural knowledge and feedback generation in a computer science tutoring system

This research takes place in the larger context of the study of one-on-one tutoring, a form of instruction that has been shown to be very effective. I conducted a study of human tutoring in the domain of Computer Science data structures, to understand which features and strategies of human tutoring are important for learning. I developed an Intelligent Tutoring System, iList, that helps students learn linked lists. One of the main advances in iList is the presence of a Procedural Knowledge Model automatically extracted from student data. This model allows iList to provide effective reactive and proactive procedural feedback while a student is solving a problem. I tested five different versions of iList, differing in the level of feedback they can provide, in multiple classrooms, with a total of more than 200 students. The evaluation study showed that iList is effective in helping students learn; students liked working with the system; and the feedback generated by the most sophisticated versions of the system is helpful in keeping the students on the right path.

[1]  Antonija Mitrovic,et al.  DB-Suite: Experiences with Three Intelligent, Web-Based Database Tutors. , 2004 .

[2]  Trude Heift,et al.  Error-specific and individualised feedback in a Web-based language tutoring system: Do they read it? , 2001, ReCALL.

[3]  Frantisek Franek,et al.  A Prototype for an Intelligent Tutoring System for Students Learning to Program in Java , 2004 .

[4]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[5]  A. Graesser,et al.  LEARNING WHILE HOLDING A CONVERSATION WITH A COMPUTER , 2005 .

[6]  Jeannette M. Wing.  Computational thinking and thinking about computing , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[7]  Chi-Jen Lin,et al.  An approach to developing computational supports for reciprocal tutoring , 2002, Knowl. Based Syst..

[8]  Valerie J. Shute,et al.  Intelligent Tutoring Systems: Past, Present, and Future. , 1994 .

[9]  Takashi Yamauchi,et al.  Learning from human tutoring , 2001, Cogn. Sci..

[10]  Stellan Ohlsson,et al.  Expert vs. Non-expert Tutoring: Dialogue Moves, Interaction Patterns and Multi-utterance Turns , 2009, CICLing.

[11]  Davide Fossati,et al.  Supporting Computer Science Curriculum: Exploring and Learning Linked Lists with iList , 2009, IEEE Transactions on Learning Technologies.

[12]  M. Lepper,et al.  Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. , 1993 .

[13]  Davide Fossati,et al.  Be Brief, And They Shall Learn: Generating Concise Language Feedback for a Computer Tutor , 2008, Int. J. Artif. Intell. Educ..

[14]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[15]  M. Weaver,et al.  Do students value feedback? Student perceptions of tutors’ written responses , 2006 .

[16]  Davide Fossati,et al.  The Role of Positive Feedback in Intelligent Tutoring Systems , 2008, ACL.

[17]  Stephen B. Gilbert,et al.  Evaluating an Authoring Tool for Model-Tracing Intelligent Tutoring Systems , 2008, Intelligent Tutoring Systems.

[18]  Vincent Aleven,et al.  Limitations of Student Control: Do Students Know When They Need Help? , 2000, Intelligent Tutoring Systems.

[19]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[20]  Sandra Katz,et al.  Gender and race in predicting achievement in computer science , 2003, IEEE Technol. Soc. Mag..

[21]  Ray C. Fair,et al.  Principles of economics , 1993 .

[22]  Kenneth R. Koedinger,et al.  A Response Time Model For Bottom-Out Hints as Worked Examples , 2008, EDM.

[23]  Stellan Ohlsson,et al.  Computational Models of Skill Acquisition , 2008 .

[24]  Ehud Reiter,et al.  An Architecture for Data-to-Text Systems , 2007, ENLG.

[25]  Davide Fossati,et al.  Simple but effective feedback generation to tutor abstract problem solving , 2008, INLG.

[26]  Andrew R. A. Conway,et al.  Working memory capacity and its relation to general intelligence , 2003, Trends in Cognitive Sciences.

[27]  Davide Fossati,et al.  I saw TREE trees in the park: How to Correct Real-Word Spelling Mistakes , 2008, LREC.

[28]  Barbara A. Fox.  The Human Tutorial Dialogue Project: Issues in the Design of instructional Systems , 1993 .

[29]  Susan R. Goldman,et al.  Learning in complex domains: When and why do multiple representations help? , 2003 .

[30]  Patrick C. Kyllonen,et al.  Reasoning ability is (little more than) working-memory capacity?! , 1990 .

[31]  Michael F. Bunting,et al.  Working memory span tasks: A methodological review and user’s guide , 2005, Psychonomic bulletin & review.

[32]  Davide Fossati,et al.  I learn from you, you learn from me: How to make iList learn from students , 2009, AIED.

[33]  K. VanLehn,et al.  Coached program planning: dialogue-based support for novice program design , 2003, SIGCSE.

[34]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[35]  Diane J. Litman,et al.  Analyzing Dependencies Between Student Certainness States and Tutor Responses in a Spoken Dialogue Corpus , 2008 .

[36]  Abbie Brown,et al.  Design experiments: Theoretical and methodological challenges in creating complex interventions in c , 1992 .

[37]  Mia Stern,et al.  Applications of AI in education , 1996, CROS.

[38]  Davide Fossati,et al.  Beyond the code-and-count analysis of tutoring dialogues , 2007, AIED.

[39]  Davide Fossati,et al.  Aggregation Improves Learning: Experiments in Natural Language Generation for Intelligent Tutoring Systems , 2005, ACL.

[40]  Carolyn Penstein Rosé,et al.  Spoken Versus Typed Human and Computer Dialogue Tutoring , 2006, Int. J. Artif. Intell. Educ..

[41]  B. MacWhinney.  The CHILDES project: tools for analyzing talk , 1992 .

[42]  Barbara Di Eugenio,et al.  Knowledge Co-construction and Initiative in Peer Learning Interactions , 2009, AIED.

[43]  Elliot Soloway,et al.  Beverly Park Woolf, Eric Rubin, Paul Barth: Meno-II: An Intelligent Tutoring System for Novice Programmers , 1981, IJCAI.

[44]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[45]  Barbara A. Fox.  Cognitive and interactional aspects of correction in tutoring , 1991 .

[46]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Five Years of Evaluations , 2005, AIED.

[47]  Xin Lu,et al.  Expert Tutoring and Natural Language Feedback in Intelligent Tutoring Systems , 2006, ICCE.

[48]  Davide Fossati,et al.  A Mixed Trigrams Approach for Context Sensitive Spell Checking , 2009, CICLing.

[49]  Kalina Yacef,et al.  Educational Data Mining: a Case Study , 2005, AIED.

[50]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[51]  Davide Fossati,et al.  Learning Linked Lists: Experiments with the iList System , 2008, Intelligent Tutoring Systems.

[52]  A. Renkl.  Worked-out examples: instructional explanations support learning by self- explanations , 2002 .

[53]  Tiffany Barnes,et al.  Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data , 2008, Intelligent Tutoring Systems.