A student model for an intelligent tutoring system helping novices learn object-oriented design

Learning object-oriented design is a challenging task for many beginners. Intelligent Tutoring Systems (ITSs) can aid learners with complex problem-solving. Generally, an ITS contains three main components: an expert evaluator that observes a student's work and identifies errors in his/her solution; a student model that diagnoses the gap in the student's knowledge; and a pedagogical advisor that provides feedback to the student. Existing student models have several common problems: (1) they only consider rule-based behaviors or concepts as students' learning goals, while students oftentimes are confused about the relationship among concepts such as prerequisite, transition, similarity and distinction; (2) they do not represent layered student knowledge, taking into account domain, reasoning, monitoring and reflective knowledge; (3) they often use Bayesian networks requiring exponential time, and hence cannot provide consistent support for real time communicative tutoring; and (4) they rarely simulate students' knowledge history. This dissertation develops a student model applying Atomic Dynamic Bayesian Networks (ADBNs), which consists two connected Atomic Bayesian Networks (ABNs). It advances the state of the art of student models by: (1) representing concepts and important relationships, such as prerequisites and distinction; (2) tracking a history of student learning; and (3) integrating a student model from both open- and close-ended work. This student model does all these things in real time, so that the ITS can be responsive to students as they work on an assigned problem. We evaluated the ABN- and ADBN-based student models with 240 simulated students and 71 human subjects. The evaluation investigates the student models' behavior for different types of students and different slip and guess values. Holding slip and guess to equal and small values, ADBNs are able to produce accurate diagnostic rates for student knowledge states over students' learning history. The results also show that student models with ADBNs perform better than student models with ABNs only. We evaluated the student model which integrates the diagnosed students' learning state from closed- and open-ended exercises with the 71 human subjects. The results show that integrating diagnoses from closed- and open-ended exercises is an effective way to increase the accuracy of student models. In addition, we compared the accuracy of non-advanced-numerical student models with the student models using ADBNs. The results show that student models using ADBNs perform much better than the non-advanced-numerical student models.

[1]  Antonija Mitrovic,et al.  Tailoring Feedback by Correcting Student Answers , 2000, Intelligent Tutoring Systems.

[2]  Zhendong Niu Bayesian student modeling, user interfaces and feedback: A sensitivity analysis , 2001 .

[3]  Jim E. Greer,et al.  Adaptive Assessment Using Granularity Hierarchies and Bayesian Nets , 1996, Intelligent Tutoring Systems.

[4]  William R. Murray,et al.  A Practical Approach to Bayesian Student Modeling , 1998, Intelligent Tutoring Systems.

[5]  Mark Guzdial,et al.  A multi-national, multi-institutional study of assessment of programming skills of first-year CS students , 2001, ITiCSE-WGR '01.

[6]  Glenn D. Blank,et al.  A pedagogical framework to integrate learning style into intelligent tutoring systems , 2007 .

[7]  Stellan Ohlsson,et al.  Some principles of intelligent tutoring , 1986 .

[8]  Cristina Conati,et al.  Procedural Help in Andes: Generating Hints Using a Bayesian Network Student Model , 1998, AAAI/IAAI.

[9]  Kenneth R. Koedinger,et al.  Cognitive Tutors as Research Platforms: Extending an Established Tutoring System for Collaborative and Metacognitive Experimentation , 2006, Intelligent Tutoring Systems.

[10]  Brent Martin Constraint-Based Student Modeling: Representing Student Knowledge , 1999, New Zealand Computer Science Research Students' Conference.

[11]  Cory J. Butz,et al.  A Web-Based Intelligent Tutoring System for Computer Programming , 2004, IEEE/WIC/ACM International Conference on Web Intelligence (WI'04).

[12]  Valerie J. Shute,et al.  SMART: Student modeling approach for responsive tutoring , 1995, User Modeling and User-Adapted Interaction.

[13]  Kenneth R. Koedinger,et al.  Toward a Rapid Development Environment for Cognitive Tutors , 2003 .

[14]  Antonija Mitrovic,et al.  Evaluation of a Constraint-Based Tutor for a Database Language , 1999 .

[15]  Fang Wei,et al.  Student Modeling with Atomic Bayesian Networks , 2006, Intelligent Tutoring Systems.

[16]  Glenn D. Blank,et al.  Generating and evaluating object-oriented designs in an intelligent tutoring system , 2008 .

[17]  D. C. Howell Statistical Methods for Psychology , 1987 .

[18]  Gerhard Weber Episodic Learner Modeling , 1996 .

[19]  Ricardo Conejo,et al.  Introducing Prerequisite Relations in a Multi-layered Bayesian Student Model , 2005, User Modeling.

[20]  José-Luis Pérez-de-la-Cruz,et al.  Bayesian student modeling and the problem of parameter specification , 2001, Br. J. Educ. Technol..

[21]  Denise Whitson Gurer A bi-level physics student diagnostic utilizing cognitive models for an intelligent tutoring system , 1993 .

[22]  Antonija Mitrovic,et al.  Constraint-based knowledge representation for individualized instruction , 2006, Comput. Sci. Inf. Syst..

[23]  Peter Brusilovsky,et al.  ELM-ART: An Intelligent Tutoring System on World Wide Web , 1996, Intelligent Tutoring Systems.

[24]  J. R. Hartley,et al.  Towards more intelligent teaching systems , 1973 .

[25]  Fang Wei,et al.  From objects-first to design-first with multimedia and intelligent tutoring , 2005, ITiCSE '05.

[26]  John R. Anderson,et al.  Dynamic Student Modelling in an Intelligent Tutor for LISP Programming , 1985, IJCAI.

[27]  Kurt VanLehn,et al.  The Conceptual Helper: An Intelligent Tutoring System for Teaching Fundamental Physics Concepts , 2000, Intelligent Tutoring Systems.

[28]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[29]  David Rosenthal,et al.  An Assessment of Constraint-Based Tutors: A Response to Mitrovic and Ohlsson's Critique of "A Comparison of Model-Tracing and Constraint-Based Intelligent Tutoring Paradigms" , 2006, Int. J. Artif. Intell. Educ..

[30]  Franz Josef Radermacher,et al.  Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Judea Pearl) , 1990, SIAM Rev..

[31]  Slavomir Stankov,et al.  Model tracing - diagnostic technique in an intelligent tutoring systems , 2002 .

[32]  Antonija Mitrovic,et al.  A Critique of Kodaganallur, Weitz and Rosenthal, "A Comparison of Model-Tracing and Constraint-Based Intelligent Tutoring Paradigms" , 2006, Int. J. Artif. Intell. Educ..

[33]  KURT VANLEHN Bayesian student modeling, user interfaces and feedback : A sensitivity analysis , 2001 .

[34]  Kadir Erkan,et al.  REALIZING OF OPTIMAL CURRICULUM SEQUENCES FOR A WEB-BASED GENERAL PURPOSE INTELLIGENT TUTORING SYSTEM , 2003 .

[35]  M. Chi,et al.  Network representation of a child's dinosaur knowledge. , 1983 .

[36]  Peter Brusilovsky,et al.  ELM-ART: An Adaptive Versatile System for Web-based Instruction , 2001 .

[37]  Fang Wei,et al.  A Web-based ITS for OO Design , 2005 .

[38]  Kurt VanLehn,et al.  Student Modeling from Conversational Test Data: A Bayesian Approach Without Priors , 1998, Intelligent Tutoring Systems.

[39]  Vincent Aleven,et al.  Modeling Students' Metacognitive Errors in Two Intelligent Tutoring Systems , 2005, User Modeling.

[40]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[41]  Albert T. Corbett,et al.  Student Modeling in the ACT Programming Tutor: Adjusting a Procedural Learning Model With Declarative Knowledge , 1997 .

[42]  Vincent Aleven,et al.  An architecture to combine meta-cognitive and cognitive tutoring: Pilot testing the Help Tutor , 2005, AIED.

[43]  Shih-Hung Wu,et al.  A cognitive student model-an ontological approach , 2002, International Conference on Computers in Education, 2002. Proceedings..

[44]  Tanja Mitrovic,et al.  Constraint-based tutors: a success story , 2001, AIED.

[45]  J. Self Dormobile : a vehicle for metacognition , 1995 .

[46]  Glenn D. Blank,et al.  A design-first curriculum for teaching Java in a CS1 course , 2005, SGCS.

[47]  José-Luis Pérez-de-la-Cruz,et al.  A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation , 2002, User Modeling and User-Adapted Interaction.

[48]  B. Bloom The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[49]  Amruth N. Kumar,et al.  Model-Based Reasoning for Domain Modeling in a Web-Based Intelligent Tutoring System to Help Students Learn to Debug C++ Programs , 2002, Intelligent Tutoring Systems.

[50]  Fang Wei,et al.  Atomic Dynamic Bayesian Networks for a Responsive Student Model , 2007, AIED.

[51]  Frantisek Franek,et al.  A Prototype for an Intelligent Tutoring System for Students Learning to Program in Java , 2004 .

[52]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[53]  Fang Wei,et al.  A student model for object-oriented design and programming , 2005 .

[54]  Martha W. Evens,et al.  Building a new student model to support adaptive tutoring in a natural language dialogue system , 2000 .

[55]  Sandra Katz,et al.  Modeling the Student in Sherlock II , 1992 .

[56]  Albert T. Corbett,et al.  Modeling Student Knowledge: Cognitive Tutors in High School and College , 2000, User Modeling and User-Adapted Interaction.

[57]  John R. Anderson,et al.  The automated tutoring of introductory computer programming , 1986, CACM.