Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System

Assessment of students’ self-regulated learning (SRL) requires a method for evaluating whether observed actions are appropriate acts of self-regulation in the specific learning context in which they occur. We review research that has resulted in an automated method for context-sensitive assessment of a specific SRL strategy, help seeking while working with an intelligent tutoring system. The method relies on a computer-executable model of the targeted SRL strategy. The method was validated by showing that it converges with other measures of help seeking. Automated feedback on help seeking driven by this method led to a lasting improvement in students’ help-seeking behavior, although not in domain-specific learning. The method is unobtrusive, is temporally fine-grained, and can be applied on a large scale and over extended periods. The approach could be applied to other SRL strategies besides help seeking.

[1]  Vincent Aleven,et al.  Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor , 2006, Int. J. Artif. Intell. Educ..

[2]  Gautam Biswas,et al.  LEARNING BY TEACHING: A NEW AGENT PARADIGM FOR EDUCATIONAL SOFTWARE , 2005, Appl. Artif. Intell..

[3]  K. Koedinger,et al.  Fostering the Intelligent Novice: Learning From Errors With Metacognitive Tutoring , 2005 .

[4]  Kenneth R. Koedinger,et al.  An Open Repository and analysis tools for fine-grained, longitudinal learner data , 2008, EDM.

[5]  D. Wood,et al.  Help seeking, learning and contingent tutoring , 1999, Comput. Educ..

[6]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[7]  Vincent Aleven,et al.  Limitations of Student Control: Do Students Know When They Need Help? , 2000, Intelligent Tutoring Systems.

[8]  Stuart A. Karabenick,et al.  Help seeking in academic settings : goals, groups, and contexts , 2006 .

[9]  Roger Azevedo,et al.  Does adaptive scaffolding facilitate students ability to regulate their learning with hypermedia? q , 2004 .

[10]  Roy D. Pea,et al.  Addressing the Challenges of Inquiry-Based Learning Through Technology and Curriculum Design , 1999 .

[11]  Beverly Park Woolf,et al.  Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning , 2008 .

[12]  Vincent Aleven,et al.  The Invention Lab: Using a Hybrid of Model Tracing and Constraint-Based Modeling to Offer Intelligent Support in Inquiry Environments , 2010, Intelligent Tutoring Systems.

[13]  Kurt VanLehn,et al.  What's in a Step? Toward General, Abstract Representations of Tutoring System Log Data , 2007, User Modeling.

[14]  J. Schofield.  Computers and classroom culture , 1995 .

[15]  Philip H. Winne,et al.  Examining trace data to explore self-regulated learning , 2007 .

[16]  John B. Black,et al.  The Development of Cognitive Skills To Support Inquiry Learning , 2000 .

[17]  K. Koedinger,et al.  Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system , 2011, Learning and Instruction.

[18]  Kurt VanLehn,et al.  The Behavior of Tutoring Systems , 2006, Int. J. Artif. Intell. Educ..

[19]  Antonija Mitrovic,et al.  Evaluation of a Constraint-Based Tutor for a Database Language , 1999 .

[20]  Vincent Aleven,et al.  The Help Tutor: Does Metacognitive Feedback Improve Students' Help-Seeking Actions, Skills and Learning? , 2006, Intelligent Tutoring Systems.

[21]  Albert T. Corbett,et al.  The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning , 2012, Cogn. Sci..

[22]  R. Newman.  Adaptive help seeking: A role of social interaction in self-regulated learning. , 1998 .

[23]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[24]  Peter Brusilovsky,et al.  Adaptive Hypermedia , 2001, User Modeling and User-Adapted Interaction.

[25]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[26]  Neil T. Heffernan,et al.  Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[27]  Ryan Shaun Joazeiro de Baker,et al.  Modeling and understanding students' off-task behavior in intelligent tutoring systems , 2007, CHI.

[28]  W. V. van Joolingen,et al.  Scientific Discovery Learning with Computer Simulations of Conceptual Domains , 1998 .

[29]  K. Koedinger,et al.  Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking , 2007 .

[30]  Albert T. Corbett,et al.  Modeling Student Knowledge: Cognitive Tutors in High School and College , 2000, User Modeling and User-Adapted Interaction.

[31]  Philip H. Winne,et al.  Measuring Self-Regulated Learning , 2000 .

[32]  K. Koedinger,et al.  In Vivo Experiments on Whether Supporting Metacognition in Intelligent Tutoring Systems Yields Robust Learning , 2009 .

[33]  S. Gall,et al.  Chapter 2 Help-Seeking Behavior in Learning , 2007 .

[34]  P. Pintrich.  The role of goal orientation in self-regulated learning. , 2000 .

[35]  Héfer Bembenutty.  Feeling-of-Knowing Judgment and Self-Regulation of Learning. , 2009 .

[36]  P. Pintrich.  A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students , 2004 .

[37]  Vincent Aleven,et al.  Can Help Seeking Be Tutored? Searching for the Secret Sauce of Metacognitive Tutoring , 2007, AIED.

[38]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[39]  Philip H. Winne,et al.  Exploring students’ calibration of self reports about study tactics and achievement , 2002 .

[40]  Raymond P. Perry,et al.  The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective , 2007 .

[41]  Akane Zusho,et al.  Contextual Determinants of Motivation and Help Seeking in the College Classroom , 2007 .

[42]  B. Zimmerman,et al.  Self-regulation: Where metacognition and motivation intersect. , 2009 .

[43]  Daniel L. Schwartz,et al.  Do Learning by Teaching Environments with Metacognitive Support Help Students Develop Better Learning Behaviors , 2007 .

[44]  S. Gall.  Help-seeking: An understudied problem-solving skill in children☆ , 1981 .

[45]  Stuart Hannabuss,et al.  Strategic Help Seeking: Implications for Learning and Teaching , 1999 .

[46]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[47]  B. Zimmerman.  Attaining self-regulation: A social cognitive perspective. , 2000 .

[48]  Mary A. Mark,et al.  An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City” , 2015, International Journal of Artificial Intelligence in Education.

[49]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[50]  Albert T. Corbett,et al.  The Knowledge-Learning-Instruction (KLI) Framework: Toward Bridging the Science-Practice Chasm to Enhance Robust Student Learning , 2010 .

[51]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[52]  S. Gall,et al.  Chapter 2: Help-Seeking Behavior in Learning , 1985 .

[53]  V. Aleven,et al.  Help Seeking and Help Design in Interactive Learning Environments , 2003 .

[54]  Kenneth R. Koedinger,et al.  A Response Time Model For Bottom-Out Hints as Worked Examples , 2008, EDM.

[55]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[56]  Kenneth R. Koedinger,et al.  Using Problem-Solving Context to Assess Help Quality in Computer-Mediated Peer Tutoring , 2010, Intelligent Tutoring Systems.

[57]  Roger Azevedo,et al.  Understanding the complex nature of self-regulatory processes in learning with computer-based learning environments: an introduction , 2007 .

[58]  Daniel D. Suthers,et al.  Component-Based Construction of a Science Learning Space , 1998, Intelligent Tutoring Systems.

[59]  K. Koedinger,et al.  Exploring the Assistance Dilemma in Experiments with Cognitive Tutors , 2007 .

[60]  B. Zimmerman.  Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects , 2008, American Educational Research Journal.

[61]  B. Zimmerman,et al.  Motivation and Self-Regulated Learning: Theory, Research, and Applications , 2009 .

[62]  A. Dillon,et al.  Hypermedia as an Educational Technology: A Review of the Quantitative Research Literature on Learner Comprehension, Control, and Style , 1998 .

[63]  J. Frederiksen,et al.  Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students , 1998 .

[64]  Marsha C. Lovett,et al.  Cognotive Task Analysis in Service of Intelligent Tutoring System Design: A Case Study in Statistics , 1998, Intelligent Tutoring Systems.

[65]  Z. Mevarech,et al.  The effects of metacognitive training versus worked-out examples on students' mathematical reasoning. , 2003, The British journal of educational psychology.

[66]  S. Tobias,et al.  Knowing What You Know and What You Don't: Further Research on Metacognitive Knowledge Monitoring , 2002 .