Learner characteristics and dialogue: recognising effective and student-adaptive tutorial strategies

In recent years, there have been significant advances in tutoring systems that engage students in rich natural language dialogue. With the goal of further understanding what makes tutorial dialogue successful, this article presents a corpus-based approach to modelling the differential effectiveness of tutorial dialogue strategies with respect to learning. We present results of a study in which task-oriented, textual tutorial dialogue was collected from remote one-on-one human tutoring sessions. This article extends a previous study which found that certain dialogue acts were correlated with learning and student characteristics in the corpus. The predictive models presented here demonstrate important differences between the dialogue sequences that were correlated with learning for different groups of students. The models demonstrate that tutor directives, a type of bottom-out hint, were negatively associated with learning for students with low incoming knowledge or low self-efficacy. The findings signal the importance of tutorial dialogue that adapts to learner characteristics.

[1]  K. VanLehn The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[2]  Kristy Elizabeth Boyer,et al.  Multimodal analysis of the implicit affective channel in computer-mediated textual communication , 2012, ICMI '12.

[3]  Kristy Elizabeth Boyer,et al.  Learning Dialogue Management Models for Task-Oriented Dialogue with Parallel Dialogue and Task Streams , 2013, SIGDIAL Conference.

[4]  Kristy Elizabeth Boyer,et al.  Automatically Recognizing Facial Indicators of Frustration: A Learning-centric Analysis , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[5]  B. Bloom The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[6]  Davide Fossati,et al.  Beyond the code-and-count analysis of tutoring dialogues , 2007, AIED.

[7]  Kurt VanLehn,et al.  Do Micro-Level Tutorial Decisions Matter: Applying Reinforcement Learning to Induce Pedagogical Tutorial Tactics , 2010, Intelligent Tutoring Systems.

[8]  Johanna D. Moore,et al.  Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains Modalities , 2009, AIED.

[9]  Andrew Olney,et al.  Off Topic Conversation in Expert Tutoring: Waste of Time or Learning Opportunity , 2010, EDM.

[10]  A. Bandura GUIDE FOR CONSTRUCTING SELF-EFFICACY SCALES , 2006 .

[11]  Barbara A. Fox The Human Tutorial Dialogue Project: Issues in the Design of instructional Systems , 1993 .

[12]  Kristy Elizabeth Boyer,et al.  Investigating the Relationship Between Dialogue Structure and Tutoring Effectiveness: A Hidden Markov Modeling Approach , 2011, Int. J. Artif. Intell. Educ..

[13]  Davide Fossati,et al.  Exploring Effective Dialogue Act Sequences in One-on-one Computer Science Tutoring Dialogues , 2011, BEA@ACL.

[14]  Carolyn Penstein Rosé,et al.  Spoken Versus Typed Human and Computer Dialogue Tutoring , 2006, Int. J. Artif. Intell. Educ..

[15]  M. Lepper,et al.  Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. , 1993 .

[16]  V. Shute Focus on Formative Feedback , 2007 .

[17]  Joel A. Michael,et al.  One-on-One Tutoring by Humans and Computers , 2006 .

[18]  Diane J. Litman,et al.  When Does Disengagement Correlate with Learning in Spoken Dialog Computer Tutoring? , 2011, AIED.

[19]  Sidney K. D'Mello,et al.  Dialogue Modes in Expert Tutoring , 2008, Intelligent Tutoring Systems.

[20]  Andrew Olney,et al.  Collaborative Lecturing by Human and Computer Tutors , 2010, Intelligent Tutoring Systems.

[21]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[22]  Diane J. Litman,et al.  Adapting to Student Uncertainty Improves Tutoring Dialogues , 2009, AIED.

[23]  Johanna D. Moore,et al.  A comparative evaluation of socratic versus didactic tutoring , 2001 .

[24]  Chen-Lin C. Kulik,et al.  Educational Outcomes of Tutoring: A Meta-analysis of Findings , 1982 .

[25]  Andrew Olney,et al.  Mining Collaborative Patterns in Tutorial Dialogues , 2010, EDM 2010.

[26]  Johanna D. Moore,et al.  Content, Social, and Metacognitive Statements: An Empirical Study Comparing Human-Human and Human-Computer Tutorial Dialogue , 2010, EC-TEL.

[27]  Carolyn Penstein Rosé,et al.  Socially Capable Conversational Tutors Can Be Effective in Collaborative Learning Situations , 2010, Intelligent Tutoring Systems.

[28]  Kristy Elizabeth Boyer,et al.  The Influence of Learner Characteristics on Task-Oriented Tutorial Dialogue , 2007, AIED.

[29]  Diane J. Litman,et al.  Correlations between dialogue acts and learning in spoken tutoring dialogues , 2006, Natural Language Engineering.

[30]  Joseph P. Magliano,et al.  Collaborative dialogue patterns in naturalistic one-to-one tutoring , 1995 .

[31]  Takashi Yamauchi,et al.  Learning from human tutoring , 2001, Cogn. Sci..

[32]  A. Graesser,et al.  A Motivationally Supportive Affect-Sensitive AutoTutor , 2011 .

[33]  Andrew Olney,et al.  Individual Differences as Predictors of Learning and Engagement , 2010 .

[34]  Kristy Elizabeth Boyer,et al.  Embodied Affect in Tutorial Dialogue: Student Gesture and Posture , 2013, AIED.

[35]  Kristy Elizabeth Boyer,et al.  Recognizing Effective and Student-Adaptive Tutor Moves in Task-Oriented Tutorial Dialogue , 2012, FLAIRS Conference.

[36]  Kristy Elizabeth Boyer,et al.  A Markov Decision Process Model of Tutorial Intervention in Task-Oriented Dialogue , 2013, AIED.