One-on-one and small group conversations with an intelligent virtual science tutor

Abstract In this study we investigated students' conversations with a virtual science tutor (Marni), either individually or in small groups. These constituted two treatment conditions. Students were presented with narrated multimedia science problems and explanations followed by question-answer dialogs with the virtual tutor. Students who received either one-on-one or small group tutoring received the same set of multimedia presentations and questions posed by the virtual tutor. Students in the small group condition discussed their answer before one student from that group responded to the tutor. We asked if students receiving tutoring using the virtual tutor in groups would demonstrate learning gains equivalent to those of students receiving one-on-one tutoring. We also asked if both groups would demonstrate greater learning gains from pretest to posttest than students in business-as-usual (control) classrooms who did not receive supplemental tutoring. One hundred eighty-three (183) students (in 13 classrooms at 4 schools) participated in the study. Of the 183 students, 114 were randomly assigned to tutoring in small groups using Marni; and 69 students received one-on-one tutoring with Marni. When compared with the control group, effect sizes for were d = 0.048 for the group tutoring condition and d = 0.51 for the one-on-one tutoring condition. A two-way ANOVA suggested a main effect for tutoring group, F = 16.8, df (41,171), p

[1]  Arthur C. Graesser,et al.  The influence of modality on deep-reasoning questions , 2010, Int. J. Learn. Technol..

[2]  V. Kennedy.  The dialogic imagination , 2018 .

[3]  W. Lewis Johnson,et al.  Tactical Language and Culture Training Systems: Using AI to Teach Foreign Languages and Cultures , 2009, AI Mag..

[4]  K. Koedinger,et al.  Using Intelligent Tutor Technology to Implement Adaptive Support for Student Collaboration , 2010 .

[5]  A. King.  Effects of self-questioning training on college students' comprehension of lectures , 1989 .

[6]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[7]  Ushiri Kulatunga,et al.  Exploration of peer leader verbal behaviors as they intervene with small groups in college general chemistry , 2013 .

[8]  Ronald A. Cole,et al.  My science tutor: A conversational multimedia virtual tutor for elementary school science , 2011, TSLP.

[9]  Gordon Wells,et al.  A Sociocultural Perspective on Classroom Discourse , 1997 .

[10]  K. Topping,et al.  Participant evaluation of parent‐tutored and peer‐tutored projects in reading , 1990 .

[11]  Shimei Pan,et al.  Designing and Evaluating an Adaptive Spoken Dialogue System , 2002, User Modeling and User-Adapted Interaction.

[12]  Vincent Aleven,et al.  Investigating Effects of Embedding Collaboration in an Intelligent Tutoring System for Elementary School Students. , 2016 .

[13]  Martin Nystrand,et al.  Opening Dialogue: Understanding the Dynamics of Language and Learning in the English Classroom (Language and Literacy Series) , 1996 .

[14]  Scotty D. Craig,et al.  Vicarious Learning: Effects of Overhearing Dialog and Monologue-like Discourse in a Virtual Tutoring Session , 2003 .

[15]  S. Schworm,et al.  Learning argumentation skills through the use of prompts for self-explaining examples. , 2007 .

[16]  K. VanLehn.  The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[17]  Kurt VanLehn,et al.  Explaining Self-Explaining: A Contrast between Content and Generation , 2007, AIED.

[18]  Lei Liu,et al.  Assessing Collaborative Problem Solving with Simulation Based Tasks , 2015, CSCL.

[19]  Michelene T. H. Chi,et al.  Observing Tutorial Dialogues Collaboratively: Insights About Human Tutoring Effectiveness From Vicarious Learning , 2008, Cogn. Sci..

[20]  Maria Virvou,et al.  Collaborative Learning: Group Interaction in an Intelligent Mobile-Assisted Multiple Language Learning System. , 2014 .

[21]  M. Nystrand,et al.  Instructional Discourse, Student Engagement, and Literature Achievement. , 1991 .

[22]  Martin Nystrand,et al.  Background and Instructional Effects on Achievement in Eighth-Grade English and Social Studies , 1991 .

[23]  M. Bakhtin,et al.  Speech genres and other late essays , 1986 .

[24]  D. Kuhn.  Science as argument : Implications for teaching and learning scientific thinking , 1993 .

[25]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[26]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[27]  Philip Bell,et al.  Conceptualizations of argumentation from science studies and the learning sciences and their implications for the practices of science education , 2008 .

[28]  Shirley Simon,et al.  Learning to Teach Argumentation: Research and development in the science classroom , 2006 .

[29]  Arthur C. Graesser,et al.  Emotions During the Learning of Difficult Material , 2012 .

[30]  Wolff-Michael Roth,et al.  Science language Wanted Alive: Through the dialectical/dialogical lens of Vygotsky and the Bakhtin circle , 2014 .

[31]  H. Schweingruber,et al.  TAKING SCIENCE TO SCHOOL: LEARNING AND TEACHING SCIENCE IN GRADES K-8 , 2007 .

[32]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[33]  Wayne H. Ward,et al.  My Science Tutor: A Conversational Multimedia Virtual Tutor. , 2013 .

[34]  L. B. Resnick,et al.  Dialogue Patterns in Peer Collaboration That Promote Learning , 2015 .

[35]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[36]  Richard S. Moog,et al.  Argumentation and participation patterns in general chemistry peer‐led sessions , 2013 .

[37]  Lei Liu,et al.  A Tough Nut to Crack: Measuring Collaborative Problem Solving , 2016 .

[38]  R. Hake.  Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[39]  P. K. Murphy,et al.  Examining the Effects of Classroom Discussion on Students' Comprehension of Text: A Meta-Analysis , 2009 .

[40]  Gordon Wells,et al.  Learning to use scientific concepts , 2008 .

[41]  Brent Morgan,et al.  Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics , 2009 .

[42]  Ann L. Brown,et al.  Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities , 1984 .

[43]  Michelene T. H. Chi,et al.  Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities , 2009, Top. Cogn. Sci..

[44]  Margaret G. McKeown,et al.  Improving Comprehension with Questioning the Author: A Fresh and Expanded View of a Powerful Approach , 2006 .

[45]  P. Black,et al.  Developing the theory of formative assessment , 2009 .

[46]  Vincent Aleven,et al.  Learning Alone or Together? A Combination Can Be Best! , 2017, CSCL.

[47]  Arthur C. Graesser,et al.  Toward Spoken Human–Computer Tutorial Dialogues , 2010, Hum. Comput. Interact..

[48]  Victor Sampson,et al.  Argument-Driven Inquiry. , 2009 .

[49]  Carolyn Penstein Rosé,et al.  Socially Capable Conversational Tutors Can Be Effective in Collaborative Learning Situations , 2010, Intelligent Tutoring Systems.

[50]  Lei Liu,et al.  Interdisciplinary research agenda in support of assessment of collaborative problem solving: lessons learned from developing a Collaborative Science Assessment Prototype , 2017, Comput. Hum. Behav..

[51]  Z. Zacharia,et al.  Physical and Virtual Laboratories in Science and Engineering Education , 2013, Science.

[52]  L. Hedges.  Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[53]  Arthur C. Graesser,et al.  Overhearing Dialogues and Monologues in Virtual Tutoring Sessions: Effects on Questioning and Vicarious Learning , 2000 .

[54]  Chen-Lin C. Kulik,et al.  Educational Outcomes of Tutoring: A Meta-analysis of Findings , 1982 .

[55]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Five Years of Evaluations , 2005, AIED.

[56]  L. Vygotsky.  Thinking and Speech , 1987 .

[57]  K. Pine,et al.  The Effect of Explaining Another's Actions on Children's Implicit Theories of Balance , 2000 .

[58]  Leema K. Berland,et al.  Making sense of argumentation and explanation , 2009 .

[59]  Kurt VanLehn,et al.  Shall We Explain? Augmenting Learning from Intelligent Tutoring Systems and Peer Collaboration , 2008, Intelligent Tutoring Systems.

[60]  Anne M. Poliquin,et al.  Role of Epistemic Beliefs and Scientific Argumentation in Science Learning , 2008 .

[61]  Robert E. Slavin,et al.  What Works for Students at Risk: A Research Synthesis. , 1989 .

[62]  Stephen Krause,et al.  Differentiated Overt Learning Activities for Effective Instruction in Engineering Classrooms , 2013 .

[63]  Jonathan Osborne,et al.  Supporting Argumentation Through Students' Questions: Case Studies in Science Classrooms , 2010 .

[64]  Wolff-Michael Roth,et al.  An Integrated Theory of Thinking and Speaking that Draws on Vygotsky and Bakhtin/Vološinov , 2013 .

[65]  Katherine L. McNeill.  Elementary Students' Views of Explanation, Argumentation, and Evidence, and Their Abilities to Construct Arguments over the School Year. , 2011 .

[66]  Philip Bell,et al.  “What comes to mind when you think of science? The perfumery!”: Documenting science-related cultural learning pathways across contexts and timescales , 2014 .

[67]  Kirsten R. Butcher.  Learning from Text with Diagrams: Promoting Mental Model Development and Inference Generation. , 2006 .

[68]  D. Kuhn.  Metacognitive Development , 2000 .

[69]  P. K. Murphy,et al.  What the Discourse Tells Us: Talk and Indicators of High-Level Comprehension. , 2008 .

[70]  J. F. Voss,et al.  Learning to Reason via Instruction in Argumentation. , 1991 .

[71]  Jonathan Osborne,et al.  Arguing to Learn in Science: The Role of Collaborative, Critical Discourse , 2010, Science.

[72]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[73]  A. King,et al.  Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. , 1998 .