The Effect of Personalization in Longer-Term Robot Tutoring

The benefits of personalized social robots must be evaluated in real-world educational contexts over periods of time longer than a single session to understand their full potential to impact learning outcomes. In this work, we describe a personalization system designed for longer-term personalization that orders curriculum based on an adaptive Hidden Markov Model (HMM) that evaluates students’ skill proficiencies. We present a study investigating the effectiveness of this system in a five-session interaction with a robot tutor, taking place over the course of 2 weeks. Our system is evaluated in the context of native Spanish-speaking first-graders interacting with a social robot tutor while completing an English Language Learning educational task. Participants either received lessons: (1) ordered by our adaptive HMM personalization system which selects a lesson based on a skill that the individual participant needs more practice with (“personalized condition”) or (2) ordered randomly from among the lessons the participant had not yet seen (“non-personalized condition”). We found that participants who received personalized lessons from the robot tutor outperformed participants who received non-personalized lessons on a post-test by 2.0 standard deviations on average, corresponding to a mean learning gain in the 98th percentile.

[1]  Brian Scassellati,et al.  Personalizing Robot Tutors to Individuals’ Learning Differences , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  Klaus Zechner,et al.  Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech , 2011, ACL.

[3]  John R. Anderson,et al.  Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect , 2005, Cogn. Sci..

[4]  Michael Pressley,et al.  Scaffolding student learning: Instructional approaches and issues. , 1997 .

[5]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[6]  Tony Belpaeme,et al.  Social robot tutoring for child second language learning , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[7]  Ryan Shaun Joazeiro de Baker,et al.  Detecting Student Misuse of Intelligent Tutoring Systems , 2004, Intelligent Tutoring Systems.

[8]  R. Cassidy,et al.  Overview of Race and Hispanic Origin: 2000 , 2001 .

[9]  Susan R. Fussell,et al.  Comparing a computer agent with a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Cynthia Breazeal,et al.  Affective Personalization of a Social Robot Tutor for Children's Second Language Skills , 2016, AAAI.

[11]  Jr. G. Forney,et al.  Viterbi Algorithm , 1973, Encyclopedia of Machine Learning.

[12]  Maja J. Mataric,et al.  Designing a socially assistive robot for personalized number concepts learning in preschool children , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[13]  Cristina Conati,et al.  Empirically building and evaluating a probabilistic model of user affect , 2009, User Modeling and User-Adapted Interaction.

[14]  Mike Levy,et al.  Call Dimensions: Options and Issues in Computer Assisted Language Learning (ESL & Applied Linguistics Professional Series) , 2006 .

[15]  Ana Paiva,et al.  iCat, the chess player: the influence of embodiment in the enjoyment of a game , 2008, AAMAS.

[16]  Virginia P. Collier,et al.  A National Study of School Effectiveness for Language Minority Students' Long-Term Academic Achievement , 2002 .

[17]  A. Azzouz.  2011 , 2020, City.

[18]  Brian Scassellati,et al.  Social robots for education: A review , 2018, Science Robotics.

[19]  A. Graesser,et al.  Monitoring Affective Trajectories during Complex Learning , 2007 .

[20]  John R Anderson,et al.  Using a model to compute the optimal schedule of practice. , 2008, Journal of experimental psychology. Applied.

[21]  Kenneth R. Koedinger.  Toward Evidence for Instructional Design Principles: Examples from Cognitive Tutor Math 6 , 2002 .

[22]  Ali Meghdari,et al.  Employing Humanoid Robots for Teaching English Language in Iranian Junior High-Schools , 2014, Int. J. Humanoid Robotics.

[23]  Mike Levy,et al.  Computer-Assisted Language Learning: Context and Conceptualization , 1997 .

[24]  Brian Scassellati,et al.  Shaping productive help-seeking behavior during robot-child tutoring interactions , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Carolyn Penstein Rosé,et al.  Effects of Social Presence and Social Role on Help-Seeking and Learning , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[26]  Brian Scassellati,et al.  The Physical Presence of a Robot Tutor Increases Cognitive Learning Gains , 2012, CogSci.

[27]  Bilge Mutlu,et al.  Pay attention!: designing adaptive agents that monitor and improve user engagement , 2012, CHI.

[28]  Maja J. Mataric,et al.  Embodiment and Human-Robot Interaction: A Task-Based Perspective , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[29]  Stefan Kopp,et al.  Adaptive Robot Language Tutoring Based on Bayesian Knowledge Tracing and Predictive Decision-Making , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[30]  Karen Littleton,et al.  Learning with computers: analysing productive interaction , 1999 .

[31]  Takayuki Kanda,et al.  Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial , 2004, Hum. Comput. Interact..

[32]  Emily J. Ashurst,et al.  Robot education peers in a situated primary school study: Personalisation promotes child learning , 2017, PloS one.

[33]  Ana Paiva,et al.  Long-Term Interactions with Empathic Robots: Evaluating Perceived Support in Children , 2012, ICSR.

[34]  J. Gregory Trafton,et al.  Effective Tutoring Techniques: A Comparison of Human Tutors and Intelligent Tutoring Systems , 1992 .

[35]  J. S. Johnson,et al.  Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language , 1989, Cognitive Psychology.

[36]  Sungjin Lee,et al.  On the effectiveness of Robot-Assisted Language Learning , 2011, ReCALL.

[37]  Tony Belpaeme,et al.  Higher Nonverbal Immediacy Leads to Greater Learning Gains in Child-Robot Tutoring Interactions , 2015, ICSR.

[38]  Stefan Kopp,et al.  Guidelines for Designing Social Robots as Second Language Tutors , 2018, International Journal of Social Robotics.

[39]  Maxine Eskénazi,et al.  An overview of spoken language technology for education , 2009, Speech Commun..

[40]  Robert V. Lindsey,et al.  Improving Students’ Long-Term Knowledge Retention Through Personalized Review , 2014, Psychological science.

[41]  Maja J. Mataric,et al.  Socially Assistive Robotics for Personalized Education for Children , 2014, AAAI Fall Symposia.

[42]  Stefan Kopp,et al.  The Effect of a Robot’s Gestures and Adaptive Tutoring on Children’s Acquisition of Second Language Vocabularies , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[43]  Peter G. Fairweather,et al.  Using Speech Recognition Technology to Enhance Literacy Instruction for Emerging Readers , 2000 .

[44]  Ana Paiva,et al.  Empathic Robots for Long-term Interaction , 2014, Int. J. Soc. Robotics.

[45]  Rebecca M. Callahan.  Tracking and High School English Learners: Limiting Opportunity to Learn , 2005 .

[46]  Mohammad Obaid,et al.  Comparing a humanoid tutor to a human tutor delivering an instructional task to children , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[47]  Brian Scassellati,et al.  Give Me a Break! Personalized Timing Strategies to Promote Learning in Robot-Child Tutoring , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[48]  Fumihide Tanaka,et al.  Socialization between toddlers and robots at an early childhood education center , 2007, Proceedings of the National Academy of Sciences.

[49]  Adriana Tapus,et al.  The role of physical embodiment of a therapist robot for individuals with cognitive impairments , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[50]  Michel C. Desmarais,et al.  A review of recent advances in learner and skill modeling in intelligent learning environments , 2012, User Modeling and User-Adapted Interaction.

[51]  Ginevra Castellano,et al.  Adaptive Robotic Tutors that Support Self-Regulated Learning: A Longer-Term Investigation with Primary School Children , 2018, Int. J. Soc. Robotics.

[52]  Cynthia Breazeal,et al.  Bayesian Active Learning-Based Robot Tutor for Children's Word-Reading Skills , 2015, AAAI.

[53]  Chen-Lin C. Kulik,et al.  Effectiveness of Mastery Learning Programs: A Meta-Analysis , 1990 .

[54]  G. Saunders.  Bilingual Children: From Birth to Teens , 1988 .

[55]  Fabio Tesser,et al.  Towards long-term social child-robot interaction , 2016 .

[56]  Brian Scassellati,et al.  The effect of presence on human-robot interaction , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.