Engaging Learners in Online Video Lectures with Dynamically Scaffolding Conversational Agents

Online education creates new opportunities for learners, which has led to sharply increasing enrollment in the last few years. Despite these benefits, past research shows that the lack of individual interaction with educators creates low learner engagement that leads to high attrition rates, which remains a major challenge in the field. Dynamically scaffolding conversational agents built into online video lectures promise to address this problem by individually interacting with learners, similar to educators’ scaffolding behavior. These agents are equipped with recent natural language processing capabilities, creating human-like conversations that help learners to be more engaged in the learning process. To test our hypothesis, we built a dynamically scaffolding conversational agent named Sara and compared it with an often-implemented static conversational agent built into two online video lectures. We deployed a lab experiment with 182 learners. The neurophysiological measurements revealed that Sara significantly improved learner engagement partly explained by differences in learners’ perceptions in the way they experienced the interaction. This study connects to already existing conversational agent studies in online education and highlights the importance of including dynamically scaffolding conversational agent in online video lectures to address the problem of low learner engagement in online education.

[1]  B. D. Raad The big five personality factors : the psycholexical approach to personality , 2000 .

[2]  Haiying Li,et al.  Impact of Pedagogical Agents' Conversational Formality on Learning and Engagement , 2017, AIED.

[3]  M. Cole,et al.  Mind in society: The development of higher psychological processes. L. S. Vygotsky. , 1978 .

[4]  Donggil Song,et al.  Interacting with a conversational agent system for educational purposes in online courses , 2017, 2017 10th International Conference on Human System Interactions (HSI).

[5]  KerlyAlice,et al.  Bringing chatbots into education , 2007 .

[6]  Ting-Peng Liang,et al.  Guidelines for Neuroscience Studies in Information Systems Research , 2014, J. Manag. Inf. Syst..

[7]  Raymond J. Corsini,et al.  The concise Corsini encyclopedia of psychology and behavioral science , 2004 .

[8]  Ryan O. Murphy,et al.  Using Skin Conductance in Judgment and Decision Making Research , 2011 .

[9]  J. Bruner,et al.  The role of tutoring in problem solving. , 1976, Journal of child psychology and psychiatry, and allied disciplines.

[10]  Susan Bull,et al.  Conversational Agents in E-Learning , 2008, SGAI Conf..

[11]  Roger Azevedo,et al.  Does adaptive scaffolding facilitate students ability to regulate their learning with hypermedia? q , 2004 .

[12]  Jan Marco Leimeister,et al.  Alexa, Can You Help Us Solve This Problem?: How Conversations With Smart Personal Assistant Tutors Increase Task Group Outcomes , 2019, CHI Extended Abstracts.

[13]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[14]  Angelika Dimoka,et al.  On the Use of Neuropyhsiological Tools in IS Research: Developing a Research Agenda for NeuroIS , 2012, MIS Q..

[15]  A. Onwuegbuzie,et al.  Mixed Methods Research: A Research Paradigm Whose Time Has Come , 2004 .

[16]  Brent Morgan,et al.  Assessment with computer agents that engage in conversational dialogues and trialogues with learners , 2017, Comput. Hum. Behav..

[17]  Justine Cassell,et al.  Embodied conversational interface agents , 2000, CACM.

[18]  David Griol,et al.  Combining speech-based and linguistic classifiers to recognize emotion in user spoken utterances , 2017, Neurocomputing.

[19]  H. Bernard,et al.  Techniques to Identify Themes , 2003 .

[20]  Karen S. McNeal,et al.  Measuring Student Engagement, Knowledge, and Perceptions of Climate Change in an Introductory Environmental Geology Course , 2014 .

[21]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[22]  Chen Chen,et al.  Smart Learning Partner: An Interactive Robot for Education , 2018, AIED.

[23]  Rebecca Ferguson,et al.  Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs) , 2015, LAK.

[24]  K. Hew,et al.  Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges , 2014 .

[25]  Carolyn Penstein Rosé,et al.  Towards an Agile Approach to Adapting Dynamic Collaboration Support to Student Needs , 2014, International Journal of Artificial Intelligence in Education.

[26]  Marc A. Brackett,et al.  Emotional Intelligence in the Classroom: Skill-Based Training for Teachers and Students. , 2007 .

[27]  Viswanath Venkatesh,et al.  Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems , 2013, MIS Q..

[28]  Fernando De la Torre,et al.  Facial Expression Analysis , 2011, Visual Analysis of Humans.

[29]  Benjamin C. Heddy,et al.  The Challenges of Defining and Measuring Student Engagement in Science , 2015 .

[30]  Harry Torrance Triangulation, Respondent Validation, and Democratic Participation in Mixed Methods Research , 2012 .

[31]  Kay S. Bull,et al.  Processes for Developing Scaffolding in a Computer Mediated Learning Environment. , 1999 .

[32]  Victoria L. Rubin,et al.  Artificially intelligent conversational agents in libraries , 2010, Libr. Hi Tech.

[33]  Susan Bull,et al.  Bringing Chatbots into education: Towards Natural Language Negotiation of Open Learner Models , 2006, SGAI Conf..

[34]  David J. Shernoff,et al.  Student engagement in high school classrooms from the perspective of flow theory. , 2003 .

[35]  Abigail Sellen,et al.  "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents , 2016, CHI.

[36]  Richard K. Lowe,et al.  An Eye Tracking Comparison of External Pointing Cues and Internal Continuous Cues in Learning with Complex Animations , 2010 .

[37]  Jing Lv,et al.  Social presence in relation to students' satisfaction and learning in the online environment: A meta-analysis , 2017, Comput. Hum. Behav..

[38]  Annika Silvervarg,et al.  Building a Social Conversational Pedagogical Agent: Design Challenges and Methodological approaches , 2010 .

[39]  Marek Hatala,et al.  Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning , 2016, Comput. Hum. Behav..

[40]  M. Chi,et al.  The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes , 2014 .

[41]  Inge Molenaar,et al.  Dynamic scaffolding of socially regulated learning in a computer-based learning environment , 2012, Comput. Educ..

[42]  Judy Kay,et al.  Artificial intelligence in education : shaping the future of learning through intelligent technologies , 2003 .

[43]  James A. Landay,et al.  QuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge , 2019, CHI.

[44]  Ian Dunwell,et al.  Levels of Interaction (LoI): A Model for Scaffolding Learner Engagement in an Immersive Environment , 2010, Intelligent Tutoring Systems.

[45]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[46]  P. Venables,et al.  Direct measurement of skin conductance: a proposal for standardization. , 1971, Psychophysiology.

[47]  Ioana Lepadatu Use self-talking for learning progress , 2012 .

[48]  Kazuhisa Seta,et al.  A Conversational Agent to Encourage Willingness to Communicate in the Context of English as a Foreign Language , 2015, KES.

[49]  K. VanLehn The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[50]  Charles R. Graham,et al.  Learner Engagement in Blended Learning Environments: A Conceptual Framework , 2019, Online Learning.

[51]  Arthur C. Graesser,et al.  Intelligent Tutoring Systems with Conversational Dialogue , 2001, AI Mag..

[52]  Alexander L. Francis,et al.  Psychophysiological measurement of affective responses during speech perception , 2018, Hearing Research.

[53]  Cher Ping Lim Engaging learners in online learning environments , 2004 .

[54]  E. Deci,et al.  Quality of Learning With an Active Versus Passive Motivational Set , 1984 .