Intelligent intervention by conversational agent through chatlog analysis

E-learning systems based on a conversational agent provide the basis of an intuitive, responsive, engaging interface for the online learner. This paper proposes an approach to intelligent intervention and strategic pedagogical design for improving student engagement when chatting with a conversational agent. First, we used previous conversational logs to detect and classify interaction behaviors of learners. And then we designed a set of strategies for intelligent intervention to improve learners’ engagement when conversing with the conversational agents.We implemented a multiagent framework to apply the strategy-based intervention. The effectiveness of learner interaction behaviors and the impact of intelligent intervention by the conversational agent were evaluated through chatlog analysis. Although not all of the quantitative tests were sensitive enough to detect the effect of the interventions, the findings suggest that the detection of behaviours was accurate. The interventions were observed to have the desired effect on behaviours associated with conversational engagement.

[1]  Michael Procter,et al.  Improving Conversation Engagement Through Data-Driven Agent Behavior Modification , 2016, Canadian Conference on AI.

[2]  Nicole Novielli,et al.  HMM modeling of user engagement in advice-giving dialogues , 2010, Journal on Multimodal User Interfaces.

[3]  Ashish Kapoor,et al.  Multimodal affect recognition in learning environments , 2005, ACM Multimedia.

[4]  Justine Cassell,et al.  Embodied Conversational Agents: Representation and Intelligence in User Interfaces , 2001, AI Mag..

[5]  Bob Heller.  Conversational Agents as Historical Figures: Individual Differences and Perceptions of Agent and Social Presence , 2016 .

[6]  Yukiko I. Nakano,et al.  Estimating user's engagement from eye-gaze behaviors in human-agent conversations , 2010, IUI '10.

[7]  Arthur C. Graesser,et al.  Automatic detection of learner’s affect from conversational cues , 2008, User Modeling and User-Adapted Interaction.

[8]  Carolyn Penstein Rosé,et al.  Linguistic Reflections of Student Engagement in Massive Open Online Courses , 2014, ICWSM.

[9]  Robert Heller,et al.  Animated Pedagogical Agents: The Effect of Visual Information on a Historical Figure Application , 2009, Int. J. Web Based Learn. Teach. Technol..

[10]  Cindy K. Chung,et al.  When Small Words Foretell Academic Success: The Case of College Admissions Essays , 2014, PloS one.

[11]  A. Wise,et al.  The Effects of Teacher Social Presence on Student Satisfaction, Engagement, and Learning , 2004, ICLS.

[12]  Alessandro De Gloria,et al.  Towards a conversational agent architecture to favor knowledge discovery in serious games , 2011, Advances in Computer Entertainment Technology.

[13]  Bilge Mutlu,et al.  Pay attention!: designing adaptive agents that monitor and improve user engagement , 2012, CHI.

[14]  Bob Heller,et al.  Conversational Agents in Virtual Worlds: Immersion and the Conversational Record , 2014 .

[15]  Bob Heller,et al.  Embodied and Embedded Intelligence: Actor Agents on Virtual Stages , 2012 .

[16]  Maiga Chang,et al.  Smart learning environments , 2015 .

[17]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[18]  Stefan Kopp,et al.  Why emotions should be integrated into conversational agents , 2007 .

[19]  A. Graesser,et al.  Improving Comprehension Through Discourse Processing , 2002 .

[20]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[21]  Toyoaki Nishida,et al.  Designing conversational agents: effect of conversational form on our comprehension , 2006, AI & SOCIETY.

[22]  Kostas Karpouzis,et al.  Feature Extraction and Selection for Inferring User Engagement in an HCI Environment , 2009, HCI.

[23]  Marilyn A. Walker,et al.  Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text , 2007, J. Artif. Intell. Res..

[24]  Maiga Chang,et al.  MAS Controlled NPCs in 3D Virtual Learning Environment , 2013, 2013 International Conference on Signal-Image Technology & Internet-Based Systems.

[25]  Mary T. Johnson,et al.  Development of Virtual Patient Simulations for Medical Education , 2009 .

[26]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[27]  Ana Paiva,et al.  Detecting user engagement with a robot companion using task and social interaction-based features , 2009, ICMI-MLMI '09.

[28]  G. Veletsianos,et al.  What Do Learners and Pedagogical Agents Discuss When Given Opportunities for Open-Ended Dialogue? , 2013 .

[29]  Carolyn Penstein Rosé,et al.  Architecture for Building Conversational Agents that Support Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.

[30]  Karen O'Shea.  An approach to conversational agent design using semantic sentence similarity , 2012, Applied Intelligence.

[31]  Qianli Xu,et al.  Designing engagement-aware agents for multiparty conversations , 2013, CHI.

[32]  Yair Neuman,et al.  Literal and Metaphorical Sense Identification through Concrete and Abstract Context , 2011, EMNLP.