An Architecture to Develop Multimodal Educative Applications with Chatbots

Animated characters are beginning to be used as pedagogical tools, as they have the power to capture students' attention and foster their motivation for discovery and learning. However, in order for them to be widely employed and accepted as a learning resource, they must be easy to use and friendly. In this paper we present an architecture that facilitates building interactive pedagogical chatbots that can interact with students in natural language. Our proposal provides a modular and scalable framework to develop such systems efficiently. Additionally, we present Geranium, a system that helps children to appreciate and protect their environment with an interactive chatbot developed following our scheme.

[1]  Luke K. Fryer,et al.  Bots as Language Learning Tools , 2006 .

[2]  David Griol,et al.  A statistical approach to spoken dialog systems design and evaluation , 2008, Speech Commun..

[3]  Carolyn Penstein Rosé,et al.  The Architecture of Why2-Atlas: A Coach for Qualitative Physics Essay Writing , 2002, Intelligent Tutoring Systems.

[4]  Alexander I. Rudnicky,et al.  Instruction Taking in the TeamTalk System , 2010, AAAI Fall Symposium: Dialog with Robots.

[5]  Marc Cavazza,et al.  How was your day?: a companion ECA , 2010, AAMAS.

[6]  S. Marsella,et al.  Interactive Pedagogical Drama for Health Interventions , 2003 .

[7]  Liu Yi,et al.  Designing Interaction Models in a Multiparty 3D Learning Environment , 2004 .

[8]  Thomas Bub,et al.  The Verbmobil prototype system – a software engineering perspective , 1999, Natural Language Engineering.

[9]  Sebastian Möller,et al.  Memo: towards automatic usability evaluation of spoken dialogue services by user error simulations , 2006, INTERSPEECH.

[10]  Naoyuki Kanda,et al.  A multi-expert model for dialogue and behavior control of conversational robots and agents , 2011, Knowl. Based Syst..

[11]  Robbert-Jan Beun,et al.  Embodied Conversational Agents: Effects on Memory Performance and Anthropomorphisation , 2003, IVA.

[12]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[13]  Yanghee Kim,et al.  Desirable Characteristics of Learning Companions , 2007, Int. J. Artif. Intell. Educ..

[14]  W. Lewis Johnson,et al.  A Pedagogical Agent for Psychosocial Intervention on a Handheld Computer , 2004, AAAI Technical Report.

[15]  Oliver Lemon,et al.  Accurate statistical spoken language understanding from limited development resources , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Gérard Bailly,et al.  Gaze, conversational agents and face-to-face communication , 2010, Speech Commun..

[17]  Johanna D. Moore,et al.  A comparative evaluation of socratic versus didactic tutoring , 2001 .

[18]  Jack Mostow,et al.  Why and How Our Automated Reading Tutor Listens , 2012 .

[19]  A. Baskin,et al.  Studying with the prince: The computer as a learning companion , 1988 .

[20]  Futoshi Asano,et al.  Spoken Language Interface of the Jijo-2 Office Robot , 2001, ISRR.

[21]  Ning Wang,et al.  The Politeness Effect in an Intelligent Foreign Language Tutoring System , 2008, Intelligent Tutoring Systems.

[22]  Milica Gasic,et al.  Spoken language understanding from unaligned data using discriminative classification models , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Vincent Aleven,et al.  Evaluating the Effectiveness of a Tutorial Dialogue System for Self-Explanation , 2004, Intelligent Tutoring Systems.

[24]  T. Bickmore.  Relational agents : effecting change through human-computer relationships , 2003 .

[25]  Brady Clark,et al.  Responding to Student Uncertainty in Spoken Tutorial Dialogue Systems , 2006, Int. J. Artif. Intell. Educ..

[26]  Mattias Heldner,et al.  Towards human-like spoken dialogue systems , 2008, Speech Commun..

[27]  Wolfgang Wahlster,et al.  SmartKom: Foundations of Multimodal Dialogue Systems , 2006, SmartKom.

[28]  J. V. Kuppevelt,et al.  Advances in natural multimodal dialogue systems , 2005 .

[29]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[30]  Arturo Espinosa-Romero,et al.  Talking to Godot: dialogue with a mobile robot , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[32]  Eric Atwell,et al.  Fostering Language Learner Autonomy Through Adaptive Conversation Tutors , 2007 .

[33]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.

[34]  Michael F. McTear,et al.  An approach to multi-strategy dialogue management , 2005, INTERSPEECH.

[35]  David Traum,et al.  The Information State Approach to Dialogue Management , 2003 .

[36]  Yanghee Kim,et al.  Simulating Instructional Roles through Pedagogical Agents , 2005, Int. J. Artif. Intell. Educ..

[37]  Kuansan Wang.  SALT: a spoken language interface for web-based multimodal dialog systems , 2002, INTERSPEECH.

[38]  Kristiina Jokinen.  Evaluation of adaptivity and user expertise in a speech-based e-mail system , 1996 .

[39]  Arthur C. Graesser,et al.  AutoTutor: A simulation of a human tutor , 1999, Cognitive Systems Research.

[40]  Susan Bull,et al.  CALMsystem: A Conversational Agent for Learner Modelling , 2007, Knowl. Based Syst..

[41]  Judy Kay,et al.  Artificial intelligence in education : shaping the future of learning through intelligent technologies , 2003 .

[42]  Timothy W. Bickmore,et al.  Establishing and maintaining long-term human-computer relationships , 2005, TCHI.

[43]  Bruce Edmonds,et al.  A conversational intelligent tutoring system to automatically predict learning styles , 2012, Comput. Educ..

[44]  Scotty D. Craig,et al.  Integrating Affect Sensors in an Intelligent Tutoring System , 2004 .

[45]  Ronald Rosenfeld,et al.  Towards a universal speech interface , 2000, INTERSPEECH.

[46]  Susan Bull,et al.  Children's Interactions with Inspectable and Negotiated Learner Models , 2008, Intelligent Tutoring Systems.

[47]  Michael F. McTear,et al.  Book Review , 2005, Computational Linguistics.

[48]  Carolyn Penstein Rosé,et al.  Architecture for Building Conversational Agents that Support Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.

[49]  Arthur C. Graesser,et al.  Teaching Tactics and Dialog in AutoTutor , 2001 .

[50]  Eric Atwell,et al.  Fostering language learner autonomy via adaptive conversation tutors , 2007 .

[51]  Kenneth D. Forbus.  CyclePad: An Articulate Virtual Laboratory for Engineering Thermodynamics , 1999, Artif. Intell..

[52]  Maarten Sierhuis,et al.  Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots , 2006, AAAI Fall Symposium: Aurally Informed Performance.

[53]  Alexander I. Rudnicky,et al.  Olympus: an open-source framework for conversational spoken language interface research , 2007, HLT-NAACL 2007.

[54]  Diane J. Litman,et al.  ITSPOKE: An Intelligent Tutoring Spoken Dialogue System , 2004, NAACL.

[55]  Chi-Jen Lin,et al.  Redefining the learning companion: the past, present, and future of educational agents , 2003, Comput. Educ..

[56]  Victor Zue,et al.  WebGALAXY: Beyond Point and Click - a Conversational Interface to a Browser , 1997, Comput. Networks.

[57]  G. Tsaganou,et al.  Dialogue-Based Reflective System for Historical Text Comprehension , 2003 .

[58]  Niels Ole Bernsen,et al.  Animating an interactive conversational character for an educational game system , 2005, IUI.

[59]  Joel R. Tetreault,et al.  Using system and user performance features to improve emotion detection in spoken tutoring dialogs , 2006, INTERSPEECH.

[60]  Ramón López-Cózar,et al.  Relations between de-facto criteria in the evaluation of a spoken dialogue system , 2008, Speech Commun..

[61]  Britta Wrede,et al.  Why and How to Model Multi-Modal Interaction for a Mobile Robot Companion , 2007, Interaction Challenges for Intelligent Assistants.

[62]  Esma Aïmeur,et al.  Some justifications about the learning by disturbin g strategy , 1997 .

[63]  C. Vaquero,et al.  VOCALIZA: AN APPLICATION FOR COMPUTER-AIDED SPEECH THERAPY IN SPANISH LANGUAGE , 2006 .

[64]  Miguel Angel Salichs,et al.  End-user programming of a social robot by dialog , 2011, Robotics Auton. Syst..

[65]  Pierre Dillenbourg,et al.  People Power: A Human-Computer Collaborative Learning System , 1992, Intelligent Tutoring Systems.

[66]  Masahiro Araki,et al.  Spoken, Multilingual and Multimodal Dialogue Systems: Development and Assessment , 2005 .

[67]  Norman I. Badler,et al.  Creating Interactive Virtual Humans: Some Assembly Required , 2002, IEEE Intell. Syst..

[68]  Niels Ole Bernsen,et al.  Evaluation and usability of multimodal spoken language dialogue systems , 2004, Speech Commun..

[69]  Ying-Hong Wang,et al.  Enhanced Semantic Question Answering System for e-Learning Environment , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).