Towards computer-assisted language learning with robots, wikipedia and CogInfoCom

Computer-Assisted Language Learning is being transformed by the use of humanoid robots combined with Wikipedia-based information access and CogInfoCom systems. The learner's engagement in language-learning dialogues is strengthened by the robot's non-verbal behaviour such as nodding, gesturing and face-tracking, and the learner's level of interest can be estimated by monitoring the learner's non-verbal signals through gaze-tracking and proximity measurement. In this paper we discuss new approaches in which robotassisted language learning is further enhanced by the use of CogInfoCom channels to extend the learner's engagement and interest. We focus on the robot's use of Wikipedia-based information access to greatly extend the range of topics that the robot can talk about during language-learning spoken dialogues, and describe natural language processing techniques to generate questions from Wikipedia information, so the robot can ask new questions about the learner's preferred topics. We predict that the learner's interest level is increased when the learner can navigate in Wikipedia to dialogue topics of personal interest rather than being restricted to topics chosen by the teacher.

[1]  Sungjin Lee,et al.  On the effectiveness of Robot-Assisted Language Learning , 2011, ReCALL.

[2]  Arthur C. Graesser,et al.  When is Reading Just as Effective as One-on-One Interactive Human Tutoring? , 2005 .

[3]  Ehud Reiter,et al.  Generating Readable Texts for Readers with Low Basic Skills , 2005, ENLG.

[4]  Emer Gilmartin,et al.  Multimodal conversational interaction with a humanoid robot , 2012, 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom).

[5]  Masafumi Nishida,et al.  Multimodal Corpus of Multi-party Conversations in Second Language , 2012, LREC.

[6]  Graham Wilcock.  WikiTalk: A Spoken Wikipedia-based Open-Domain Knowledge Access System , 2012, Coling 2012.

[7]  Hiroki Yamazaki,et al.  Creation of Learner Corpus and Its Application to Speech Recognition , 2008, LREC.

[8]  Masafumi Nishida,et al.  Multimodal corpus of multiparty conversations in L1 and L2 languages and findings obtained from it , 2015, Lang. Resour. Evaluation.

[9]  Sophie Rosset,et al.  Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice , 2013 .

[10]  Michelene T. H. Chi,et al.  Construction of shared knowledge during collaborative learning , 1997, CSCL.

[11]  Masafumi Nishida,et al.  Sentence Classification with Grammatical Errors and Those Out of Scope of Grammar Assumption for Dialogue-Based CALL Systems , 2012, TSD.

[12]  Nick Campbell,et al.  Investigating the use of Non-verbal Cues in Human-Robot Interaction with a Nao robot , 2012, 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom).

[13]  Tatsuya Kawahara,et al.  Speech-Based Interactive Information Guidance System using Question-Answering Technique , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[14]  Kristiina Jokinen,et al.  Situated Interaction in a Multilingual Spoken Information Access Framework , 2016 .

[15]  Ian H. Witten,et al.  Learning to link with wikipedia , 2008, CIKM '08.

[16]  Sandra Katz,et al.  Is a Dialogue-Based Tutoring System that Emulates Helpful Co-constructed Relations During Human Tutoring Effective? , 2015, AIED.

[17]  Kristiina Jokinen,et al.  Constructive Interaction for Talking about Interesting Topics , 2012, LREC.

[18]  Peter Baranyi,et al.  A unified terminology for the structure and semantics of CogInfoCom channels , 2012 .

[19]  Tsuneo Kato,et al.  Joining-in-type Humanoid Robot Assisted Language Learning System , 2016, LREC.

[20]  Kristiina Jokinen,et al.  Constructive Dialogue Modelling - Speech Interaction and Rational Agents , 2009, Wiley series in agent technology.

[21]  Kristiina Jokinen,et al.  Integration of gestures and speech in human-robot interaction , 2012, 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom).

[22]  Barbara Di Eugenio,et al.  Knowledge Co-construction and Initiative in Peer Learning Interactions , 2009, AIED.

[23]  Kristiina Jokinen,et al.  Multimodal Open-Domain Conversations with the Nao Robot , 2012, Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice.

[24]  Kristiina Jokinen,et al.  Adding Speech to a Robotics Simulator , 2011 .

[25]  Ian H. Witten,et al.  An open-source toolkit for mining Wikipedia , 2013, Artif. Intell..

[26]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[27]  Dimitra Anastasiou,et al.  Evaluation of WikiTalk - User Studies of Human-Robot Interaction , 2013, HCI.

[28]  Kristiina Jokinen,et al.  Multilingual WikiTalk: Wikipedia-based talking robots that switch languages , 2015, SIGDIAL Conference.

[29]  Rada Mihalcea,et al.  Wikify!: linking documents to encyclopedic knowledge , 2007, CIKM '07.