Starting a Conversation with Strangers in Virtual Reykjavik: Explicit Announcement of Presence

Virtual Reykjavik is an Icelandic language and culture training application for foreigners learning Icelandic. In this video game-like environment, the user is asked to solve given tasks in the game and in order to complete them he/she must interact with the characters, e.g. by conversing with them on contextspecific topics. To make this a reality, a model for how natural conversations start in a specific situations has been developed, based on data from that same situation in real life: a stranger asking another stranger for directions to a particular place in downtown Reykjavik. This involved defining a multimodal annotation scheme, outlining the communicative functions and behaviors associated with them. However, current annotation schemes lacked the appropriate function for this specific case, which lead us to finding and proposing an appropriate communicative function – the Explicit Announcement of Presence. A study was conducted to explore and better understand how conversation is initiated in first encounters between people who do not know each other. Human-to-human conversations were analyzed for the purpose of modelling a realistic conversation between human users and virtual agents. Results from the study have lead to the inclusion of the communicative function in the human-to-agent conversation system. By playing the game the learners will be exposed to situations that they may encounter in real life, and therefore the interaction is based on real life data, rather than textbook examples. We believe that this application will help bridge the gap from the class room to the real world, preparing learners to initiate conversations with real Icelandic speakers.

[1]  Peter Wittenburg,et al.  Annotation by Category: ELAN and ISO DCR , 2008, LREC.

[2]  Asli Ozyurek,et al.  Annotation of sign and gesture cross-linguistically , 2008, LREC 2008.

[3]  Bente Tobiesen Meyer Designing serious games for foreign language education in a global perspective , 2009 .

[4]  H. Vilhjálmsson,et al.  When Strangers Meet Collective Construction of Procedural Conversation in Embodied Conversational Agents , 2015 .

[5]  Timothy W. Bickmore,et al.  First Impressions in Human--Agent Virtual Encounters , 2016, ACM Trans. Comput. Hum. Interact..

[6]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[7]  Costanza Navarretta,et al.  The MUMIN multimodal coding scheme , 2005 .

[8]  Hussain Hafiz Sajjad,et al.  Task Based Language Learning and Teaching , 2015 .

[9]  Nicole C. Krämer,et al.  Embodied Conversational Agents , 2009 .

[10]  A. Kendon Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[11]  Stacy Marsella,et al.  Serious Games for Language Learning: How Much Game, How Much AI? , 2005, AIED.

[12]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.

[13]  Florian Schiel,et al.  The SmartKom Multimodal Corpus at BAS , 2002, LREC.

[14]  EmoTV 1 : Annotation of Real-life Emotions for the Specification of Multimodal Affective Interfaces , 2005 .

[15]  William Littlewood,et al.  Communicative Language Teaching: An Introduction , 1981 .

[16]  Costanza Navarretta,et al.  The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena , 2007, Lang. Resour. Evaluation.

[17]  Jón Guðnason,et al.  Almannarómur: an open icelandic speech corpus , 2012, SLTU.

[18]  Ágnes Abuczki,et al.  An overview of multimodal corpora, annotation tools and schemes , 2013 .

[19]  Stefan Kopp,et al.  The Next Step towards a Function Markup Language , 2008, IVA.

[20]  Dirk Heylen,et al.  Representing Communicative Functions in SAIBA with a Unified Function Markup Language , 2014, IVA.

[21]  付伶俐 打磨Using Language,倡导新理念 , 2014 .

[22]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.

[23]  I. Coyne Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? , 1997, Journal of advanced nursing.