Proposal of a Model to Determine the Attention Target for an Agent in Group Discussion with Non-verbal Features

In recent years, companies are seeking for communication skill from their employers. More and more companies adopt group discussions in employer recruitment to evaluate the ap- plicants' communication skill. However, the opportunity to improve communication skill in group discussion is limited due to the lack of partners. In order to solve this issue, our ongoing project is aiming to build a virtual agent or a robot that can participate group discussion, so that its users can re- peatedly practice group discussion with it. In this paper, we propose the models in directing the agent's attention toward the other participants in three situations:when the agent is speaking, when the agent is listening, and when no partic- ipant is speaking. First, we gathered a data corpus of the discussion of 10 four-people groups. We then use low-level non-verbal features including attention of other participant, voice prosody, head movements, and speech turn extracted in the 10-hour corpus to train support vector machine models to determine the agent's attention on the other participants, or the material. The performance of the detection models in F-measure range between 0.4 and 0.6.

[1]  Massimo Zancanaro,et al.  Overt or subtle? Supporting group conversations with automatically targeted directives , 2014, IUI.

[2]  Jordi Vitrià,et al.  You are fired! Nonverbal role analysis in competitive meetings , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  David R. Traum,et al.  Issues in Multiparty Dialogues , 2003, Workshop on Agent Communication Languages.

[4]  Hedda Lausberg,et al.  Methods in Gesture Research: , 2009 .

[5]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[6]  Takanori Maesako,et al.  HyperMirror: toward pleasant-to-use video mediated communication system , 1998, CSCW '98.

[7]  Anton Nijholt,et al.  Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes , 2001, CHI.

[8]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[9]  Tanja Schultz,et al.  Identifying the addressee in human-human-robot interactions based on head pose and speech , 2004, ICMI '04.

[10]  Jean-Marc Odobez,et al.  Training on the job: behavioral analysis of job interviews in hospitality , 2016, ICMI.

[11]  Hung-Hsuan Huang,et al.  Making virtual conversational agent aware of the addressee of users' utterances in multi-user conversation using nonverbal information , 2011, ICMI '11.

[12]  Catherine Pelachaud,et al.  Mining a multimodal corpus for non-verbal behavior sequences conveying attitudes , 2014, LREC.

[13]  Naoki Mukawa,et al.  Video cut editing rule based on participants' gaze in multiparty conversation , 2003, MULTIMEDIA '03.

[14]  Hung-Hsuan Huang,et al.  Predicting Influential Statements in Group Discussions using Speech and Head Motion Information , 2014, ICMI.

[15]  Jill R. Kickul,et al.  Emergent Leadership Behaviors: The Function of Personality and Cognitive Ability in Determining Teamwork Performance and KSAs , 2000 .

[16]  Brian Scassellati,et al.  Emotional Storytelling in the Classroom: Individual versus Group Interaction between Children and Robots , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[17]  Hung-Hsuan Huang,et al.  Estimating communication skills using dialogue acts and nonverbal features in multiple discussion datasets , 2016, ICMI.

[18]  Daniel Gatica-Perez,et al.  One of a kind: inferring personality impressions in meetings , 2013, ICMI '13.

[19]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[20]  Scott E. Hudson,et al.  Towards Robot Autonomy in Group Conversations: Understanding the Effects of Body Orientation and Gaze , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[21]  Catherine Pelachaud,et al.  Expressing social attitudes in virtual agents for social coaching , 2014, AAMAS.

[22]  A. Kendon Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[23]  Patrick Gebhard,et al.  A Job Interview Simulation: Social Cue-Based Interaction with a Virtual Character , 2013, 2013 International Conference on Social Computing.